Computational rationality: A converging paradigm for intelligence in brains, minds, and machines
Abstract
After growing up together, and mostly growing apart in the second half of the 20th century, the fields of artificial intelligence (AI), cognitive science, and neuroscience are reconverging on a shared view of the computational foundations of intelligence that promotes valuable cross-disciplinary exchanges on questions, methods, and results. We chart advances over the past several decades that address challenges of perception and action under uncertainty through the lens of computation. Advances include the development of representations and inferential procedures for large-scale probabilistic inference and machinery for enabling reflection and decisions about tradeoffs in effort, precision, and timeliness of computations. These tools are deployed toward the goal of computational rationality: identifying decisions with highest expected utility, while taking into consideration the costs of computation in complex real-world problems in which most relevant calculations can only be approximated. We highlight key concepts with examples that show the potential for interchange between computer science, cognitive science, and neuroscience.
Get full access to this article
View all available purchase options and get full access to this article.
Already a Subscriber?Sign In
References and Notes
1
D. Koller, N. Friedman, Probabilistic Graphical Models: Principles and Techniques (MIT Press, Cambridge, MA, 2009).
2
J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (Morgan Kaufmann Publishers, Los Altos, CA, 1988).
3
S. Russell, P. Norvig, Artificial Intelligence: A Modern Approach (Pearson, Upper Saddle River, NJ, 2009).
4
J. von Neumann, O. Morgenstern, Theory of Games and Economic Behavior (Princeton Univ. Press, Princeton, NJ, 1947).
5
Tenenbaum J. B., Kemp C., Griffiths T. L., Goodman N. D., How to grow a mind: Statistics, structure, and abstraction. Science 331, 1279–1285 (2011). 10.1126/science.1192788
6
Turing A. M., On computable numbers, with an application to the Entscheidungsproblem. Proc. Lond. Math. Soc. 2, 230–265 (1936).
7
Turing A. M., Computing machinery and intelligence. Mind 59, 433–460 (1950). 10.1093/mind/LIX.236.433
8
J. von Neumann, The Computer and the Brain (Yale Univ. Press, New Haven, CT, 1958).
9
H. A. Simon, Models of Man (Wiley, New York, 1957).
10
Good I. J., Rational decisions. J. R. Stat. Soc. B 14, 107–114 (1952).
11
E. Horvitz, Reasoning about beliefs and actions under computational resource constraints. in Proceedings of the 3rd International Conference on Uncertainty in Artificial Intelligence (Mountain View, CA, July 1987), pp. 429–444 (1987).
12
Russell S., Wefald E., Principles of metareasoning. Artif. Intell. 49, 361–395 (1991).
13
E. Horvitz, G. Cooper, D. Heckerman, Reflection and action under scarce resources: Theoretical principles and empirical study. in Proceedings of IJCAI, January 1989, pp. 1121–1127 (1989).
14
E. Horvitz, G. Rutledge, Time-dependent utility and action under uncertainty. in Proceedings of the 7th International Conference on Uncertainty in Artificial Intelligence (Morgan Kaufmann Publishers, San Francisco, 1991), pp. 151–158.
15
E. Horvitz, Y. Ruan, G. Gomes, H. Kautz, B. Selman, D. M. Chickering, A Bayesian approach to tackling hard computational problems. in Proceedings of 17th Conference on Uncertainty in Artificial Intelligence (Morgan Kaufmann Publishers, San Francisco, 2001), pp. 235–244.
16
Horvitz E., Principles and applications of continual computation. Artif. Intell. 126, 159–196 (2001).
17
Burns E., Ruml W., Do M. B., Heuristic search when time matters. J. Artif. Intell. Res. 47, 697–740 (2013).
18
C. H. Lin, A. Kolobov, A. Kamar, E. Horvitz, Metareasoning for planning under uncertainty. In Proceedings of IJCAI (2015).
19
Dean T., Kaelbling L. P., Kirman J., Nicholson A., Planning under time constraints in stochastic domains. Artif. Intell. 76, 35–74 (1995).
20
N. Hay, S. Russell, D. Tolpin, S. Shimony, Selecting computations: Theory and applications. in Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (2012), pp. 346–355.
21
D. Heckerman, J. S. Breese, E. Horvitz, The compilation of decision models. in Proceedings of the 5th Conference on Uncertainty in Artifical Intelligence, July 1989 (1989), pp. 162–173.
22
Cooper G., The computational complexity of probabilistic inference using bayesian belief networks. Artif. Intell. 42, 393–405 (1990).
23
Peterson C. R., Beach L. R., Man as an intuitive statistician. Psychol. Bull. 68, 29–46 (1967). 10.1037/h0024722
24
Tversky A., Kahneman D., Judgments under uncertainty: Heuristics and biases. Science 185, 1124–1131 (1974). 10.1126/science.185.4157.1124
25
G. Gigerenzer, Rationality for Mortals: How People Cope with Uncertainty (Oxford Univ. Press, Oxford, 2008).
26
J. R. Anderson, The Adaptive Character of Thought (Lawrence Erlbaum, Hillsdale, NJ, 1990).
27
M. Oaksford, N. Chater, Bayesian Rationality (Oxford Univ. Press, Oxford, 2007).
28
Griffiths T. L., Tenenbaum J. B., Structure and strength in causal induction. Cognit. Psychol. 51, 334–384 (2005). 10.1016/j.cogpsych.2005.05.004
29
K. Doya, S. Ishii, A. Pouget, R. P. N. Rao, Eds. The Bayesian Brain: Probabilistic Approaches to Neural Coding (MIT Press, Cambridge, MA, 2007).
30
Griffiths T. L., Lieder F., Goodman N. D., Rational use of cognitive resources: Levels of analysis between the computational and the algorithmic. Top. Cogn. Sci. 7, 217–229 (2015). 10.1111/tops.12142
31
Denison S., Bonawitz E., Gopnik A., Griffiths T. L., Rational variability in children’s causal inferences: The Sampling Hypothesis. Cognition 126, 285–300 (2013). 10.1016/j.cognition.2012.10.010
32
Vul E., Frank M., Alvarez G., Tenenbaum J. B., Explaining human multiple object tracking as resource-constrained approximate inference in a dynamic probabilistic model. Adv. Neural Inf. Process. Syst. 29, 1955–1963 (2009).
33
Gershman S. J., Vul E., Tenenbaum J. B., Multistability and perceptual inference. Neural Comput. 24, 1–24 (2012). 10.1162/NECO_a_00226
34
Sanborn A. N., Griffiths T. L., Navarro D. J., Rational approximations to rational models: Alternative algorithms for category learning. Psychol. Rev. 117, 1144–1167 (2010). 10.1037/a0020511
35
Buesing L., Bill J., Nessler B., Maass W., Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons. PLOS Comput. Biol. 7, e1002211 (2011). 10.1371/journal.pcbi.1002211
36
Isard M., Blake A., CONDENSATION—Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29, 5–28 (1998).
37
Levy R., Reali F., Griffiths T. L., Modeling the effects of memory on human online sentence processing with particle filters. Adv. Neural Inf. Process. Syst. 21, 937–944 (2009).
38
Vul E., Goodman N., Griffiths T. L., Tenenbaum J. B., One and done? Optimal decisions from very few samples. Cogn. Sci. 38, 599–637 (2014). 10.1111/cogs.12101
39
Kool W., McGuire J. T., Rosen Z. B., Botvinick M. M., Decision making and the avoidance of cognitive demand. J. Exp. Psychol. Gen. 139, 665–682 (2010). 10.1037/a0020198
40
Kool W., Botvinick M., A labor/leisure tradeoff in cognitive control. J. Exp. Psychol. Gen. 143, 131–141 (2014). 10.1037/a0031048
41
McGuire J. T., Botvinick M. M., Prefrontal cortex, cognitive control, and the registration of decision costs. Proc. Natl. Acad. Sci. U.S.A. 107, 7922–7926 (2010). 10.1073/pnas.0910662107
42
Lieder F., Plunkett D., Hamrick J. B., Russell S. J., Hay N., Griffiths T. L., Algorithm selection by rational metareasoning as a model of human strategy selection. Adv. Neural Inf. Process. Syst. 27, 2870–2878 (2014).
43
Lewis R. L., Howes A., Singh S., Computational rationality: Linking mechanism and behavior through bounded utility maximization. Top. Cogn. Sci. 6, 279–311 (2014). 10.1111/tops.12086
44
Payne J. W., Bettman J. R., Johnson E. J., Adaptive strategy selection in decision making. J. Exp. Psychol. Learn. Mem. Cogn. 14, 534–552 (1988).
45
Rieskamp J., Otto P. E., SSL: A theory of how people learn to select strategies. J. Exp. Psychol. Gen. 135, 207–236 (2006).
46
F. Lieder, M. Hsu, T. L. Griffiths, The high availability of extreme events serves resource-rational decision- making. in Proc. 36th Ann. Conf. Cognitive Science Society (Austin, TX, 2014).
47
Daw N. D., Niv Y., Dayan P., Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005). 10.1038/nn1560
48
Killcross S., Coutureau E., Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 13, 400–408 (2003). 10.1093/cercor/13.4.400
49
Yin H. H., Knowlton B. J., Balleine B. W., Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004).
50
Daw N. D., Gershman S. J., Seymour B., Dayan P., Dolan R. J., Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011). 10.1016/j.neuron.2011.02.027
51
Keramati M., Dezfouli A., Piray P., Speed/accuracy trade-off between the habitual and the goal-directed processes. PLOS Comput. Biol. 7, e1002055 (2011).
52
Dickinson A., Actions and habits: The development of behavioural and autonomy. Philos. Trans. R. Soc. London B Biol. Sci. 308, 67–78 (1985). 10.1098/rstb.1985.0010
53
Otto A. R., Gershman S. J., Markman A. B., Daw N. D., The curse of planning: Dissecting multiple reinforcement-learning systems by taxing the central executive. Psychol. Sci. 24, 751–761 (2013). 10.1177/0956797612463080
54
Lee S. W., Shimojo S., O’Doherty J. P., Neural computations underlying arbitration between model-based and model-free learning. Neuron 81, 687–699 (2014). 10.1016/j.neuron.2013.11.028
55
Gelly S., Kocsis L., Schoenauer M., Sebag M., Silver D., Szepesvari C., Teytaud O., The grand challenge of computer Go: Monte Carlo tree search and extensions. Commun. ACM 55, 106–113 (2012). 10.1145/2093548.2093574
56
Johnson A., Redish A. D., Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189 (2007).
57
Pfeiffer B. E., Foster D. J., Hippocampal place-cell sequences depict future paths to remembered goals. Nature 497, 74–79 (2013). 10.1038/nature12112
58
Mnih V., Kavukcuoglu K., Silver D., Rusu A. A., Veness J., Bellemare M. G., Graves A., Riedmiller M., Fidjeland A. K., Ostrovski G., Petersen S., Beattie C., Sadik A., Antonoglou I., King H., Kumaran D., Wierstra D., Legg S., Hassabis D., Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). 10.1038/nature14236
59
C. J. Maddison, A. Huang, I. Sutskever, D. Silver, Move evaluation in Go using deep convolutional neural networks. http://arxiv.org/abs/1412.6564 (2014).
60
Guo X., Singh S., Lee H., Lewis R., Wang X., Deep learning for real-time Atari game play using offline Monte-Carlo tree search planning. Adv. Neural Inf. Process. Syst. 27, 3338–3346 (2014).
61
G. M. J.-B. Chaslot, S. Bakkes, I. Szita, P. Spronck, Monte-Carlo tree search: A new framework for game AI. in Proc. Artif. Intell. Interact. Digit. Entertain. Conf. (Stanford, CA, 2008), pp. 216–217.
Information & Authors
Information
Published In
Science
Volume 349 | Issue 6245
17 July 2015
17 July 2015
Copyright
Copyright © 2015, American Association for the Advancement of Science.
Submission history
Published in print: 17 July 2015
Acknowledgments
We are grateful to A. Gershman and the three referees for helpful comments. This research was partly supported by the Center for Brains, Minds and Machines (CBMM), funded by National Science Foundation Science and Technology Center award CCF-1231216.
Authors
Metrics & Citations
Metrics
Article Usage
Altmetrics
Citations
Export citation
Select the format you want to export the citation of this publication.
Cited by
- Computational Rational Engineering and Development: Synergies and Opportunities, Intelligent Systems and Applications, (744-763), (2022).https://doi.org/10.1007/978-3-030-82193-7_50
- Differential prioritization of intramaze cue and boundary information during spatial navigation across the human lifespan, Scientific Reports, 11, 1, (2021).https://doi.org/10.1038/s41598-021-94530-9
- Flexible modulation of sequence generation in the entorhinal–hippocampal system, Nature Neuroscience, 24, 6, (851-862), (2021).https://doi.org/10.1038/s41593-021-00831-7
- Using functional fields to represent accounts of the psychological processes that produce actions, The Handbook of Personality Dynamics and Processes, (643-667), (2021).https://doi.org/10.1016/B978-0-12-813995-0.00024-8
- The asymmetric learning rates of murine exploratory behavior in sparse reward environments, Neural Networks, 143, (218-229), (2021).https://doi.org/10.1016/j.neunet.2021.05.030
- Optimal attentional allocation in the presence of capacity constraints in uncued and cued visual search, Journal of Vision, 21, 5, (3), (2021).https://doi.org/10.1167/jov.21.5.3
- Data, measurement, and causal inferences in machine learning: opportunities and challenges for marketing, Journal of Marketing Theory and Practice, 29, 1, (65-77), (2021).https://doi.org/10.1080/10696679.2020.1860683
- Multi-source brain computing with systematic fusion for smart health, Information Fusion, 75, (150-167), (2021).https://doi.org/10.1016/j.inffus.2021.03.009
- Being Realist about Bayes, and the Predictive Processing Theory of Mind, The British Journal for the Philosophy of Science, 72, 1, (185-220), (2021).https://doi.org/10.1093/bjps/axy059
- The development of machine intelligence in a computational universe, Technology in Society, 65, (101553), (2021).https://doi.org/10.1016/j.techsoc.2021.101553
- See more
Loading...
View Options
Get Access
Log in to view the full text
AAAS login provides access to Science for AAAS Members, and access to other journals in the Science family to users who have purchased individual subscriptions.
- Become a AAAS Member
- Activate your AAAS ID
- Purchase Access to Other Journals in the Science Family
- Account Help
Log in via OpenAthens.
Log in via Shibboleth.
More options
Purchase digital access to this article
Download and print this article for your personal scholarly, research, and educational use.
Buy a single issue of Science for just $15 USD.
View options
PDF format
Download this article as a PDF file
Download PDF