Advertisement

Machines learn what people know implicitly

AlphaGo has demonstrated that a machine can learn how to do things that people spend many years of concentrated study learning, and it can rapidly learn how to do them better than any human can. Caliskan et al. now show that machines can learn word associations from written texts and that these associations mirror those learned by humans, as measured by the Implicit Association Test (IAT) (see the Perspective by Greenwald). Why does this matter? Because the IAT has predictive value in uncovering the association between concepts, such as pleasantness and flowers or unpleasantness and insects. It can also tease out attitudes and beliefs—for example, associations between female names and family or male names and career. Such biases may not be expressed explicitly, yet they can prove influential in behavior.
Science, this issue p. 183; see also p. 133

Abstract

Machine learning is a means to derive artificial intelligence by discovering patterns in existing data. Here, we show that applying machine learning to ordinary human language results in human-like semantic biases. We replicated a spectrum of known biases, as measured by the Implicit Association Test, using a widely used, purely statistical machine-learning model trained on a standard corpus of text from the World Wide Web. Our results indicate that text corpora contain recoverable and accurate imprints of our historic biases, whether morally neutral as toward insects or flowers, problematic as toward race or gender, or even simply veridical, reflecting the status quo distribution of gender with respect to careers or first names. Our methods hold promise for identifying and addressing sources of bias in culture, including technology.

Get full access to this article

View all available purchase options and get full access to this article.

Supplementary Material

Summary

Materials and Methods
Supplementary Text
Table S1
References

Resources

File (caliskan-sm.pdf)

References and Notes

1
M. Stubbs, Text and Corpus Analysis: Computer-Assisted Studies of Language and Culture (Blackwell, Oxford, 1996).
2
J. A. Bullinaria, J. P. Levy, Extracting semantic representations from word co-occurrence statistics: A computational study. Behav. Res. Methods 39, 510–526 (2007).
3
T. Mikolov, J. Dean, Distributed representations of words and phrases and their compositionality. Adv. Neural Inf. Process. Syst. 2013, 3111–3119 (2013).
4
C. M. Bishop, Pattern Recognition and Machine Learning (Springer, London, 2006).
5
A. G. Greenwald, D. E. McGhee, J. L. Schwartz, Measuring individual differences in implicit cognition: The implicit association test. J. Pers. Soc. Psychol. 74, 1464–1480 (1998).
6
T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, A. T. Kalai, Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Adv. Neural Inf. Process. Syst. 2016, 4349–4357 (2016).
7
M. Bertrand, S. Mullainathan, Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94, 991–1013 (2004).
8
M. Bertrand, D. Chugh, S. Mullainathan, Implicit discrimination. Am. Econ. Rev. 95, 94–98 (2005).
9
B. A. Nosek, M. Banaji, A. G. Greenwald, Harvesting implicit group attitudes and beliefs from a demonstration web site. Group Dyn. 6, 101–115 (2002).
10
B. A. Nosek, M. R. Banaji, A. G. Greenwald, Math = male, me = female, therefore math ≠ me. J. Pers. Soc. Psychol. 83, 44–59 (2002).
11
B. A. Nosek, F. L. Smyth, N. Sriram, N. M. Lindner, T. Devos, A. Ayala, Y. Bar-Anan, R. Bergh, H. Cai, K. Gonsalkorale, S. Kesebir, N. Maliszewski, F. Neto, E. Olli, J. Park, K. Schnabel, K. Shiomura, B. T. Tulbure, R. W. Wiers, M. Somogyi, N. Akrami, B. Ekehammar, M. Vianello, M. R. Banaji, A. G. Greenwald, National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proc. Natl. Acad. Sci. U.S.A. 106, 10593–10597 (2009).
12
P. D. Turney, P. Pantel, From frequency to meaning: Vector space models of semantics. J. Artif. Intell. Res. 37, 141 (2010).
13
J. Pennington, R. Socher, C. D. Manning, GloVe: Global Vectors for Word Representation. EMNLP 14, 1532–1543 (2014).
14
T. MacFarlane, Extracting semantics from the Enron corpus, University of Bath, Department of Computer Science Technical Report Series; CSBU-2013-08; http://opus.bath.ac.uk/37916/ (2013).
15
W. Lowe, S. McDonald, The direct route: Mediated priming in semantic space, Proceedings of the Twenty-Second Annual Conference of the Cognitive Science Society (LEA, 2000), pp. 806–811.
16
M. Sahlgren, The distributional hypothesis. Ital. J. Linguist. 20, 33 (2008).
17
G. Lupyan, The centrality of language in human cognition. Lang. Learn. 66, 516–553 (2016).
18
S. Barocas, A. D. Selbst, Big data’s disparate impact. Calif. Law Rev. 104, 2477899 (2014).
19
C. Dwork, M. Hardt, T. Pitassi, O. Reingold, R. Zemel, Fairness through awareness, Proceedings of the 3rd Innovations in Theoretical Computer Science Conference (ACM, 2012), pp. 214–226.
20
M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, S. Venkatasubramanian, Certifying and removing disparate impact, Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM, 2015), pp. 259–268.
21
K. R. Thórisson, Integrated A.I. systems. Minds Mach. 17, 11–25 (2007).
22
M. Hanheide, M. Göbelbecker, G. S. Horn, A. Pronobis, K. Sjöö, A. Aydemir, P. Jensfelt, C. Gretton, R. Dearden, M. Janicek, H. Zender, G.-J. Kruijff, N. Hawes, J. L. Wyatt, Robot task planning and explanation in open and uncertain worlds. Artif. Intell. 2015, j.artint.2015.08.008 (2015).
23
L. L. Monteith, J. W. Pettit, Implicit and explicit stigmatizing attitudes and stereotypes about depression. J. Soc. Clin. Psychol. 30, 484–505 (2011).

(0)eLetters

eLetters is an online forum for ongoing peer review. Submission of eLetters are open to all. eLetters are not edited, proofread, or indexed. Please read our Terms of Service before submitting your own eLetter.

Log In to Submit a Response

No eLetters have been published for this article yet.

Information & Authors

Information

Published In

Science
Volume 356 | Issue 6334
14 April 2017

Submission history

Received: 17 November 2016
Accepted: 9 March 2017
Published in print: 14 April 2017

Permissions

Request permissions for this article.

Acknowledgments

We are grateful to W. Lowe for substantial assistance in the design of our significance tests; T. Macfarlane for pilot research as a part of his undergraduate dissertation; and S. Barocas, M. Brundage, K. Crawford, C. Lai, and M. Salganik for extremely useful comments on a draft of this paper. We have archived the code and data on Harvard Dataverse (doi: 10.7910/DVN/DX4VWP).

Authors

Affiliations

Center for Information Technology Policy, Princeton University, Princeton, NJ, USA.
Center for Information Technology Policy, Princeton University, Princeton, NJ, USA.
Department of Computer Science, University of Bath, Bath BA2 7AY, UK.
Center for Information Technology Policy, Princeton University, Princeton, NJ, USA.

Notes

*
Corresponding author. Email: [email protected] (A.C.); [email protected] (J.J.B.); [email protected] (A.N.)

Metrics & Citations

Metrics

Article Usage
Altmetrics

Citations

Export citation

Select the format you want to export the citation of this publication.

Cited by

  1. Based on billions of words on the internet, people = men, Science Advances, 8, 13, (2022)./doi/10.1126/sciadv.abm2463
    Abstract
  2. Neural embeddings of scholarly periodicals reveal complex disciplinary organizations, Science Advances, 7, 17, (2021)./doi/10.1126/sciadv.abb9004
    Abstract
  3. Artificial intelligence in small bowel capsule endoscopy ‐ current status, challenges and future promise, Journal of Gastroenterology and Hepatology, 36, 1, (12-19), (2021).https://doi.org/10.1111/jgh.15341
    Crossref
  4. A Bayesian LSTM model to evaluate the effects of air pollution control regulations in Beijing, China, Environmental Science & Policy, 115, (26-34), (2021).https://doi.org/10.1016/j.envsci.2020.10.004
    Crossref
  5. A literature review on artificial intelligence and ethics in online learning, Intelligent Systems and Learning Data Analytics in Online Education, (111-131), (2021).https://doi.org/10.1016/B978-0-12-823410-5.00006-1
    Crossref
  6. ‘Toward a Global Social Contract for Trade’ - a Rawlsian approach to Blockchain Systems Design and Responsible Trade Facilitation in the New Bretton Woods era, Journal of Responsible Technology, 6, (100011), (2021).https://doi.org/10.1016/j.jrt.2021.100011
    Crossref
  7. Explaining Crossmodal Correspondences Between Colours and Tastes, i-Perception, 12, 3, (204166952110182), (2021).https://doi.org/10.1177/20416695211018223
    Crossref
  8. A Generic and Model-Agnostic Exemplar Synthetization Framework for Explainable AI, Machine Learning and Knowledge Discovery in Databases, (190-205), (2021).https://doi.org/10.1007/978-3-030-67661-2_12
    Crossref
  9. Cultural cartography with word embeddings, Poetics, (101567), (2021).https://doi.org/10.1016/j.poetic.2021.101567
    Crossref
  10. How search engines may help reduce drug-related suicides, Drug and Alcohol Dependence, 226, (108874), (2021).https://doi.org/10.1016/j.drugalcdep.2021.108874
    Crossref
  11. See more
Loading...

View Options

Check Access

Log in to view the full text

AAAS ID LOGIN

AAAS login provides access to Science for AAAS Members, and access to other journals in the Science family to users who have purchased individual subscriptions.

Log in via OpenAthens.
Log in via Shibboleth.
More options

Register for free to read this article

As a service to the community, this article is available for free. Login or register for free to read this article.

Purchase this issue in print

Buy a single issue of Science for just $15 USD.

View options

PDF format

Download this article as a PDF file

Download PDF

Full Text

FULL TEXT

Media

Figures

Multimedia

Tables

Share

Share

Share article link

Share on social media