Advertisement
GET OUR E-ALERTS
Review
SENSORS

Electronic skins and machine learning for intelligent soft robots

Benjamin Shih https://orcid.org/0000-0001-8463-2694, Dylan Shah https://orcid.org/0000-0003-3943-6170, Jinxing Li, Thomas G. Thuruthel, Yong-Lae Park https://orcid.org/0000-0002-2491-2114, Fumiya Iida https://orcid.org/0000-0001-9246-7190, Zhenan Bao https://orcid.org/0000-0002-0972-1715, Rebecca Kramer-Bottiglio https://orcid.org/0000-0003-2324-8124, and Michael T. Tolley https://orcid.org/0000-0001-7821-7777 [email protected]
Science Robotics • 15 Apr 2020 • Vol 5, Issue 41 • DOI: 10.1126/scirobotics.aaz9239
get access
Related Research Article
Prosthesis with neuromorphic multilayered e-dermis perceives touch and pain

Abstract

Soft robots have garnered interest for real-world applications because of their intrinsic safety embedded at the material level. These robots use deformable materials capable of shape and behavioral changes and allow conformable physical contact for manipulation. Yet, with the introduction of soft and stretchable materials to robotic systems comes a myriad of challenges for sensor integration, including multimodal sensing capable of stretching, embedment of high-resolution but large-area sensor arrays, and sensor fusion with an increasing volume of data. This Review explores the emerging confluence of e-skins and machine learning, with a focus on how roboticists can combine recent developments from the two fields to build autonomous, deployable soft robots, integrated with capabilities for informative touch and proprioception to stand up to the challenges of real-world environments.
Get full access to this article

View all available purchase options and get full access to this article.

Get Access
Already a Subscriber?

REFERENCES AND NOTES

1
D. Rus, M. T. Tolley, Design, fabrication and control of soft robots. Nature 521, 467–475 (2015).
2
S. Wagner, S. P. Lacour, J. Jones, I. H. Pai-hui, J. C. Sturm, T. Li, Z. Suo, Electronic skin: Architecture and components. Phys. E. 25, 326–334 (2004).
3
J. Case, M. Yuen, M. Mohammed, R. Kramer, Sensor skins: An overview, in Stretchable Bioelectronics for Medical Devices and Systems (Springer, 2016), pp. 173–191.
4
M. Amjadi, K.-U. Kyung, I. Park, M. Sitti, Stretchable, skin-mountable, and wearable strain sensors and their potential applications: A review. Adv. Funct. Mater. 26, 1678–1698 (2016).
5
N. Yogeswaran, W. Dang, W. T. Navaraj, D. Shakthivel, S. Khan, E. O. Polat, S. Gupta, H. Heidari, M. Kaboli, L. Lorenzelli, G. Cheng, R. Dahiya, New materials and advances in making electronic skin for interactive robots. Adv. Robot. 29, no. 21, 1359–1373 (2015).
6
R. Dahiya, N. Yogeswaran, F. Liu, L. Manjakkal, E. Burdet, V. Hayward, H. Jörntell, Large-area soft e-skin: The challenges beyond sensor designs. Proc. IEEE 107, 2016–2033 (2019).
7
H. Wang, M. Totaro, L. Beccai, Toward perceptive soft robots: Progress and challenges. Adv. Sci. 5, 1800541 (2018).
8
K. Chin, T. Hellebrekers, C. Majidi, Machine learning for soft robotic sensing and control. Adv. Intell. Syst. 2020, 1900171 (2020).
9
D. S. Shah, M. C. Yuen, L. G. Tilton, E. J. Yang, R. Kramer-Bottiglio, Morphing robots using robotic skins that sculpt clay. IEEE Robot. Autom. Lett. 4, 2204–2211 (2019).
10
B. Shih, D. Drotman, C. Christianson, Z. Huo, R. White, H. I. Christensen, M. T. Tolley, Custom soft robotic gripper sensor skins for haptic object visualization, in 2017 IEEE/RSJ IROS (IEEE, 2017), pp. 494–501.
11
T. Arnold, M. Scheutz, The tactile ethics of soft robotics: Designing wisely for human–robot interaction. Soft Robot. 4, 81–87 (2017).
12
S. Sundaram, P. Kellnhofer, Y. Li, J.-Y. Zhu, A. Torralba, W. Matusik, Learning the signatures of the human grasp using a scalable tactile glove. Nature 569, 698–702 (2019).
13
J. Viventi, D.-H. Kim, L. Vigeland, E. S. Frechette, J. A. Blanco, Y.-S. Kim, A. E. Avrin, V. R. Tiruvadi, S.-W. Hwang, A. C. Vanleer, D. F. Wulsin, K. Davis, C. E. Gelber, L. Palmer, J. Van der Spiegel, J. Wu, J. Xiao, Y. Huang, D. Contreras, J. A. Rogers, B. Litt, Flexible, foldable, actively multiplexed, high-density electrode array for mapping brain activity in vivo. Nat. Neurosci. 14, 1599–1605 (2011).
14
D.-H. Kim, J.-H. Ahn, W. M. Choi, H.-S. Kim, T.-H. Kim, J. Song, Y. Y. Huang, Z. Liu, C. Lu, J. A. Rogers, Stretchable and foldable silicon integrated circuits. Science 320, 507–511 (2008).
15
T. Someya, Y. Kato, T. Sekitani, S. Iba, Y. Noguchi, Y. Murase, H. Kawaguchi, T. Sakurai, Conformable, flexible, large-area networks of pressure and thermal sensors with organic transistor active matrixes. Proc. Natl. Acad. Sci. U.S.A. 102, 12321–12325 (2005).
16
T. Sekitani, Y. Noguchi, K. Hata, T. Fukushima, T. Aida, T. Someya, A rubberlike stretchable active matrix using elastic conductors. Science 321, 1468–1472 (2008).
17
X. Ren, K. Pei, B. Peng, Z. Zhang, Z. Wang, X. Wang, P. K. Chan, A low-operating-power and flexible active-matrix organic-transistor temperature-sensor array. Adv. Mater. 28, 4832–4838 (2016).
18
S. Wang, J. Xu, W. Wang, G.-J. N. Wang, R. Rastak, F. Molina-Lopez, J. W. Chung, S. Niu, V. R. Feig, J. Lopez, T. Lei, S.-K. Kwon, Y. Kim, A. M. Foudeh, A. Ehrlich, A. Gasperini, Y. Yun, B. Murmann, J. B.-H. Tok, Z. Bao, Skin electronics from scalable fabrication of an intrinsically stretchable transistor array. Nature 555, 83–88 (2018).
19
Z. Huang, Y. Hao, Y. Li, H. Hu, C. Wang, A. Nomoto, T. Pan, Y. Gu, Y. Chen, T. Zhang, W. Li, Y. Lei, N. H. Kim, C. Wang, L. Zhang, J. W. Ward, A. Maralani, X. Li, M. F. Durstock, A. Pisano, Y. Lin, S. Xu, Three-dimensional integrated stretchable electronics. Nat. Electronics, 473–480 (2018).
20
B. Shih, C. Christianson, K. Gillespie, S. Lee, J. Mayeda, Z. Huo, M. T. Tolley, Design considerations for 3D printed, soft, multimaterial resistive sensors for soft robotics. Front. Robot. AI 6, 30 (2019).
21
B. C.-K. Tee, A. Chortos, A. Berndt, A. K. Nguyen, A. Tom, A. McGuire, Z. C. Lin, K. Tien, W.-G. Bae, H. Wang, P. Mei, H.-H. Chou, B. Cui, K. Deisseroth, T. N. Ng, Z. Bao, A skin-inspired organic digital mechanoreceptor. Science 350, 313–316 (2015).
22
A. Chortos, J. Liu, Z. Bao, Pursuing prosthetic electronic skin. Nat. Mater. 15, 937–950 (2016).
23
Y. Kim, A. Chortos, W. Xu, Y. Liu, J. Y. Oh, D. Son, J. Kang, A. M. Foudeh, C. Zhu, Y. Lee, S. Niu, J. Liu, R. Pfattner, Z. Bao, T.-W. Lee, A bioinspired flexible organic artificial afferent nerve. Science 360, 998–1003 (2018).
24
W. W. Lee, Y. J. Tan, H. Yao, S. Li, H. H. See, M. Hon, K. A. Ng, B. Xiong, J. S. Ho, C. Tee, A neuro-inspired artificial peripheral nervous system for scalable electronic skins. Sci. Robot. 4, eaax2198 (2019).
25
P. A. Xu, A. Mishra, H. Bai, C. Aubin, L. Zullo, R. F. Shepherd, Optical lace for synthetic afferent neural networks. Sci. Robot. 4, eaaw6304 (2019).
26
H. A. Sonar, M. C. Yuen, R. Kramer-Bottiglio, J. Paik, An any-resolution pressure localization scheme using a soft capacitive sensor skin, in 2018 IEEE International Conference on Soft Robotics (RoboSoft) (IEEE, 2018), pp. 170–175.
27
Y.-L. Park, B.-R. Chen, R. J. Wood, Design and fabrication of soft artificial skin using embedded microchannels and liquid conductors. IEEE Sensors J. 12, 2711–2718 (2012).
28
T. Hellebrekers, O. Kroemer, C. Majidi, Soft magnetic skin for continuous deformation sensing. Adv. Intell. Syst. 1, 1900025 (2019).
29
D. H. Kim, N. Lu, R. Ma, Y. S. Kim, R. H. Kim, S. Wang, J. Wu, S. M. Won, H. Tao, A. Islam, K. J. Yu, T. I. Kim, R. Chowdhury, M. Ying, L. Xu, M. Li, H. J. Chung, H. Keum, M. McCormick, P. Liu, Y. W. Zhang, F. G. Omenetto, Y. Huang, T. Coleman, J. A. Rogers, Epidermal electronics. Science 333, 838–843 (2011).
30
M. Kaltenbrunner, T. Sekitani, J. Reeder, T. Yokota, K. Kuribara, T. Tokuhara, M. Drack, R. Schwödiauer, I. Graz, S. Bauer-Gogonea, S. Bauer, T. Someya, An ultra-lightweight design for imperceptible plastic electronics. Nature 499, 458–463 (2013).
31
R. F. Shepherd, F. Ilievski, W. Choi, S. A. Morin, A. A. Stokes, A. D. Mazzeo, X. Chen, M. Wang, G. M. Whitesides, Multigait soft robot. Proc. Natl. Acad. Sci. U.S.A. 108, 20400–20403 (2011).
32
D. Drotman, S. Jadhav, M. Karimi, P. Dezonia, M. T. Tolley, 3D printed soft actuators for a legged robot capable of navigating unstructured terrain, in 2017 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2017), pp. 5532–5538.
33
E. W. Hawkes, L. H. Blumenschein, J. D. Greer, A. M. Okamura, A soft robot that navigates its environment through growth. Sci. Robot. 2, eaan3028 (2017).
34
R. K. Katzschmann, J. DelPreto, R. MacCurdy, D. Rus, Exploration of underwater life with an acoustically controlled soft robotic fish. Sci. Robot. 3, eaar3449 (2018).
35
M. Wehner, R. L. Truby, D. J. Fitzgerald, B. Mosadegh, G. M. Whitesides, J. A. Lewis, R. J. Wood, An integrated design and fabrication strategy for entirely soft, autonomous robots. Nature 536, 451–455 (2016).
36
G. Soter, A. Conn, H. Hauser, J. Rossiter, Bodily aware soft robots: Integration of proprioceptive and exteroceptive sensors, in 2018 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2018), pp. 2448–2453.
37
L. Chin, J. Lipton, M. C. Yuen, R. Kramer-Bottiglio, D. Rus, Automated recycling separation enabled by soft robotic material classification, in 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft) (IEEE, 2019), pp. 102–107.
38
R. Deimel, O. Brock, A novel type of compliant and underactuated robotic hand for dexterous grasping. Int. J. Robot. Res. 35, 161–185 (2016).
39
T. G. Thuruthel, B. Shih, C. Laschi, M. T. Tolley, Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4, eaav1488 (2019).
40
L. Scimeca, P. Maiolino, D. Cardin-Catalan, A. P. del Pobil, A. Morales, and F. Iida, Non-destructive robotic assessment of mango ripeness via multi-point soft haptics, in 2019 International Conference on Robotics and Automation (ICRA) (IEEE, 2019), pp. 1821–1826.
41
R. A. Bilodeau, E. L. White, R. K. Kramer, Monolithic fabrication of sensors and actuators in a soft robotic gripper, in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 2324–2329.
42
N. Farrow, N. Correll, A soft pneumatic actuator that can sense grasp and touch, in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), pp. 2317–2323.
43
H. Zhao, K. OBrien, S. Li, R. F. Shepherd, Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides. Sci. Robot. 1, eaai7529 (2016).
44
K. B. Justus, T. Hellebrekers, D. D. Lewis, A. Wood, C. Ingham, C. Majidi, P. R. LeDuc, C. Tan, A biosensing soft robot: Autonomous parsing of chemical signals through integrated organic and inorganic interfaces. Sci. Robot. 4, eaax0765 (2019).
45
A. Alspach, J. Kim, K. Yamane, Design of a soft upper body robot for physical human-robot interaction, in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) (IEEE, 2015), pp. 290–296.
46
P. Mittendorfer, E. Dean, G. Cheng, 3D spatial self-organization of a modular artificial skin, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, 14 to 18 September 2014 (IEEE, 2014), pp. 3969–3974.
47
J. W. Booth, D. Shah, J. C. Case, E. L. White, M. C. Yuen, O. Cyr-Choiniere, R. Kramer-Bottiglio, Omniskins: Robotic skins that turn inanimate objects into multi- functional robots. Sci. Robot. 3, eaat1853 (2018).
48
P. Mittendorfer, G. Cheng, 3D surface reconstruction for robotic body parts with artificial skins, in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, 7 to 12 October 2012 (IEEE, 2012), pp. 4505–4510.
49
R. K. Kramer, C. Majidi, R. Sahai, R. J. Wood, Soft curvature sensors for joint angle proprioception, in Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on (IEEE, 2011)pp. 1919–1926.
50
J. T. Muth, D. M. Vogt, R. L. Truby, Y. Mengüc, D. B. Kolesky, R. J. Wood, J. A. Lewis, Embedded 3D printing of strain sensors within highly stretchable elastomers. Adv. Mater. 26, 6307–6312 (2014).
51
A. Alspach, J. Kim, K. Yamane, Design and fabrication of a soft robotic hand and arm system, in 2018 IEEE International Conference on Soft Robotics (RoboSoft), 24 to 28 April 2018 (IEEE, 2018), pp. 369–375.
52
J. Jung, M. Park, D. W. Kim, Y.-L. Park, Optically sensorized elastomer air chamber for proprioceptive sensing of soft pneumatic actuators. IEEE Robot. Autom. Lett. 5, 2333–2340 (2020).
53
I. Van Meerbeek, C. De Sa, R. Shepherd, Soft optoelectronic sensory foams with proprioception. Sci. Robot. 3, eaau2489 (2018).
54
G. Schwartz, B. C.-K. Tee, J. Mei, A. L. Appleton, D. H. Kim, H. Wang, Z. Bao, Flexible polymer transistors with high pressure sensitivity for application in electronic skin and health monitoring. Nat. Commun. 4, 1859 (2013).
55
J. Byun, Y. Lee, J. Yoon, B. Lee, E. Oh, S. Chung, T. Lee, K.-J. Cho, J. Kim, Y. Hong, Electronic skins for soft, compact, reversible assembly of wirelessly activated fully soft robots. Sci. Robot. 3, eaas9020 (2018).
56
H. Jeong, L. Wang, T. Ha, R. Mitbander, X. Yang, Z. Dai, S. Qiao, L. Shen, N. Sun, N. Lu, Modular and reconfigurable wireless e-tattoos for personalized sensing. Adv. Mate. Technol. 4, 1900117 (2019).
57
Y. Mengüc, Y.-L. Park, H. Pei, D. Vogt, P. M. Aubin, E. Winchell, L. Fluke, L. Stirling, R. J. Wood, C. J. Walsh, Wearable soft sensing suit for human gait measurement. Int. J. Robot. Res. 33, 1748–1764 (2014).
58
C. M. Boutry, M. Negre, M. Jorda, O. Vardoulis, A. Chortos, O. Khatib, Z. Bao, A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics. Sci. Robot. 3, eaau6914 (2018).
59
P. Piacenza, S. Sherman, M. Ciocarlie, Data-driven super-resolution on a tactile dome. IEEE Robot. Autom. Lett. 3, 1434–1441 (2018).
60
C. Larson, J. Spjut, R. Knepper, R. Shepherd, A deformable interface for human touch recognition using stretchable carbon nanotube dielectric elastomer sensors and deep neural networks. Soft Robot. 6, 611–620 (2019).
61
D. Kim, J. Kwon, B. Jeon, Y.-L. Park, Adaptive calibration of soft sensors using optimal transportation transfer learning for mass production and long-term usage. Adv. Intell. Syst. 2020, 1900178 (2020).
62
D. Kim, J. Kwon, S. Han, Y.-L. Park, S. Jo, Deep full-body motion network (dfm-net) for a soft wearable motion sensing suit. IEEE/ASME Trans. Mechatron. 24, 18453663 (2018).
63
M. Riesenhuber, T. Poggio, Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).
64
Y. Bengio, A. Courville, P. Vincent, Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35, 1798–1828 (2013).
65
P. Mirowski, R. Pascanu, F. Viola, H. Soyer, A. J. Ballard, A. Banino, M. Denil, R. Goroshin, L. Sifre, K. Kavukcuoglu, D. Kumaran, R. Hadsell, Learning to navigate in complex environments. arXiv:1611.03673 (2016).
66
T. Chen, S. Gupta, A. Gupta, Learning exploration policies for navigation. arXiv:1903.01959 (2019).
67
J. Fu, S. Levine, P. Abbeel, One-shot learning of manipulation skills with online dynamics adaptation and neural network priors, in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2016), pp. 4019–4026.
68
A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, S. Levine, Learning complex dexterous manipulation with deep reinforcement learning and demonstrations. arXiv:1709.10087 (2017).
69
OpenAI, M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. M. Grew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, W. Zaremba, Learning dexterous in-hand manipulation. arXiv:1808.00177 (2018).
70
D. P. Bertsekas, Dynamic Programming and Optimal Control (Athena Scientific, 1995), vol. 1.
71
J. Burgner-Kahrs, D. C. Rucker, H. Choset, Continuum robots for medical applications: A survey. IEEE Trans. Robot. 31, 1261–1280 (2015).
72
T. George Thuruthel, Y. Ansari, E. Falotico, C. Laschi, Control strategies for soft robotic manipulators: A survey. Soft Robot. 5, 149–163 (2018).
73
J. Morrow, H.-S. Shin, C. Phillips-Grafflin, S. Jang, J. Torrey, R. Larkins, S. Dang, Y.-L. Park, D. Berenson, Improving soft pneumatic actuator fingers through integration of soft sensors, position and force control, and rigid fingernails. in 2016 IEEE International Conference on Robotics and Automation (ICRA), 16 to 21 May 2016 (IEEE, 2016), pp. 5024–5031.
74
S. Y. Kim, R. Baines, J. Booth, N. Vasios, K. Bertoldi, R. Kramer-Bottiglio, Re-configurable soft body trajectories using unidirectionally stretchable composite laminae. Nat. Commun. 10, 3464 (2019).
75
N. Cheney, R. MacCurdy, J. Clune, H. Lipson, Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding, in Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (ACM, 2013), pp. 167–174.
76
T. Hoshi, H. Shinoda, 3D shape measuring sheet utilizing gravitational and geomagnetic fields, in 2008 SICE Annual Conference, 20 to 22 August 2008, pp. 915–920.
77
A. Hermanis, R. Cacurs, M. Greitans, Acceleration and magnetic sensor network for shape sensing. IEEE Sensors J. 16, 1271–1280 (2016).
78
C. Rendl, D. Kim, S. Fanello, P. Parzer, C. Rhemann, J. Taylor, M. Zirkl, G. Scheipl, T. Rothlnder, M. Haller, S. Izadi, Flexsense: A transparent self-sensing deformable surface, in Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST’14), Honolulu, Hawaii, USA (ACM, 2014), pp. 129–138.
79
T. L. T. Lun, K. Wang, J. D. L. Ho, K.-H. Lee, K. Y. Sze, K.-K. Kwok, Real-time surface shape sensing for soft and flexible structures using fiber Bragg gratings. IEEE Robot. Autom. Lett. 4, 1454–1461 (2019).
80
L. Pinto, D. Gandhi, Y. Han, Y.-L. Park, A. Gupta, The curious robot: Learning visual representations via physical interactions, in European Conference on Computer Vision (Springer, 2016) pp. 3–18.
81
M. R. Cutkosky, R. D. Howe, W. R. Provancher, Force and tactile sensors, in Springer Handbook of Robotics, B. Siciliano, O. Khatib, Eds. (Springer, 2008), pp. 455–476.
82
M. Li, Y. Bekiroglu, D. Kragic, A. Billard, Learning of grasp adaptation through experience and tactile sensing, in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2014), pp. 3339–3346.
83
N. Wettels, V. J. Santos, R. S. Johansson, G. E. Loeb, Biomimetic tactile sensor array. Adv. Robot. 22, 829–849 (2008).
84
F. Veiga, J. Peters, T. Hermans, Grip stabilization of novel objects using slip prediction. IEEE Trans. Haptics 11, 531–542 (2018).
85
Y. Chebotar, M. Kalakrishnan, A. Yahya, A. Li, S. Schaal, S. Levine, Path integral guided policy search, in 2017 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2017), pp. 3381–3388.
86
H. Van Hoof, N. Chen, M. Karl, P. van der Smagt, J. Peters, Stable reinforcement learning with autoencoders for tactile and visual data, in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2016), pp. 3928–3934.
87
Y. Ohmura, Y. Kuniyoshi, Humanoid robot which can lift a 30kg box by whole body contact and tactile feedback, in 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2007) pp. 1136–1141.
88
P. Mittendorfer, E. Yoshida, G. Cheng, Realizing whole-body tactile interactions with a self-organizing, multi-modal artificial skin on a humanoid robot. Adv. Robot. 29, 51–67 (2015).
89
K. Hertkorn, M. A. Roa, C. Borst, Planning in-hand object manipulation with multifingered hands considering task constraints, in 2013 IEEE International Conference on Robotics and Automation (IEEE, 2013), pp. 617–624.
90
H. Van Hoof, T. Hermans, G. Neumann, J. Peters, Learning robot in-hand manipulation with tactile features, in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) (IEEE, 2015), pp. 121–127.
91
G. E. Loeb, G. A. Tsianos, J. A. Fishel, N. Wettels, S. Schaal, Understanding haptics by evolving mechatronic systems, in Progress in brain research (Elsevier, 2011), vol. 192, pp. 129–144.
92
N. Jamali, C. Sammut, Majority voting: Material classification by tactile sensing using surface texture. IEEE Trans. Robot. 27, 508–521 (2011).
93
J. Sinapov, V. Sukhoy, R. Sahai, A. Stoytchev, Vibrotactile recognition and categorization of surfaces by a humanoid robot. IEEE Trans. Robot. 27, 488–497 (2011).
94
H. Yousef, M. Boukallel, K. Althoefer, Tactile sensing for dexterous in-hand manipulation in robotics––A review. Sensors Actuators A Phys. 167, 171–187 (2011).
95
N. Jamali, C. Sammut, Material classification by tactile sensing using surface textures, in 2010 IEEE International Conference on Robotics and Automation (IEEE, 2010), pp. 2336–2341.
96
B. S. Homberg, R. K. Katzschmann, M. R. Dogar, D. Rus, Haptic identification of objects using a modular soft robotic gripper, in Intelligent Robots and Systems (IROS), 2015 IEEE/RSJ International Conference on (IEEE, 2015), pp. 1698–1705.
97
J. Gottlieb, P.-Y. Oudeyer, M. Lopes, A. Baranes, Information-seeking, curiosity, and attention: Computational and neural mechanisms. Trends Cogn. Sci. 17, 585–593 (2013).
98
S. J. Lederman, R. L. Klatzky, Hand movements: A window into haptic object recognition. Cogn. Psychol. 19, 342–368 (1987).
99
L. Pape, C. M. Oddo, M. Controzzi, C. Cipriani, A. Förster, M. C. Carrozza, J. Schmidhuber, Learning tactile skills through curious exploration. Front. Neurorobot. 6, 6 (2012).
100
J. A. Fishel, G. E. Loeb, Bayesian exploration for intelligent identification of textures. Front. Neurorobot. 6, 4 (2012).
101
Q. Li, C. Schürmann, R. Haschke, H. Ritter, A control framework for tactile servoing, in Robotics: Science and Systems (2013).
102
Z. Su, J. A. Fishel, T. Yamamoto, G. E. Loeb, Use of tactile feedback to control exploratory movements to characterize object compliance. Front. Neurorobot. 6, 7 (2012).
103
M. Kaboli, K. Yao, D. Feng, G. Cheng, Tactile-based active object discrimination and target object search in an unknown workspace. Auton. Robot. 43, 123–152 (2019).
104
H. Iwata, S. Sugano, Whole-body covering tactile interface for human robot coordination, in Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No. 02CH37292) (IEEE, 2002), vol. 4, pp. 3818–3824.
105
M. Frigola, A. Casals, J. Amat, Human-robot interaction based on a sensitive bumper skin, in 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2006), pp. 283–287.
106
S. Haddadin, E. Croft, Physical human–robot interaction, in Springer Handbook of robotics (Springer, 2016), pp. 1835–1874.
107
D. Silvera-Tawil, D. Rye, M. Velonaki, Artificial skin and tactile sensing for socially interactive robots: A review. Robot. Auton. Syst. 63, 230–243 (2015).
108
G. Canepa, R. Petrigliano, M. Campanella, D. De Rossi, Detection of incipient object slippage by skin-like sensing and neural network processing. IEEE Trans. Syst. Man Cybern. B Cybern. 28, 348–356 (1998).
109
G. Cheng, E. Dean-Leon, F. Bergner, J. R. G. Olvera, Q. Leboutet, P. Mittendorfer, A comprehensive realization of robot skin: Sensors, sensing, control, and applications. Proc. IEEE 107, 2034–2051 (2019).
110
A. Del Prete, S. Denei, L. Natale, F. Mastrogiovanni, F. Nori, G. Cannata, G. Metta, Skin spatial calibration using force/torque measurements, in 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE, 2011), pp. 3694–3700.
111
R. Calandra, S. Ivaldi, M. P. Deisenroth, E. Rueckert, J. Peters, Learning inverse dynamics models with contacts, in 2015 IEEE International Conference on Robotics and Automation (ICRA) (IEEE, 2015), pp. 3186–3191.
112
E. Dean-Leon, J. R. Guadarrama-Olvera, F. Bergner, G. Cheng, Whole-body active compliance control for humanoid robots with robot skin, in 2019 International Conference on Robotics and Automation (ICRA) (IEEE, 2019), pp. 5404–5410.
113
E. Steltz, A. Mozeika, N. Rodenberg, E. Brown, H. M. Jaeger, JSEL: Jamming skin enabled locomotion, in 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems (2009), pp. 5672–5677.
114
M. Ishida, D. Drotman, B. Shih, M. Hermes, M. Luhar, M. T. Tolley, Morphing structure for changing hydrodynamic characteristics of a soft underwater walking robot. IEEE Robot. Autom. Lett. 4, 4163–4169 (2019).
115
J. H. Pikul, S. Li, H. Bai, R. T. Hanlon, I. Cohen, R. F. Shepherd, Stretchable surfaces with programmable 3D texture morphing for synthetic camouflaging skins. Science 358, 210–214 (2017).
116
H. A. Sonar, A. P. Gerratt, S. P. Lacour, J. Paik, Closed-loop haptic feedback control using a self-sensing soft pneumatic actuator skin. Soft Robot. 7, 22–29 (2019).
117
J. Wirekoh, L. Valle, N. Pol, Y.-L. Park, Sensorized, flat, pneumatic artificial muscle embedded with biomimetic microfluidic sensors for proprioceptive feedback. Soft Robot. 6, 768–777 (2019).
118
R. S. Dahiya, G. Metta, M. Valle, G. Sandini, Tactile sensing—From humans to humanoids. IEEE Trans. Robot. 26, 1–20 (2009).
119
J. C. Stevens, Aging and spatial acuity of touch. J. Gerontol. 47, P35–P40 (1992).
120
R. S. Johansson, A. B. Vallbo, Tactile sensibility in the human hand: Relative and absolute densities of four types of mechanoreceptive units in glabrous skin. J. Physiol. 286, 283–300 (1979).
121
F. Mancini, A. Bauleo, J. Cole, F. Lui, C. A. Porro, P. Haggard, G. D. Iannetti, Whole-body mapping of spatial acuity for pain and touch. Ann. Neurol. 75, 917–924 (2014).
122
A. B. Vallbo, R. S. Johansson, Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Hum. Neurobiol. 3, 3–14 (1984).
123
R. S. Johansson, J. R. Flanagan, Coding and use of tactile signals from the fingertips in object manipulation tasks. Nat. Rev. Neurosci. 10, 345–359 (2009).
124
M. J. Wells, Octopus: Physiology and Behaviour of an Advanced Invertebrate (Springer Science & Business Media, 2013).
125
M. Wells, Tactile discrimination of surface curvature and shape by the octopus. J. Exp. Biol. 41, 433–445 (1964).
126
K. Takei, T. Takahashi, J. C. Ho, H. Ko, A. G. Gillies, P. W. Leu, R. S. Fearing, A. Javey, Nanowire active-matrix circuitry for low-voltage macroscale artificial skin. Nat. Mater. 9, 821–826 (2010).
127
W. Lee, Y. Liu, Y. Lee, B. K. Sharma, S. M. Shinde, S. D. Kim, K. Nan, Z. Yan, M. Han, Y. Huang, Y. Zhang, J.-H. Ahn, J. A. Rogers, Two-dimensional materials in functional three-dimensional architectures with applications in photodetection and imaging. Nat. Commun. 9, 1417 (2018).
128
O. Khatib, X. Yeh, G. Brantner, B. Soe, B. Kim, S. Ganguly, H. S. Stuart, S. Wang, M. Cutkosky, A. Edsinger, P. Mullins, M. Barham, C. R. Voolstra, K. N. Salama, M. L’Hour, V. Creuze, Ocean one: A robotic avatar for oceanic discovery. IEEE Roboti. Autom. Mag. 23, 20–29 (2016).
129
D. Falanga, K. Kleber, D. Scaramuzza, Dynamic obstacle avoidance for quadrotors with event cameras. Sci. Robot. 5, eaaz9712 (2020).

Information & Authors

Information

Published In

Science Robotics
Volume 5 | Issue 41
April 2020

Copyright

Copyright © 2020 The Authors, some rights reserved; exclusive licensee American Association for the Advancement of Science. No claim to original U.S. Government Works.
This is an article distributed under the terms of the Science Journals Default License.

Submission history

Received: 22 October 2019
Accepted: 24 March 2020

Permissions

Request permissions for this article.

Acknowledgments

We also thank members of the University of California San Diego Bioinspired Robotics and Design Lab and the Yale Faboratory for helpful comments. Funding: B.S. and M.T.T. acknowledge support from the Office of Naval Research (grant nos. N00014-17-1-2062 and N00014-18-1-2277). D.S. was supported by a NASA Space Technology Research Fellowship (grant 80NSSC17K0164). T.G.T. was supported by the SHERO project, a Future and Emerging Technologies (FET) program of the European Commission (grant agreement ID 828818). Z.B. acknowledges support of related research by the Air Force Office of Scientific Research Materials Chemistry Program (grant no. FA9550-18-1-0143). R.K.-B. acknowledges support of related research by the NSF Emerging Frontiers and Multidisciplinary Activities Program (award no. 1830870). Author contributions: B.S., D.S., J.L., and T.G.T conceptualized the review and wrote the manuscript. Y.-L.P., F.I., Z.B., R.K.-B, and M.T.T guided the topics of discussion and structure of the review. All authors provided feedback and revisions. Competing interests: The authors declare that they have no competing financial interests. Data and materials availability: Correspondence and requests for materials should be addressed to M.T.T. (email: [email protected]).

Authors

Affiliations

Benjamin Shih https://orcid.org/0000-0001-8463-2694
Department of Mechanical and Aerospace Engineering, University of California, San Diego, CA, USA.
View all articles by this author
Dylan Shah https://orcid.org/0000-0003-3943-6170
Department of Mechanical Engineering and Materials Science, Yale University, CT, USA.
View all articles by this author
Jinxing Li
Departments of Chemical Engineering and Material Science and Engineering, Stanford University, CA, USA.
View all articles by this author
Thomas G. Thuruthel
Department of Engineering, University of Cambridge, UK.
View all articles by this author
Yong-Lae Park https://orcid.org/0000-0002-2491-2114
Department of Mechanical and Aerospace Engineering, Seoul National University, South Korea.
View all articles by this author
Fumiya Iida https://orcid.org/0000-0001-9246-7190
Department of Engineering, University of Cambridge, UK.
View all articles by this author
Zhenan Bao https://orcid.org/0000-0002-0972-1715
Departments of Chemical Engineering and Material Science and Engineering, Stanford University, CA, USA.
View all articles by this author
Rebecca Kramer-Bottiglio https://orcid.org/0000-0003-2324-8124
Department of Mechanical Engineering and Materials Science, Yale University, CT, USA.
View all articles by this author
Michael T. Tolley* https://orcid.org/0000-0001-7821-7777 [email protected]
Department of Mechanical and Aerospace Engineering, University of California, San Diego, CA, USA.
View all articles by this author

Funding Information

NASA Space Technology Research Fellowship: 80NSSC17K0164
Future and Emerging Technologies (FET) programme of the European Commission: 828818
Air Force Office Scientific Research Materials Chemistry Program: FA9550-18-1-0143

Notes

*Corresponding author. Email: [email protected]

Metrics & Citations

Metrics

Article Usage
Altmetrics

Citations

Export citation

Select the format you want to export the citation of this publication.

Cited by
    • Guizhou Cao,
    • Huige Shi,
    • Dawei Xia,
    • Yi Zheng,
    • Yanhong Liu,
    • Hongnian Yu,
    Observer-Based Adaptive Sliding Mode Control for Soft Actuators with Input Constraints, Proceedings of 2021 Chinese Intelligent Automation Conference, (524-533), (2022).https://doi.org/10.1007/978-981-16-6372-7_58
    Crossref
    • R. S. Diteesawat,
    • T. Helps,
    • M. Taghavi,
    • J. Rossiter,
    Electro-pneumatic pumps for soft robotics, Science Robotics, 6, 51, (2021)./doi/10.1126/scirobotics.abc3721
    Abstract
    • Youcan Yan,
    • Zhe Hu,
    • Zhengbao Yang,
    • Wenzhen Yuan,
    • Chaoyang Song,
    • Jia Pan,
    • Yajing Shen,
    Soft magnetic skin for super-resolution tactile sensing with force self-decoupling, Science Robotics, 6, 51, (2021)./doi/10.1126/scirobotics.abc8801
    Abstract
    • Riyaz Ali Shaik,
    • Elizabeth Rufus,
    Recent trends and role of large area flexible electronics in shape sensing application – a review, Industrial Robot: the international journal of robotics research and application, ahead-of-print, ahead-of-print, (2021).https://doi.org/10.1108/IR-10-2020-0234
    Crossref
    • Yun Liang,
    • Peng Xiao,
    • Feng Ni,
    • Ling Zhang,
    • Tao Zhang,
    • Shuai Wang,
    • Wei Zhou,
    • Wei Lu,
    • Shiao-Wei Kuo,
    • Tao Chen,
    Biomimetic underwater self-perceptive actuating soft system based on highly compliant, morphable and conductive sandwiched thin films, Nano Energy, 81, (105617), (2021).https://doi.org/10.1016/j.nanoen.2020.105617
    Crossref
    • Annalisa T. Taylor,
    • Thomas A. Berrueta,
    • Todd D. Murphey,
    Active learning in robotics: A review of control principles, Mechatronics, 77, (102576), (2021).https://doi.org/10.1016/j.mechatronics.2021.102576
    Crossref
    • Florian Hartmann,
    • Marie Jakešová,
    • Guoyong Mao,
    • Marta Nikić,
    • Martin Kaltenbrunner,
    • Vedran Đerek,
    • Eric Daniel Głowacki,
    Scalable Microfabrication of Folded Parylene‐Based Conductors for Stretchable Electronics, Advanced Electronic Materials, 7, 4, (2001236), (2021).https://doi.org/10.1002/aelm.202001236
    Crossref
    • Libo Gao,
    • Meng Wang,
    • Weidong Wang,
    • Hongcheng Xu,
    • Yuejiao Wang,
    • Haitao Zhao,
    • Ke Cao,
    • Dandan Xu,
    • Lei Li,
    Highly Sensitive Pseudocapacitive Iontronic Pressure Sensor with Broad Sensing Range, Nano-Micro Letters, 13, 1, (2021).https://doi.org/10.1007/s40820-021-00664-w
    Crossref
    • He Yu,
    • Xiaonan Yang,
    • Yunlu Lian,
    • Mingyuan Wang,
    • Yiming Liu,
    • Zhe Li,
    • Yadong Jiang,
    • Jun Gou,
    An integrated flexible multifunctional wearable electronic device for personal health monitoring and thermal management, Sensors and Actuators A: Physical, 318, (112514), (2021).https://doi.org/10.1016/j.sna.2020.112514
    Crossref
    • Zhixing Zhang,
    • Lin Tang,
    • Can Chen,
    • Huitao Yu,
    • Huihui Bai,
    • Ling Wang,
    • Mengmeng Qin,
    • Yiyu Feng,
    • Wei Feng,
    Liquid metal-created macroporous composite hydrogels with self-healing ability and multiple sensations as artificial flexible sensors, Journal of Materials Chemistry A, 9, 2, (875-883), (2021).https://doi.org/10.1039/D0TA09730F
    Crossref
  11. See more

View Options

Get Access

Log in to view the full text

AAAS ID LOGIN

AAAS login provides access to Science for AAAS Members, and access to other journals in the Science family to users who have purchased individual subscriptions.

Log in via OpenAthens.
via OpenAthens
Log in via Shibboleth.
via Shibboleth

More options

Register for free to read this article

As a service to the community, this article is available for free. Login or register for free to read this article.

View options

PDF format

Download this article as a PDF file

Download PDF

Media

Figures

Multimedia

Tables

Share

Share

Share article link

Share on social media

Current Issue

Go to Science Robotics
Table of Contents

LATEST NEWS