Adrian Cheok, professor of pervasive computing at City University London and director of the Mixed Reality Lab at the National University of Singapore, is on a mission to transform cyberspace into a multi-sensory world. He wants to tear through the audiovisual paradigm of the internet by developing devices able to transmit smells, tastes, and tactile sensations over the web.
Lying on the desk in Cheok’s labis one of his inventions: a device that connects to a smartphone and shoots out a given person’s scent when they send you a message or post on your Facebook wall. Then there’s a plexiglass cubic box you can stick your tongue in to taste internet-delivered flavours. Finally, a small plastic and silicone gadget with a pressure sensor and a moveable peg in the middle. It’s a long-distance-kissing machine: You make out with it, and your tongue and lip movements travel over the internet to your partner’s identical device—and vice versa.
“It’s still a prototype but we’ll be able to tweak it and make it transmit a person’s odour, and create the feeling of human body temperature coming from it,” Cheok says, grinning as he points at the twin make-out machines. Just about the only thing Cheok’s device can’t do is ooze digital saliva.
I caught up with Cheok to find out more about his work toward a “multi-sensory internet.”
The make-out device, plugged into an iPhone
Motherboard: Can you tell us a bit more about what you’re doing here, and what this multi-sensory internet is all about?
There is a problem with the current internet technology. The problem is that, online, everything is audiovisual and behind a screen. Even when you interact with your touchscreen, you’re still touching a piece of glass. It’s like being behind a window all the time. Also, on the internet you can’t use all your senses—touch, smell and taste—like you do in the physical world.
Here we are working on new technologies that will allow people to use all their senses while communicating through the Internet. You’ve already seen the kissing machine, and the device that sends smell-messages to your smartphone. We’ve also created devices to hug people via the web: You squeeze a doll and somebody wearing a particular bodysuit feels your hug on their body.
What about tastes and smells? How complex are the scents you can convey through your devices?
We’re still at an early stage, so right now each device can just spray one simple aroma contained in a cartridge. But our long-term goal is acting directly on the brain to produce more elaborated perceptions.
We want to transmit smells without using any chemical, so what we’re going to do is use magnetic coils to stimulate the olfactory bulb [part of the brain associated with smell]. At first, our plan was to insert them through the skull, but unfortunately the olfactory part of the brain is at the bottom, and doing deep-brain stimulation is very difficult.
And having that stuff going on in your brain is quite dangerous, I suppose.
Not much—magnetic fields are very safe. Anyway, our present idea is to place the coils at the back of your mouth. There is a bone there called the palatine bone, which is very close to the region of your brain that makes you perceive smells and tastes. In that way we’ll be able to make you feel them just by means of magnetic actuation.
Cheok demonstrates the taste-transmitter
But why should we send smells and tastes to each other in first place?
For example, somebody may want to send you a sweet or a bitter message to tell you how they’re feeling. Smell and taste are strongly linked with emotions and memories, so a certain smell can affect your mood; that’s a totally new way of communicating. Another use is commercial. We are working with the fourth best restaurant in the world, in Spain, to make a device people can use to smell the menu through their phones.
Can you do the same thing also when it comes to tactile sensations? I mean, can you put something in my brain to make me feel hugged?
It is possible, and there are scientists in Japan who are trying to do that. But the problem with that is that, for the brain, the boundary between touch and pain is very thin. So, if you perform such stimulation you may very easily trigger pain.
It looks like you’re particularly interested in cuddling distant people. When I used to live in Rome, I once had a relationship with a girl living in Turin and it sucked because, well, you can’t make out online. Did you start your research because of a similar episode?
Well, I have always been away from my loved ones. I was born in Australia, but I moved to Japan when I was very young, and I have relatives living in Greece and Malaysia. So maybe my motivation has been my desire to feel closer to my family, rather than to a girl. But of course I know that the internet has globalized our personal networks, so more and more people have long-distance relationships. And, even if we have internet communications, the issue of physical presence is very relevant for distant lovers. That’s why we need to change the internet itself.
The scent device in action
So far you have worked on a long-distance-hugging device and a long-distance-kissing machine. We also have gadgets that can transmit a person’s body odour. If I connect the dots, the next step will be a device for long-distance sex.
Actually, I am currently doing some research about that. You see, the internet has produced a lot of lonely people, who only interact with each other online. Therefore, we need to create technologies that bring people physically—and sexually—together again. Then, there’s another aspect of the issue…
What’s that?
As you noticed, if you put all my devices together, what you’re going to have soon are sorts of “multi-sensory robots”. And I think that, within our lifetime, humans will be able to fall in love with robots and, yeah, even have sex with them.
It seems to me all the work you’re doing here may be very attractive for the internet pornography business.
Of course, one of the big industries that could be interested in our prototypes is the internet sex industry. And, frankly speaking, that being a way of bringing happiness, I think there’s nothing wrong with that. Sex is part of people’s lives. In addition, very often the sex industry has helped to spur technology.
But so far I haven’t been contacted by anybody from that sector. Apparently, there’s quite a big gap between people working in porn and academia.
Visitors can unwrap an Egyptian mummy virtually and even explore amulets buried with the body. Photo: Interspectral
In the third of a series of reports on the highlights of the Singapore Science Festival this year, we look how scientists are making it possible to create virtual worlds real enough to smell and taste.
SINGAPORE — See animals in a zoo that are so real, that you can smell them, when you are merely standing in a room with a video feed. Or send a kiss virtually, one that can be felt.
These are the “realities” Professor Adrian Cheok wants to create, in his mission to make communicating digitally more realistic — by making it possible to send touch, smells and tastes over the Internet.
Prof Cheok, who is from the Imagineering Institute, is among the scientists working in the field of simulation technology, to change the way people experience not just communications, but also science.
For example, children in Spain with certain diseases, who typically spend many hours in the hospital, can now “visit” the zoo virtually through a live 3D video feed, as part of an ongoing study by the Malaysia-based Institute and the University of Valencia to see if virtual zoos have a positive effect on the children.
Unlike sound and light, which are frequency-based and can be sent digitally, taste and smell are chemical-based, so the challenge is in using electrical signals to stimulate these senses, said Prof Cheok, who is working on a device that sends kisses virtually — with the help of a silicon device attached to a mobile phone — and another which artificially produces taste sensations.
Taste and smell are the “most difficult senses” to digitise, said Prof Cheok, and he is working on a project that has electrodes implanted inside the nose to stimulate olfactory receptor neurons to produce a smell.
Another device connects to the tongue to change its temperature, producing a sweet taste through thermal energy.
“In the future it may not be silver electrodes, it could be cutlery which has electrical signals … (with) a tiny wireless electrode in your nose connected to your smartphone,” he said.
Currently, simulation technology in the market is focused on augmented reality, and more widely seen in advertising and entertainment, he said. But researchers focus on inventing new technology and then work with businesses to figure out how it can be made into useful commercial products.
Simulation technology can recreate novel experiences without causing harm, such as allowing people to “unwrap” a mummy — via a visualisation table at the Science Centre Singapore.
Created using X-rays, laser scans and photos of a real mummy, Neswaiu, a wealthy Egyptian priest who lived in the third century BC, visitors can zoom in on the touchscreen to examine Neswaiu’s sarcophagus, and then peel off layers of the body on a touchscreen to study the anatomy, internal organs and even the amulets buried together with him.
The technology behind the Mummy Explorer was originally created for visual medical images so doctors could perform virtual autopsies, but the team “quickly realised there were wider opportunities for the use of the technology”, said Prof Anders Ynnerman, the scientist behind the technology.
The explorer “is a very nice way of being able to engage people … (and let) the general public have a feeling for what scientists are doing and what the scientific exploration process is like”, since they can actively participate and freely explore the mummies themselves.
The first interactive touch table was completed in 2009, and has been used in other museums, like the British Museum and the Museum of Mediterranean and Near Eastern Antiquities in Stockholm.
Prof Ynnerman said the British Museum was initially worried that the dazzle of a digital artefact would leave the real mummy in the cold. But they found that people spent thrice the amount of time at the exhibit, looking at the mummy, his digital counterpart, and then at the mummy again to study it closely.
He hopes that with such interactive visualisations can shorten the distance between scientific research and the general public, so that through exploring, they can feel “part of the scientific discovery”.
¿Cuánto nos falta para tocar, oler y besar por internet?
Conseguir transmitir los 5 sentidos a través de una conexión remota. Esa es la obsesión del australiano Adrian David Cheok, profesor de la Universidad de Singapur, quien gira por todo el mundo enseñando sus inventos para probar sabores a través de chips, oler e incluso darse besos. Su tesis es sencilla:convertir Internet en un espacio conectado… emocionalmente. Digitalizar los 5 sentidos —olfato, vista, oído, tacto y gusto— , replicando el complejo mundo real a través de hardware.
Para ello su laboratorio lleva trabajando en proyectos multisensoriales desde hace más de dos años. Ya que nuestra comunicación se fundamenta, en gran medida, en el uso de Internet y el smartphone mediante las redes sociales, Cheok cree que es ahí donde debemos orientar nuestros progresos.
Enviarás besos a través del móvil
Digitalizar los sentidos supondría un cambio radical en nuestra forma de comunicarnos. El primer progreso de Cheok en este campo gira en torno al tacto. Siguiendo los pasos de Kissenger, aquel robot de la compañía Lovotics a través del que podíamos mandar besos, el device de Cheok es un rectángulo de silicona, de color rosado, con una clavija para fijar a la base del smartphone. Mediante conexión WiFi puede comunicarse con un aparato análogo.
El dispositivo dispone de sensores de presión: un algoritmo calcula en tiempo real los índice de presión y dirección cuando se produce el beso. Pero también podemos usarlo, simple y llanamente, para transmitir sentimientos positivos, de buena suerte, para saludar, o para felicitar el cumpleaños a nuestra abuela. Si no queremos cerrar los ojos, a través de la pantalla del móvil vemos la frente o el rostro de la persona besada. El cacharro en cuestión es lavable y tiene la forma de unos labios estándar. No son los de tu pareja ideal, no tienen ningún cálido aroma, pero la textura tampoco se asemeja a una simple tubería de PVC.
Cheok establece un paralelismo interesante: antiguamente copiábamos nuestras canciones favoritas a dos o tres amigos, y las pasábamos en cintas. Ese fue el nacimiento de la digitalización. Pero ahora todo el mundo puede descargar la canción en un par de segundos, podemos chivar un descubrimiento emergente y que se convierta en algo de popularidad mundial en varios días. Cheok persigue esa comunicación, esa forma de transmitir información de forma más directa, a la antigua usanza.
Hay algo psicológico en el contacto. Las relaciones a larga distancia se rompen por esta brecha. Uno puede pasarse horas colgado del Skype y, aunque mejora sobremanera la malinterpretación y deformación de los chats, sigue siendo algo frío. Cuando cuelgas la llamada sientes que un silencio se apodera de ti. La emociones son difíciles de asir. Y llevamos un montón de años intentándolo.
De 100 años a 10: la carrera por digitalizar los sentidos
Como señala el propio Cheok, la digitalización del audio nos tomó un siglo, 50 años hacer lo mismo con el vídeo —y ya hemos llegado hacia la representación realista del mismo, mediante el HDR—. «Digitalizar el olor o el sabor no nos llevará más de 10 o 15 años, debido a la aceleración de la tecnología».
La digitalización de los olores o sabores siempre ha estado ahí de alguna u otra manera. Dejar una rosa disecada entre las páginas de algún pasaje concreto: de las cartas aromáticas a los cuentos de Gerónimo Stilton. En la industria cinematográfica la digitalización de olores fue bastante explícita y caótica. El suizo Hans Laube presentó en la Expo Mundial de Nueva York de 1939 ‘Scentovision’, un sistema que añadía banda aromática a la película. De esta manera, dosificando pequeñas dosis con tubitos escondidos tras las butacas de cada espectador, se amplificaría el poder narrativo de la obra. No llegó a mayores.
El productor de Minneapolis Michael Todd hizo lo propio, siguiendo las pesquisas de su padre, y presentó ‘Smell-O-Vision’ bajo el eslogan «¡Primero se movieron (1895)! ¡Después comenzaron a hablar (1927)! ¡Ahora huelen!». Fue un fracaso que nunca llegó a despegar. Así las cosas, los años siguieron con tibias investigaciones. Terrence Malick apoyó una idea similar para su ‘The New World’ (2005), su relectura del mito de Pocahontas con aromas de tierra labrada y madera recién cortada. En 2013, varios cines de Nagoya (Japón) proyectaron ‘Iron Man 3’ recurriendo la tecnología 4DX o, lo que es lo mismo, una mezcla de vídeo 3D apoyado con efectos físicos —luz estroboscópica, niebla, aire caliente—, junto a olores dispersados por los conductos de ventilación.
Fuera del circuito comercial, la técnica para digitalizar olores (cromatografía) ha evolucionado por dipares derroteros. La empresa californiana Digiscent presentó hace algunos años iSmell, aplicación para incorporar olores a diferentes plataformas tecnológicas, como un simple correo electrónico. Un aparato receptor sintetiza el olor a través de un chip que asimila y analiza la huella que dejan los olores, y otro imprime y reproduce el olor asimilado. La compañía japonesa Chaku hizo algo similar en 2012 con las “narices electrónicas” y desarrolló Chat Perf, una forma de enviar olores a distancia mediante un atomizador conectado al puerto de iPhone, una suerte de smell-o-phone para pequeños consumidores.
Es fácil digitalizar el audio y la visión, son números
El audio y la imagen se miden en frecuencias: espectro visible, audible, y bits de resolución. Ya se sabe: los 8 bits de color hacen referencia a las 256 sombras de cada color primario: 256 de rojo, 256 de verde y 256 de azul. Con el audio sucede algo similar. Medimos la altura dentro del espacio audible —de los 20 Hertzios a los 20.000—, medimos el timbre sonoro, y medimos la intensidad, en decibelios. Con estos tres valores sabemos si el sonido de un fagot es “grave”, “metálico” y “suave”, basándonos en la potencia de la fuente sonora.
El gusto y el olfato se basan en moléculas sensoras. Son como una versión más hormoral y menos cerebral. Para interpretarlas no nos bastan los números, necesitamos una estimulación sensorial, que en el cerebro se traduce en electricidad. Es como enviar pequeños paquetes de datos codificados. El receptor revisa el matasellos y determina «este aroma a cítrico es agradable y fresco». Así que, como apunta Cheok en el vídeo, si queremos sentir esa conjunción aromática-olfativa, necesitamos condensar esa huella en bits de información y transmitirla eléctricamente a nuestro cerebro. Hablar su mismo idioma.
El problema de no poder digitalizar una molécula
El camino usual es acudir a la química. Como decíamos, las imágenes y los sonidos son señales de vibración y luz, mientras que los aromas y sabores son señales químicas. ¿Cómo podemos estimular estos receptores sin usar químicos? No podemos transmitir moléculas ni partículas a través de Internet. ¿O sí? Podemos detectar la diabetes mediante lectores de glucosa. El dulzor se convierte en una señal digital que el lector de insulina interpreta. Esa información la podemos exportar incluso a una tarjeta de memoria.
El problema de digitalizar una molécula deviene no tanto por la información misma de la molécula sino por cómo la interpretamos. Me explico: en el estudio de la cóclea se concluye que podemos detectar frecuencias limitadas, entre 20Hz y 20kHz, como decíamos. En el estudio del ojo sabemos cómo engañar y adaptarnos a sus características, con sus 7-8 millones de conos y 120 millones de bastones —receptores de color y luminancia, respectivamente—.
El olfato es otro cantar. Un humano sano puede detectarunos 1.200 millonesde olores distintos. Y se memorizan mejor que cualquier otro recuerdo generado por una sensación. Las posibilidades combinatorias generan una posibilidad combinatoria prácticamente infinita. ¿Cómo sabemos qué moléculas descartar y cuáles seleccionar?
Ni siquiera podríamos construir un modelo estándar: no todas las personas tienen las papilas gustativas iguales. Aunque todo el mundo tiene receptores similares (dulce, salado, amargo, ácido y umami) la forma en la que interpretamos esas moléculas es única, como una huella dactilar.
Solución: estimular al cerebro con electricidad
De igual manera que hacen las conexiones sinápticas de nuestras neuronas, la única forma eficaz de interpretar correctamente estas moléculas a través de miles de kilómetros es mediante la estimulación eléctrica. Tal vez hayáis oído hablar del ‘brain hacking’, una forma de estimulación transcraneal de corriente directa. Consiste en aplicar, de manera controlada, pequeñas descargas con objeto de tratar patologías, estimular la memoria en personas con amnesia, u ordenar que cierta parte del cuerpo ejecute cierta función —como provocar el crecimiento de pelo—.
El profesor de ingeniería biomédica de la Universidad de NYC dejó bien claro, eneste artículo de Nature, que la versión casera y voluntaria de este hackeo es verdaderamente peligrosa. De hecho, aún no se conocen los efectos secundarios a largo plazo de estas prácticas. En este particular no estamos hablando de dopaje, estamos hablando de reinterpretación, de leer los olores como podríamos leer un libro. Y los dispositivos actuales aún no alcanzan esta meta con eficacia.
Cambiarás la manera de relacionarte con las personas
La idea de Cheok es eminentemente social. En el mundo real sólo puedes dar una abrazo a una persona, si logramos compartir los sentidos de manera eficaz podremos dar abrazos a miles, de manera simultánea. Su meta es replicar el comportamiento del mundo real a escala global.
Un ejemplo: la cultura del lenguaje emoji. Lo que en principio era mera anécdota y refuerzo para transmitir un mensaje eminentemente textual se ha convertido en el núcleo del mensaje. Podemos lanzar un emoji tan cargado de significado como cualquier carta. La teoría de Cheok sólo necesita algo de tiempo extra. Como seres emocionales y sociales, la comunicación seguirá más allá de cualquier forma de digitalización, de los implantes cibernéticos o la humanización de Internet.
Tu amigo del futuro no existirá, será un robot
El tiempo es una centrifugadora. Hemos pasado del planteamiento de usar robots para comunicarnos, normalizando su uso en actos cotidianos, a implicarnos en las tareas del hogar y, cómo no, hacerlos partícipes de nuestra infidelidad poniéndolos a repartir abrazos por contrato. ¿Vamos camino, como en la inteligente cinta de Masamune Shirow ‘Ghost in the Shell’ (1995), hacia una dependencia emocional con los robots? ¿O sólo se trata de medir nuestras posibilidades como creadores? Fijaos en el mito de Pinocho: su padre-creador Geppetto no buscan insuflar vida, sino sustituar una serie de querencias afectivas.
‘Her’ (Spike Jonze, 2014) es popularmente conocida por ampliar cierto debate sobre la Inteligencia Artificial. ¿Puede una persona enamorarse de una simple voz, una Inteligencia Artificial intangible? Porque no se trata de ficción, sino de un debate lógico entre lo que podemos tocar y lo que podemos sentir. Y de ahí saltaríamos a otro debate: de volcarse ese cerebro artificial a un cuerpo humanoide, como en la popular ‘Ex Machina’ (Alex Gardland, 2015), ¿tomaría plena consciencia de sus capacidades, sentiría el deseo de satisfacer a su dueño o marchar libremente, con las implicaciones emocionales que implica?
«¿Por qué debería volar en un avión? ¿Por qué debería siquiera salir de casa para ir a trabajar?». El propio Cheok, en el vídeo de cabecera, asume algunos comportamientos: ya que el ser humano parece abocado poco a poco a cierto inmovilismo, a las rutas guiadas por museos virtuales en vez de visitar los reales, ¿preferiremos a un robot antes que una persona, por el simple hecho de resultarnos más cómodo?
Los progresos en Inteligencia Artificial arrojan un futuro híbrido: sólo hay que observar los vehículos autoconducidos. Por un lado tenemos algoritmos de aprendizaje automático, vinculados a la generación procedural, y que procesarán el lenguaje humano de una forma mucho más intuitiva que los asistentes actuales (Cortana, Siri, etcétera). Gracias a la conexión permanente, al “Internet de las cosas”, todo gadget que no incluye cierta tecnología de computación e IA se considerará estúpida, obsoleta.
El análisis visual de datos ha mejorado exponencialmente gracias a los ojos biónicos, e incluso con proyectos como Tango, de Google, los sensores ya analizan e interpretan el volumen de los cuerpos en el espacio.
La industria que más esperaría este cambio no es apta para menores de 18 años
Las salas multisensoriales —que acuñan su nombre holandés “snoezelen” de las palabras “impregnarse” y “soñar”— aparecieron en los años 70 para el desarrollo cognitivo y han ido adaptándose, poco a poco, al mercado de lo sexual. La tecnología Teledildonics es una versión hardcore del Kissenger, donde las sensaciones táctiles se comunican a través de un enlace de datos entre los participantes. Y sí, hace referencia al sexo robótico o, como mínimo, a la estimulación erótica a distancia.
Un cambio en la comunicación sensorial a distancia revolucionaría de arriba abajo el mercado adulto. Las páginas dedicadas a ello encontrarían un filón para sus clientes o usuarios premium. El mejor ejemplo lo podemos encontrar en la Realidad Virtual: canales, aplicaciones y líneas de venta dedicadas exclusivamente a ello. Ellos fueron los pioneros, los primeros en adaptar los nuevos formatos, del DVD al Blu-Ray, del uso de cierta ropa o implantar comportamientos y lenguajes sociales.
Así pues, si los robots aún se comunican de forma muy aséptica, habrá quien aproveche esa inmersión, afianzando esa conexión entre personas de maneras más literales. Aunque algunas marcas quieran evitarlo por contrato.
No falta mucho para que algún chef de prestigio mundial -de los de estrellas Michelin, esferificaciones nitrogenadas y platos a 200 euros- comience a enviar sus creaciones por Internet. No hablamos de un servicio a domicilio (¡qué poco glamour!) sino de un envío literal. Es decir, entrar en una web, seleccionar una receta de nombre imposible, conectarse unos sensores y… disfrutar a través de estímulos en tus sentidos del placer de la alta cocina. Tu cerebro interpretará aquello como la más deliciosa de las comidas, aunque en términos nutricionales sea la nada absoluta. Adrian Cheok cree que, al ritmo al que avanzan las nuevas tecnologías, esto será posible en 15 o 20 años. Tal vez menos si los experimentos que su equipo del Imagineering Institute, laboratorio que él mismo dirige, tienen éxito en próximas fechas. De momento ya han conseguido cosas tan asombrosas dentro de este campo como transmitir olores a través de una aplicación móvil, reproducir el sabor dulce mediante un dispositivo que estimula las papilas gustativas o engañar al cerebro haciéndole creer que la cantidad de comida ingerida es mayor que la consumida realmente.
Cheok es uno de los investigadores más importantes del mundo en el terreno de la denominada “realidad mixta”, un concepto que sobrepasa la realidad aumentada y la realidad virtual, puesto que su objetivo es combinar ambos mundos -el físico y el digital- en tiempo real para que puedan convivir e interactuar. El trabajo de Cheok desde el Imagineering Institute es “construir la Internet del futuro” mediante la convergencia de la tecnología, el arte y la creatividad. La idea de transmitir todo tipo de sensaciones a través de Internet se sustenta, según Cheok, en “el deseo de los seres humanos de comunicarse; la tecnología debe orientarse a satisfacer estas necesidades fundamentales, porque es esencial para nuestra felicidad”. Ser feliz, por ejemplo, sintiendo el abrazo o el beso de la persona amada aunque se encuentre a muchos kilómetros de distancia, es uno de los experimentos que ya han llevado a cabo con éxito desde el equipo de Cheok.
Pero sus planes no se detienen en el envío de información digital entre dos seres humanos, por mucho que esto despierte nuestras emociones. Quiere ir mucho más allá: “El siguiente paso es que esa persona no tiene porqué ser real. Puede ser una creación virtual. Tendremos robots o personajes virtuales que serán nuestros amigos y podremos comunicarnos con ellos a través de nuestros cinco sentidos gracias a Internet. La combinación de la inteligencia artificial y la tecnología multisensorial lo hará posible en el futuro. Tendremos amigos e incluso amantes que no serán físicos”.
In this episode, presenters of The Gadget Show, Amy Williams and Jason Bradbury, visit Professor Adrian Cheok at the Imagineering Lab in London and try out some of his whacky devices for the senses.
David Levy and Adrian David Cheok, founders of the annual Congress on Love and Sex with Robots, assert we’re very close now to creating a sex robot, going as far as to say that such robots will be normalized by 2050. If recent press cycles are any indication, most of us believe this.
But we’re wrong.
That we have made immense progress in a number of disciplines that directly relate to the development of humanoid robots is not in dispute. Over the past few weeks, one of Google’s artificial neural networks has shown that by using reinforcement learning it could best the world champion of Go, an abstract game that is harder than chess. This type of learning-on-the-go has serious implications for robots, which have difficulty negotiating new terrains, objects and situations.
Likewise, developments in the molding of polymers are revolutionizing the field of soft robotics, allowing robots to better physically deal with real-world dynamic environments and handle objects that fall outside their programming parameters. We’ve even created thin polymer films with built-in sensors that sense pressure and heat, not unlike skin.
But we’re wrong to think that these and so many other other advances will coalesce as easily as people seem to imagine to create a commercially-available, self-aware, humanoid sex robot.
The self-aware humanoid we imagine when we think of a sex robot will need a variety of sensors to have an awareness of its environment. It will need to have a sophisticated artificial intelligence. It will need machine learning to respond to changes and negotiate situations outside its programing. It will need natural-language processing. To overcome the uncanny valley, its movements and expressions will have to be matched to human expectation. Its skin will require nanotechnology to replicate the lifelike lack of uniformity of human flesh, and its eyes will require a different nanotechnology to simulate the wetness of our own.
Sex robots are going to require multiple disciplines to come together because they’re not simply mechanized sleeves or dildos. Unlike a haptic dildo or robotic sleeve, sex robots are not going to be simple enough for a single genius to put together on their own.
But if the hurdles facing haptic dildos, robotic sleeves and other creations within the nascent sextech industry are any indication, we are never going to get to the sex robots at all.
Andrew Quitmeyer, cofounder of the sextech startup Comingle, had no idea he was being sued when he responded to an unsolicited message from an attorney asking if Comingle already had legal representation.
“I was like, ‘LOL, representation for what?'” he told me over the phone. That’s when he learned that a firm by the name of TZU Technologies had filed a lawsuit against his startup for infringing on a patent held by TZU.
Comingle, like most sextech startups, was a small operation. Knowing that every human body is different, the company was building an open-source platform to help users hack its sex toys so that any user would be able to modify vibration patterns to suit their preferences.
Comingle had been lucky. Harnessing the power of social media, the startup had managed to use crowdfunding to get around the problem of financing that plagues so many companies in sextech.
It’s an unfortunate reality that many sextech companies find it difficult to get small business loans due to morality clauses and banks’ concerns over “reputational risk.” And investors too are wary of sextech. Quitmeyer has lost count of the number of times he was invited to show investors a deck, only to be told afterward that while Comingle’s work is great, investors simply don’t fund things that fall under the category of “sex, drugs, and rock ‘n’ roll.”
“The amount of publicity that we’ve been able to gain at Comingle—if we were any other Silicon Valley startup, we’d already be in our A-round of funding upwards of millions of dollars,” Quitmeyer said.
“We’ve been kicked out of two accelerators!” he added. “We passed all their hoops and training and customer discovery and at the end, when they’re supposed to give you space and funding and support, they came back and said, ‘we checked with the higher-ups and turns out we’re not comfortable dealing with sex stuff. Goodbye.’ Months lost.”
It’s not all puritanism: in a country where sex is so politicized, investors are right to worry about the impact of future government regulation. Something as simple as limiting the online purchase of adult technology to credit cards—as opposed to also allowing debit cards which might enable minors to have access to these products—could dramatically impact a product’s likelihood of success, since not all adults have credit cards or want to use them for such purchases.
Sextech companies also face restriction from other companies: Google and Apple, for example, grudgingly allow sex-related health apps, but their acceptance of sextech that exists solely for pleasure and titillation has so far been spotty. Would Play or the App Store let you gear up your sexbot as you begin your commute home from work in the same way they let you do with your Nest? Their track record doesn’t bode well for sexbots.
This turns off investors, too. Sean Percival, a venture partner with the seed investor firm 500 Startups in Mountain View, told me that being barred by such key distribution channels is a serious handicap for a company.
“Getting rejected [by a main distribution channel like Play or the App Store] would make it difficult for you to scale,” Percival said.
Like most people, Percival was only vaguely aware of the existence third-party app stores that cater to adult apps. But even MiKandi, the best known Android third-party store for adult apps, only has 5 million users—a tiny fraction compared to Google’s Play Store, which reported a billion users last year.
The situation makes investment dicey, disadvantaging sextech.
But funding is difficult to come by, and after factoring in all the costs of bringing a product to market and running the day-to-day operations of business, there’s little left over for the research and development required to take the industry to the next level, and certainly not to enable them to start the work needed to bring sexbots into being.
And this is all before a sextech company is targeted by a patent troll.
A patent troll is generally a business that makes most of its money by buying patents and getting companies that infringe on them to pay a licensing fee. In the US, inventors who file for patents are given exclusive rights to their inventions for 20 years, a monopoly meant to enable them to recoup the cost of developing their invention.
This makes sense in theory. For example, a pharmaceutical company that spends years developing a vaccine would have no reason to make the hefty investment required for that research without the protection of a monopoly. At the same time, by disclosing its invention to the state and allowing it to eventually go into the public domain, the risk of losing that knowledge is minimized.
But the system isn’t perfect, and its flaws have allowed patent trolls and frivolous lawsuits to proliferate. In 2011, it was estimated that patent litigation cost the tech industry $29 billion in legal and licensing fees. That figure has been on the rise: it was $6.7 billion in 2005 and $12.6 billion in 2008. With total spending on research and development in the United States at $247 billion, some have argued that patent infringement lawsuits have in effect levied a 10 percent tax on innovation. But this is only true when patent trolls go after big companies. When they go after small ones, they often sink a company.
The Real Touch is a perfect example of the chilling effect of patent trolls on sextech innovation. The North Carolina-based teledildonics company Real Touch made a splash in 2009 with a sleeve that plugged into a computer via USB and transmitted sensations based on scenes of a viewer’s choosing. The sleeve didn’t only squeeze: it simulated heat, wetness, friction and intensity, all of which were coded differently depending on whether the scene being watched featured vaginal sex, anal sex, oral sex, manual sex, and so on.
Real Touch was our first serious step toward the pleasures we imagine when we think of sex robots, and at $200, it wasn’t completely inaccessible in terms of price. Yet by 2013—and despite being featured on HBO’s Sex/Now and Amazon’s Betas—Real Touch was no longer manufacturing new units. The revenue from sales could not keep up with licensing demands made by patent trolls.
It says something that despite not having sold any new devices since 2014, Real Touch’s parent company, Internet Services LLC, was nevertheless named in the lawsuit by TZU that also targeted Comingle in June of last year.
This lawsuit is over the infringement of a patent for “a stimulation device receiving the control signal from a user interface.”
If that patent seems overly broad, you have identified the main problem with our country’s patent system. As the software industry has been saying for decades, the main cause of frivolous patent lawsuits falls squarely on the shoulders of the United States Patent and Trademark Office (USPTO).
“Patents are too easily issued right now,” said Carter Laren, a Bay Area-investor and director at the startup launchpad Founder Institute. “You get lots of junk patents issued that are obvious or even though there is prior art [evidence that an invention is already known] and they should never have been issued.”
Laren is rare among investors in that he was willing to fund a sextech startup. Almost ten years ago, he put his money on OhMiBod, a small New Hampshire company that makes a line of vibrators that can be operated via wifi or Bluetooth. (Since its launch in 2006, OhMiBod has raised $750,000. Compare that to TechCrunch’s 2013 figure of$41 million raised for the average successful startup.)
When OhMiBod started to draw serious media attention, the startup was immediately targeted by a patent troll.
“It was some guy who hadn’t produced anything, had written a patent that was something about combining audio signals with vibrations,” Laren recalled. “I did some internet research—there was prior art for his patent. What he was claiming easily could have been knocked out in court. Our lawyers agreed that in court we would win. But they said, ‘this is what it’s going to cost you in court. It’s probably just worth settling.'”
An invention must be new, useful, and nonobvious to qualify for a patent. But if patent examiners at the USPTO aren’t well-versed in advances in a certain sector (and they usually aren’t), things that are known within an industry can appear nonobvious to them, resulting in the issue of junk patents that are used to squeeze money out of innovators instead of protecting legitimate inventions.
“Very rarely do you end up with any patent examiner who understands the patents, actually really understands any of the prior art, and pushes as much as he or she should,” Laren said. “The whole thing is kind of just a joke. If you’ve ever gone to the Patent Office, it’s like going through the motions. I think a large percentage of patents that get issued, I would classify as obvious.”
The sextech space is having a junk patent frenzy not unlike that which has chilled innovation in the software space. There’s big money to be made in getting a junk patent filed and going after settlements from anyone who infringes.
Quitmeyer thinks these egregious patent tactics have an even easier time getting by in sextech because of the stigma of sex.
“Once you have the word sex in it, it appears that the Patent Office gets a little freaked out or something and they just let anything pass,” he told me. “Someone applies to the Patent Office, and they’re like, ‘Hey! I have an idea! Here’s a thing with more than one vibrating motor. Isn’t that crazy? Most things have one motor, what if we have two?’ When you start looking at these patents, it gets freaky and scary [seeing what has managed to get a patent].”
The example he gives may sound hyperbolic, but it isn’t. The company Wing Pow filed a patent in 2009 for a “massage device with a plurality of vibrators.” It was awarded. Wing Pow also holds a patent for a “mechanized dildo” that isn’t even a vague improvement over the Pearl Rabbit, a vibrator that was already well known to America when the Wing Pow patent was filed in 2008, having made an appearance a whole decade earlier on an episode on the HBO show Sex and the City.
Patent trolls are not unaware that a number of the patents that they acquire and enforce are dubious. They rely on the high cost of launching a legal defense to cow companies into paying speedy settlements.
“The idea of issuing the patent is, well, you can go win in a court case and show the prior art and show that it was obvious and throw the patent out,” said Laren. “That’s true, but that costs money. So you end up in a shakedown situation. Anyone can come in and harass you with a shitty, crappy patent, and you have to pay them off to go away.”
The cost of defending a patent in court can easily run into the six figures. In 2011, aBoston University study in 2012 found that small- and medium-sized companies, like those that dominate the sextech space, spend $420,000 to litigate a patent dispute on average, with the median at $70,000. Two separate studies, that from Boston University and another from Santa Clara University School of Law, determined that over half of patent troll lawsuits targeted small and medium companies with a median revenue of $10 million.
By going after small companies that don’t have the resources to wage a costly legal battle, patent trolls are almost guaranteed a speedy settlement. An analysis in 2013indicated that while patent trolls filed 67 percent of all new patent infringement cases (up from 28 percent in 2008), only 20 percent of decisions involved them, illustrating the general tendency for such cases to settle. These settlements add up, enabling patent trolls to go after bigger, more lucrative targets.
Laren is glad OhMiBod settled.
“You’re not just paying lawyers,” Laren explained. “When you’re a small business, that distraction costs you. You gotta spend your time and energy dealing with this instead of growing the business, and that has a real impact. When we looked at everything, it was clearly going to cost in the six figures, and it was easier to just pay this guy off than it was to keep moving forward and fight him even though we knew were going to win.”
Julie Samuels, the president of Engine, a San Francisco-based nonprofit that advocates on behalf of tech startups, reiterates this. Speaking to Vice News, shenoted, “What we’ve seen, particularly in the software space, is that patents aren’t incentivizing innovation. Instead, they’re having a chilling effect because no one wants to be sued.”
Patent trolls demand a median of $180,000 as a licensing fee to make their lawsuits go away. Quitmeyer describes the six-figure pay-off as the “sweet spot.”
This “sweet spot” is what TZU Technologies was after when they came for Comingle. As mentioned earlier, the patent held by TZU is so vague, TZU could easily threaten legal action against any technology company that relays a command from a processor to a device that vibrates. The reason TZU doesn’t do that is that the patent is so dubious that any company with the money to take the company on would win, so like most trolls, TZU focuses on small outfits that can’t afford the fight.
(In a moment of incredible hubris last year, TZU sued Kickstarter for enabling the violation of its patent by allowing infringing companies to use the crowdfunding platform to raise money, but it quickly dropped the case when the well-positioned Kickstarter decided to take the case to court.)
“[The TZU patent] is being asserted against anyone who does any sort of sex toy that uses a network or a wireless communication device to operate a sexual device,” said Franklin Veaux, winner of last year’s highest award in sextech, Arse Elektronika’s Golden Kleene. “So for example, anyone using Bluetooth, anyone making an internet-controlled toy, even people using wireless remotes are getting hit by this patent troll.”
Veaux is trying to help Comingle and other targets of TZU get together the necessary prior art to kill the patent once and for all. Veaux himself developed a prototype for a teledildonics device called the Symphony using the technology TZU’s patent covers—three years before the patent was filed.
“The patent troll uses essentially mob-style tactics,” Quitmeyer told me. He recalled talking to TZU’s attorneys: “They looked at us and said, ‘hey, we can make this all go away. Just let us know how much money you have.’ I get phone calls in the middle of the night—which I assume are scare tactics—from anonymous numbers where people will leave me phone messages that say stuff like, ‘hey, Andy, I hear that you don’t like patents. Well, sometimes you just have to play by the rules. I’ll see you soon.’ Isn’t that insane?”
Comingle didn’t have the money to pay off the trolls, much less to fight a costly legal battle. The cofounders were about to close up shop when the Electronic Frontier Foundation got them in touch with a lawyer willing to fight their case pro bono.
But since being sued by TZU last summer, Comingle continued to receive additional cease and desist letters relating to other patents, among them from the aforementioned junk patent bully Wing Pow.
“When we get a cease and desist letter, we have to go to a lawyer—and even with a discounted lawyer who likes our cause, writing a letter for us to get [patent trolls] to back off still costs $700 or $800,” Quitmeyer lamented. “It’s just a nonstop feeding frenzy with these guys.”
Last month, with the case against TZU still ongoing and after a barrage of cease and desist letters, Comingle’s cofounders finally made the decision to cancel their project.
This week’s podcast brings you two stories about how humans interact with artificial intelligence. Radio Motherboard is available on iTunes and all podcast apps.
“This terrible case [against TZU], (that’s still going), feels like it will never disappear, and this sort of bullshit can take not only massive financial tolls, but extremely mental tolls as well,” they wrote on a post announcing the decision. “It’s difficult to keep working non-stop, every day, when there is a huge thing like an international lawsuit looming over you.”
Stories like those of Comingle and Real Touch are significant because sex robots are not simply mechanized sleeves or dildos. They’re not going to be simple enough for a single genius to put together like she might a haptic sex toy or a chat bot. They’re going to need a lot of tech, and that means patents.
Look at your smartphone. That sleek little machine that fits into your pocket contains some 250,000 different patented components. How many do you think will make up the robots we have in mind? Only one thing is certain: licensing will be steep. This means serious investment, and given the current attitudes toward sex-related products in this country that is not likely.
