Managing Director Tan Sri Dato’ Azman bin Hj Mokhtar of the Government of Malaysia’s strategic investment fund, the Khazanah Nasional Berhad, is impressed with City talent in the heart of London’s Silicon Roundabout.
A delegation from the Government of Malaysia’s strategic investment fund, the Khazanah National Berhad, visited the Hangout on January 30th.
Located at the epicentre of London’s Tech City, the Hangout provides a unique working environment and incubation space for City University London academics, students and start-ups to foster relationships with investors in the area in order to get their businesses off the ground.
Khazanah Nasional Berhad promotes economic growth and makes strategic investments on behalf of the Government of Malaysia and is keenly interested in partnering with technology companies and products originating at City. With an investment portfolio comprising over 50 major companies in Malaysia and abroad worth £30bn, Khazanah is involved in a broad spectrum of industries.
Led by managing director Tan Sri Dato’ Azman bin Hj Mokhtar, the Malaysian delegation listened attentively to a variety of presentations and investment opportunities. The Khazanah managing director was impressed with the “high level of creativity” being nurtured at City.
These included a presentation on taste and smell actuation via mobile phone from Professor of Pervasive Computing, Professor Adrian Cheok and PhD students from his Mixed Reality Lab; BarPassOfficial (for payment and collection of drink orders via smartphone); Mashmachines (a new media player bringing together sound, lighting, and video into a single user interface); Popcord (an innovative lightweight mobile phone charger); TechCityNews (London’s leading tech sector news and analysis resource); Modafirma (a social commerce platform allowing emerging and independent fashion designers to reach and sell directly to a global audience); and AtomicDataLabs (a software and data management company building applications in large datasets).
Also in attendance were Pro Vice Chancellor for Research & Enterprise, Professor John Fothergill; Director of Enterprise, Dr Sue O’Hare; Dean of the School of Engineering & Mathematical Sciences and the School of Informatics, Professor Roger Crouch; Professor of Dependability and Security, Professor Kevin Jones; Manager of the London City Incubator and Hangout founder, Leo Castellanos; and, Andrew Humphries, co-founder of The Bakery.
The New Economy interviews Professor Adrian Cheok of City University London to find out about a new technology that will allow people to taste and smell through their mobile phones
Scientists are coming increasingly closer to developing technology that will allow us to use all our senses on the internet. Professor Adrian Cheok, City University London, explains how mobile phones will soon allow people to taste and smell, what the commercial benefits of this technology might be, and just how much it’s going to cost.
The online world: it’s all about the visual and sound experience, but with three other senses, it can leave us short. Studies have demonstrated that more than half of human communication is non-verbal, so scientists are working on ways to communicate taste, touch, and smell over the internet. I’ve come to the City University London to meet Professor Adrian Cheok, who’s at the forefront of augmented reality, with new technology that will allow you to taste and smell through a mobile phone.
The New Economy: Adrian, this sounds completely unbelievable. Have you really found a way to transmit taste and smells via a mobile device?
Adrian Cheok: Yes, in our laboratory research we’ve been making devices which can connect electrical and digital signals to your tongue, as well as your nose. So for example, for taste we’ve created a device which you put on your tongue, and it has electrodes. What those do is artificially excite your taste receptors. So certain electrical signals will excite the receptors, and that will produce artificial taste sensations in your brain. So you will be able to experience, for example, salty, sweet, sour, bitter – the basic tastes on your tongue – without any chemicals.
It’s a device that you can attach to your mobile phone, and these devices will emit chemicals
And with smell we’re going in a couple of tracks. One is using chemicals, it’s a device that you can attach to your mobile phone, and these devices will emit chemicals. So that means that with apps and software on your phone, you can send someone a smell message. For example, you might get a message on Facebook, and it can send the smell of a flower. Or if your friend’s not in a very good mood it might be a bitter smell.
So the next stage of that is, we’re making devices which will have electrical and magnetic signals being transmitted to your olfactory bulb, which is behind your nose. It’ll be a device which you put in the back of your mouth, it will have magnetic coils, and similar to the electrical taste actuation, it will excite the olfactory bulb using electrical currents. And then this will produce an artificial smell sensation in your brain.
Already scientists have been able to connect optical fibre to neurons of mice, and that means that we can connect electrical signals to neurons.
With the rate of change, for example with Moore’s Law, you get exponential increase in technology. I think within our lifetimes we’re going to see direct brain interface. So in fact what you will get is essentially, you can connect all these signals directly to your brain, and then you will be able to experience a virtual reality without any of these external devices. But essentially connecting to the neural sensors of your brain. And of course that also connects to the internet. So essentially what we will have is direct internet connection to our brain. And I think that will be something we will see in our lifetime.
The New Economy: So direct brain interface – that sounds kind of dangerous. I mean, could there be any side-effects?
Adrian Cheok: Well we’re still at the very early stages now. So scientists could connect, for example, one optical fibre to the neuron of a mouse. And so what it has shown is that we can actually connect the biological world of brains to the digital world, which is computers.
Of course, this is still at an extremely early stage now. You know, the bio-engineers can connect one single neuron, so, we’re not anywhere near that level where we can actually connect to humans. You would have to deal with a lot of ethical and also privacy, social issues, risk issues.
So essentially what we will have is direct internet connection to our brain
Now if you have a virus on your computer, the worst it can do is cause your computer to crash. But you know, you could imagine a worst case: someone could reprogram your brain. So we’d have to think very carefully.
The New Economy: Well why is it important to offer smell over the internet?
Adrian Cheok: Fundamentally, smell and taste are the only two senses which are directly connected to the limbic system of the brain. And the limbic system of the brain is the part of the brain responsible for emotion and memory. So it is true that smell and taste can directly and subconsciously trigger your emotion, trigger your memory.
Now that we’re in the internet age, where more and more of our communication is through the internet, through the digital world, that we must bring those different senses – touch, taste and smell – to the internet; so that we can have a much more emotional sense of presence.
The New Economy: What will this be used for?
Adrian Cheok: Like all media, people want to recreate the real world. When cinema came out, people were filming, you know, scenes of city streets. To be able to capture that on film was quite amazing. But as the media developed, then it became a new kind of expression. And I believe it will be the same for the taste and smell media. Now that it’s introduced, at first people will just want to recreate smell at a distance. So for example, you want to send someone the smell of flowers, so Valentine’s Day for example, maybe you can’t meet your lover or your friend, but you can send the virtual roses, and the virtual smell of the roses to his or her mobile phone.
At the next stage it will lead to, I think, new kinds of creation. For example, music before; if you wanted to play music, you needed to play with an instrument, like a violin or a guitar. But now the young people can compose music completely digitally. Even there’s applications on your mobile phone, you can compose music with your finger, and it’s really professional. Similarly, that will be for smell and taste. We’ll go beyond just recreating the real world to making new kinds of creation.
The New Economy: So will it also have a commercial use?
Adrian Cheok: For advertising, because smell is a way to trigger emotions and memory subconsciously. Now, you can shut your eyes, and you can block your ears, but it’s very rare that you ever block your nose, because you can’t breathe properly! So people don’t block their nose, and that means advertising can always be channelled to your nose. And also we can directly trigger memory or an emotion. That’s very powerful.
You want to send someone the smell of flowers, so Valentine’s Day for example, maybe you can’t meet your lover or your friend, but you can send the virtual roses
We received interest from one of the major food manufacturers, and we’re having a meeting again soon. They make frozen food, and the difficulty to sell frozen food is, you can’t smell it. You just see these boxes in the freezer, but because it’s frozen, there’s no smell. But they want to have our devices so that when you pick up the frozen food maybe it’s like a lasagne, well you can have a really nice smell of what it would be.
The New Economy: How expensive will this be?
Adrian Cheok: We’re aiming to make devices which are going to be cheap. Because I think only by being very cheap can you make mass-market devices. So our current device, actually to manufacturer it, it’s only a few dollars.
The PassWord radio programme is the UK’s only regular-hour long foray into the world of science and technology and not only tells you about the latest developments in the area but what those technologies mean to you.
The radio show is produced by Jane Whyatt, an award winning radio producer for the BBC, and presented by Peter Warren, an award winning investigative journalist who has worked for the BBC and the Sunday Times Insight team, Password is a fascinating insight into the high-tech world that we now all inhabit. To hear our weekly technology radio programme PassWord with Peter Warren tune in to London’s Resonance 104.4FM or www.resonancefm.com at 1530 GMT on Sundays. You can listen live to Radio Castle’s PassWord show on Mondays 1100-1200 online www.radiocastle.com.
Chef Andoni Luis Aduriz at the Mugaritz restaurant is working on a new digital app that lets users recreate one of the restaurant’s dishes on their screen before a device attached to the bottom of the phone kicks out the actual smell of the dish.
The device, called Scentee, is created by a Japanese company and it’s this technology that ProfessorAdrian Cheok from the City University in London harnessed while working with Chef Aduriz.
Scentee is basically a small tank that sits on the bottom of the phone. It’s controlled by an application and once triggered it emits a puff of odor. The idea is that you could one day send someone the scent of a coffee, great wine, or in the case of Mugaritz, actually smell a dish before you taste it.
Cheok says: “The Digital Food app opens up new vistas for people around the world who may not have had the opportunity to physically dine in the restaurant to virtually experience the real smell of gourmet food prepared by one of the world’s top restaurants and chefs. The revolutionary new device brings the sense of smell to mobile phone communications.”
The Scentee device was shown this week at Madrid Fusion, it can be ordered online but it’s hard to find an English or American seller – at the moment the Japanese arm of Amazon stock the devices for ￥3,654 ($35) and the cartridges at ￥525 ($5).The app can be downloaded for iPhone and Androidand the technology really is paving the way for a future of Scentagram style social media apps, supermarkets that let you smell the meals you’re buying and restaurants that trick diners by offering mixed scents and tastes at the table.
Cheok says the early future of the device will be driven by the adoption of private companies. He also notes that it could have wider applications away from the food and drink world, offering real medical benefits, saying that people with smell and taste disabilities could benefit from the technology: “Their experience of the world could be enhanced using technology. It could also be used by using familiar smells to trigger memories for elderly patients, reminding them to do things such as take their medication.”
Even though we are communicating in more ways than ever using online technology, we remain largely confined to the audiovisual when we do it. An app that has been on sale in Japan for some time aims to add smell into the mix.
Now that so many interactions are mediated by means of a screen, we find ourselves behind a window all the time. The virtual world of the internet does not permit us to use all of our senses; touch, taste and smell are off-limits.
The next frontier is allowing people to use all their senses to communicate over the internet and that’s what we’re trying to do at the Mixed Reality Lab, using the Scentee app. So instead of sending a picture of your dinner to a friend, you’ll be able to send them the smell. It can also be synced with your alarm clock to emit a whiff of freshly brewed coffee to get you going in the morning.
The app works when the user presses an icon on the smartphone screen. The app comes with a small tank which is plugged in to the smartphone and will light up and release a puff of scent is released from the top. The individual tanks are each filled with various food aromas so different smells can be sent.
Daily specials, straight to your nose
Given the close relationship between smell and taste, this technology has potential for those who want to change the taste of their food too. If you’re dieting, you might want to spray the smell of beef into the air when you’re eating a salad to trick yourself into thinking you are having something more substantial.
This kind of technology is already being taken up in the restaurant industry, where digital dining is becoming an increasingly popular way to deliver fine cuisine. Customers already get a mixed-media experience in some restaurants, such as when they listen to the sounds of the sea when eating fish.
My team of researchers is working with Mugaritz, a restaurant in Spain, to develop a digital food app that will enable customers to not only see the dishes on offer when they look at the menu but to be able to smell them too. So instead of only relying on traditional audio visual information to pick a meal, auxiliary technology bridges the gap between culinary creativity and customer experience.
We use Scentee to enable them to virtually prepare a recipe from Mugaritz and then smell the results when the aroma from the dish is emitted from the phone.
Private companies are likely to lead the charge in adopting them, as can be seen in the keenness with which restaurants are experimenting.
But there are also applications for health, such as for people with smell and taste disabilities. Their experience of the world could be enhanced using technology. It could also be used by using familiar smells to trigger memories for elderly patients, reminding them to do things such as take their medication.
A TRIPLE Michelin-starred restaurant in Spain is hoping to bring the concept of “Smell-O-Vision” to guests with the development of a new mobile app that will allow diners to get a whiff of their dinner from their smartphone.
Chef Andoni Luis Aduriz of Mugaritz restaurant has teamed up with a Japanese technology company to work on a digital app that would diffuse the smell of the restaurant’s dishes from a small plug-in accessory that attaches to the phone via the earphone jack.
After calling up a dish on the screen, the diffuser called Scentee will release a burst of fragrance from a cartridge that simulates the aromas of the meal.
The concept was unveiled at gastronomy festival Madrid Fusion in January, reports Fine Dining Lovers, and is being developed with a professor from City University in London.
Not only would the mobile app offer prospective diners a preview of their upcoming dinner or a chance to revisit their meal, the digital mobile device could “open up new vistas” for foodies who may not be able to travel to the San Sebastian restaurant, Professor Adrian Cheok told Fine Dining Lovers.
The latest collaboration could also pave the way for future scentagram-style social media apps that could allow friends to send one another scented messages, recreate virtual meals, and allow grocery store shoppers to smell their food before purchase, Cheok added.
For now, the device is pitched as a 4D experience that could be used as an alternative alarm clock – think coffee aromas nudging you awake – and an added layer to the video gaming experience with the smell of gunpowder released every time players click the “fire” button.
Scentee is now available worldwide and ships to 120 countries. The small balloon-like device retails for US$35 (RM117) while cartridges are US$5 (RM16). Aromas that are already available include strawberry, lavender, coffee and rosemary. The app is compatible with both iOS and Android devices.
Chef Andoni Luiz Aduriz of Spain’s three Michelin-starred restaurant Mugaritz is developing a way for diners to smell his food via cellphone, the chef announced at Madrid Fusion last week. Aduriz is working with City University London’s professor Adrian Cheok to develop scents to be used with an add-on device that is operated by the app Scentee.
According to a press release, the app “permit[s] the user to virtually prepare a recipe from the restaurant; the aroma from the finished dish is then released from the phone.” Neat? Cheok tells Fine Dining Lovers that this kind of smellovision “opens up new vistas for people around the world who may not have had the opportunity to physically dine in the restaurant to virtually experience the real smell of gourmet food prepared by one of the world’s top restaurants and chefs.”
It’s unclear when the Mugaritz scents will become available, but the technology is around now. The Scentee device retails for about $35 and cartridges are $5, which, while a little pricey, is still cheaper than going to Spain. The app itself, available for iPhone and Android, is free.
Mobile gadget emitting food flavours will be launched at Madrid Fusion on 28th January.
City University London News Release
Wednesday 22nd January 2014
City University London’s Professor Adrian Cheok spearheads the creation of the revolutionary Digital Food app Mobile gadget emitting food flavours will be launched at Madrid Fusion on 28th January.
City University London announces the collaboration of Professor of Pervasive Computing, Adrian Cheok, in Madrid Fusion on28th January 2014, the most important event in the world of gastronomy.
In Spain’s capital city, Professor Cheok will join leading Michelin starred chef, Andoni Luiz Aduriz, of Mugaritz restaurant, San Sebastian and Dr Luis Castellanos, Founder and President of “El Jardín de Junio”, to unveil Scentee, a new mobile phone app and device that is attached to an Apple iPhone to permit the user to virtually prepare a recipe from the restaurant; the aroma from the finished dish is then released from the phone.
Professor Cheok says the new device will be revolutionary:
“The Digital Food app opens up new vistas for people around the world who may not have had the opportunity to physically dine in the restaurant to virtually experience the real smell of gourmet food prepared by one of the world’s top restaurants and chefs. The revolutionary new device brings the sense of smell to mobile phone communications.”
Professor Cheok, Chef Andoni and Dr Castellanos are proposing novel methods of digital olfaction and gustation for internet interaction, specifically for creating and experiencing the digital representation of food, cooking and recipes.
City researchers are spearheading new methodological approaches for digitally stimulating the sense of taste to enable remote communication through the sense of taste and smell. They aim to obtain a controllable and accurate actuation of taste and smell using digital methods for the benefit of industry and academic research and to improve the lives persons with smell and taste disabilities. The research will also improve the everyday lives of people wanting to further enhance their social networks with the addition of these modalities.
The Digital Food app being released at Madrid Fusion 2014 will lead to new innovations and industries for digital and internet communication, entertainment, office automation, and health therapy, using digitized mobile smell.
In November 2013, Professor Cheok and his research team were presented with the title of Honorary Experts for their participation as finalists in the Telefonica Hacking Bullipedia competition.
Scientists are working on ways that websites and apps can communicate touch, taste and smell. But, wonders Rhodri Marsden, will a hyper-connective multisensory internet be more than we can stomach?
Websites and apps are frequently described by their creators as offering a “rich experience”. The beautiful designs, intuitive layouts and compelling interactivity may well be engaging and satisfying to use, but when they’re hailed as being a “feast for the senses”, it’s evident that they’re a feast for merely two.
Online entertainment is about sight and sound; everything is mediated through a glass panel and a speaker, leaving us well short of being immersed in an alternative reality. But with studies having demonstrated that more than half of human communication is non-verbal, scientists have been working on ways of communicating touch, taste and smell via the internet, and many of those experiments have been gathering pace.
“What do you smell?” asks Adrian Cheok, professor of pervasive computing at City University London. The whiff of melon is unmistakable; it emerged from a tiny device clipped to an iPhone and was triggered by Cheok standing on the other side of the room. “Right,” he says. “These devices have been commercialised in Japan – they’re selling 10,000 units a month – and they’re bringing smells into a social interface.”
It’s still early days with this technology; the device I’m holding is similar to an inkjet printer in that it contains a melon “smell sachet”, and when it’s empty you have to buy another one. Nor is it a particularly new concept. In 1999, Wired magazine ran a front cover story about a company called Digiscents that had produced a USB “personal scent synthesiser” for your computer called the iSmell. Digiscents folded two years later. But the technology that failed to excite us back then now looks slightly less gimmicky in the context of modern smartphone usage, with its super- connectivity and emoticons galore.
On the surface, Cheok’s projects are fun, almost throwaway. “I’ve worked on hugging pyjamas,” he says. “They consist of a suit you can put on your body to virtually hug someone, remotely.
Then we have these small haptic rings; if I squeeze my ring someone else will feel a squeeze on theirs through the internet – like a remote sensation of hand-holding.”
He’s also been working on a device with electrodes that excites taste receptors on the tongue, producing an artificial sensation of taste in the brain.
In the shorter term, the applications of these devices seem slightly frivolous; Cheok’s rings, for example, are being turned into a product that the music industry plans to sell to fans. “You go to the concert,” he says, “the pop star would send a special message, and if you’re wearing the ring you’d get a squeeze on your finger.” I grimace slightly, and he laughs.
“Fortunately or unfortunately,” he says, “that’s where they’ve decided that the money is – but we need to explore the boundaries of how these things can be used, because scientists and inventors can’t think of all the possibilities. For example, Thomson Reuters has been in touch to ask about using the rings to send tactile information about stock prices or currency movements.”
Our transition to an internet of all the senses is evidently dependent on the breadth of information that can be conveyed from one person to another as a series of zeroes and ones. “You have to find a way of, say, transmitting smell digitally, without using a sachet,” says Cheok. “So I’m working with a French neuroscientist, Olivier Oullier, on a device which can produce an artificial sensation of smell through magnetic actuation. The olfactory bulb in our nasal cavity that’s responsible for smell can be stimulated by pulsing magnetic fields. So this is about directly exciting the brain’s neural path by bypassing the external sensor – in this case the human body.”
This immediately plunges us into what seems like incredibly futuristic territory, where brains are communicating sensory information directly with other brains across digital networks. But it’s already been demonstrated by the synthetic neurobiology group at MIT (Massachusetts Institute of Technology) that optical fibre can be connected to neurons, and Cheok is excited about where this may lead in the relatively short term. “We will have direct connection to the brain within our lifetime,” he says, “although what level that will be I’m not sure. Physical stimulation of neurons may not produce the effects that we would hope for and predict.”
Few of us can conceive of the pace with which technological power is developing. Ray Kurzweil (author, futurist, and a director of engineering at Google) predicts that by 2025 we’ll have a computer which has the processing power of the human brain, and by 2045 it’ll have the processing power of six billion brains – ie, everyone on the planet. Cheok sees these as hugely important tipping points for society. “If you’re able to download your brain to a computer, there are major philosophical questions that we’ll have to deal with in the next 30 years, such as whether we’re human, or whether we’re computers.”
Society will also have to work out how it’s going to handle the hyper-connectivity of a multisensory internet – bearing in mind that we can already become deeply frustrated by the few kilobytes of information contained within the average overloaded email inbox. Text messages that are not replied to already provoke consternation – what about unreciprocated touches, provocative odours or unwanted tastes?
“Our brains haven’t changed to cope with infinite communication,” says Cheok. “We don’t have a mechanism for knowing when there’s too much, in the way that we do when we’ve eaten too much food.”
Marshall McLuhan, the Canadian philosopher of communication theory, famously used the term “global village” to describe the effect of connected media upon the world’s population; Cheok believes that new sensory-communication channels will demonstrate how prescient that prediction was. “For most of human history, we didn’t have privacy,” he says.
“Everyone knew who was doing what. And these developments will mean that we become more and more open – the end of secrecy, almost bringing us back to the way that life used to be in hunter-gatherer times. Except, of course, it’s now global.”
The implications of the work of Cheok and his contemporaries seem to sit midway between exciting and terrifying, but in the shorter term it’s about focusing on relatively mundane objectives, such as emitting multiple odours from a smartphone. “People will get used to this new mode of communication,” says Cheok, “and develop new languages. We don’t yet have a language of smell, or of touch; exactly the same pressure in terms of a touch can have a completely different response in the brain, depending on context. But combined with emotion and the subconscious, it’ll bring a heightened sense of presence. I want us to be able to eat together across the internet. I’ve no idea what that will feel like,” he adds, smiling, “but I’ve always believed that human communication goes far beyond the logical.”
After a very surreal chat with a professor at City University London, I was filled in on the concept of multi-sensory human communication. It’s a bit of a mouthful, but Adrian David Cheok, professor of pervasive computing at City University, explained that in the future, the internet will allow communication that goes beyond just vision and hearing. He thinks that in the future we’ll go from sharing data to sharing “experience”.
After our chat, I felt full of hope about what the future would hold. A lot of companies now expect employees to travel abroad, and everyone knows the strain that can have on families and individuals when they can’t properly communicate with those at home. But what if we could taste, smell, hug and kiss via the internet? Professor Cheok explained that 60% of human communication is non-verbal, so although long-distance communication has come a long way, it still doesn’t suffice. Here are some of his ideas and projects for interactive technology in the future:
Sometimes you might have to go for a conference and your partner is at home, in another country, or in another time zone. You have a wandering thought about them and you wished they knew they were on your mind. The RingU was invented for this purpose, a device that allows users to send ‘bi-directional’ visual and physical messages to another paired ring. You can send vibrations and colours to represent your mood and thoughts. It’s not quite the same as being there in person or a phone call, but sometimes you want to send a quick gesture just to let someone know you’re thinking of them. It could even be used in business environments, for example there has been interest in the ring from financial firms who think it would be useful to help traders to receive real-time updates on the stock exchange. The ring could vibrate to inform them of a movement in the stocks they follow.
A lot of current scientific research surrounding taste involves using a mixture of chemicals to produce different taste sensations. Professor Cheok has created a device that uses electrical signals sent to the tongue to manipulate the brain into thinking it can taste certain things. The hope for this is that in the future, taste could be digitised so that people could share what they are eating via the internet. It could also be attached to eating utensils to change the way things taste as you eat them. Imagine eating a virtual lollipop; all the taste and no calories. This research is also linked with directly manipulating sensors in the brain to produce the sensation of taste or smell, similar to a technique already being used to treat depression.
Surprisingly this is something that is currently being produced, and is called ChatPerf. There’s a small device that plugs into your smartphone which can then emit a smell through a mixture of chemicals. If you wanted to share a food smell with someone, you could text it to them. You could sync it with your alarm clock to spray a coffee smell to get you going in the morning. There has even been the idea that you could use it to help you power through a diet, as smell and taste are so closely associated; you can spray the smell of beef to make your salad taste better. It could also be used for healthcare, using familiar smells to trigger memories for elderly patients, reminding them to do things such as take medication. Or it could be used in marketing to promote products such as fabric softener or deodorant.
Cheok is currently working on a ‘bi-directional kiss messenger’ to simulate kissing via the internet. If you’re away from home for a long time, maybe you’re a jet setter, you can use these devices to call home and maybe even get a little kiss from your partner. You each plug the device into your smartphone, and the silicone pads simulate the movement of your partner’s mouth and lips.
Travelling for work can be really difficult, especially if you have kids (or cats) at home who don’t fully understand why you have to be away so often. A prototype device allowed a person to touch sensors on a doll which then transferred the pressure to a jacket on their pet. To widen this interaction to human communication, a ‘hugging pyjama’ was invented to allow the same interaction to take place between a parent and their child over long distance. This device allows you to hug someone at home from wherever you are in the world. Using pressure sensors, the jacket can apply pressure to the body in reaction to where you touch, giving the illusion that you are hugging them.
With businesses struggling to fill job posts and thousands of computer science graduates out of work, universities could play a critical role in averting an IT skills crisis by ensuring the UK is self-sufficient in IT professionals.
“In the UK, just learning the basics of programming is not enough because it has become a commoditised industry. There are hundreds of thousands of graduates in India and China who are really good at programming, so a lot of these things can be outsourced,” he said.
“Graduates in the UK cannot just rely on the technical skills of programming. They have to become much more focused on the networking and business skills required to succeed,” said Cheok.
He said all jobs require human skills and computer scientists must use their people networks: “Every job, no matter what industry, is very much human focused – it is people who control entry to a job, control promotion, control opportunities; computers don’t hire people.
“So it is really critical for people to realise that one of the best ways to improve your career is to leverage your network. Your human network is critical, so when students are at university, it’s essential they build up a network and expand it.
“Almost all opportunities that come about are through people.”
Cheok said universities play an important role and need to change: “Universities must adapt to this too, with courses such as business and computing. Pure computer science is okay if you want to be a researcher or academic. Just knowing how to program in Java and C is no good because anyone can.”
Researchers at City University London, are in talks with the finance sector about using wearable technology to provide trading executives with real-time data 24 hours a day.
The catering and healthcare industries are also interested in using pervasive computing currently in development.
Adrian David Cheok, professor of pervasive computing at the university, said people are currently fully focused on screens for information and there is a limit to what can be absorbed. By sending messages through touch and smell via the mobile internet rather than just audio-visual data, humans can consume more information.
With developments such as big data technology and 4G there is more information available 24/7, but a limited ability to absorb it.
Cheok and his team have developed a ring that can receive a message over the internet. This ring can be connected to an application that monitors big data. If there are changes in things such as stock prices, a message could be sent to the ring through the sense of touch.
“[A finance firm] is looking to use the ring for real-time data for finance professionals because you can’t be in front of your terminal 24 hours a day. But there are certain stocks and indicators they have to always monitor,” said Cheok. “By having something very personal on your body, like wearable technology, 24 hours a day they can, for example, get information about whether a stock is going up or down.
“The thing is we have access to infinite data, but to effectively interact with that data and in the physical world we need to use all of our sense for communication. Basically right now we’re using all of our concentration on screens so there is a limit to how much we can absorb and we can’t always be looking at a screen, you have to do things with your body.”
He said the research team is also talking to a Michelin star restaurant Mugaritz in San Sebastian, Spain, about supporting its advertising.
“The restaurant can fit only a limited number of people in every night, and they want to expand their customer base. How do they do that? They already have a website with photos, but people can’t understand the experience. We’re working with them to make an app, not only will you see the food, you’ll be able to smell it as well. This virtual sense of presence experience so advertising and marketing can benefit,” he said.
“It’s a good example of where audio visual data isn’t enough, if you want have experience of food, then taste and smell are essential, we need to bring in all of the senses, to communicate through the internet, so this is a real-world example of how this could be used,” added Cheok.
Smell is connected to the limbic system in the brain, it can directly trigger memory
-Adrian David Cheok, City University London
The healthcare industry is also looking at technologies being developed by the team. For example, smells can be automatically triggered in the room of a patient as a reminder to take medication.
“Smell is connected to the limbic system in the brain, it can directly trigger memory. We’re discussing with a group working with dementia patients and the biggest problem is they forget their medication and because smell directly affects memory and emotions it can be used to remind patients to take medication,” he said.
Cheok demonstrated a smell being transmitted over the internet to a mobile phone (pictured, above). This uses a chemical pack attached to a phone and a message will trigger a smell. About 10,000 of these have already been sold in Japan and the City University team expects to bring them to the UK soon.
Other innovations in development include a “hugging pyjama” that can be used by parents to hug children when they are not around. The concept could have applications in the care industry. A person hugs a jacket and it sends a message to the jacket being worn by the recipient who feels the hug.
Cheok began looking at augmented reality about 15 years ago when it was still very early research and he wanted to create augmented reality systems. He received a military grant to work on augmented reality for soldiers to help them to understand their environment in urban combat.
Communication is moving beyond barriers, says Rhodri Marsden
04 JANUARY 2014
Websites and apps are frequently described by their creators as offering a ‘rich experience’. The beautiful designs, intuitive layouts and compelling interactivity may well be engaging and satisfying to use, but when they’re hailed as being a ‘feast for the senses’, it’s evident that they’re a feast for merely two.
Online entertainment is about sight and sound; everything is mediated through a glass panel and a speaker, leaving us well short of being immersed in an alternative reality.
But with studies having demonstrated that more than half of human communication is non-verbal, scientists have been working on ways of communicating touch, taste and smell via the internet.
“What do you smell?” asks Adrian Cheok, professor of pervasive computing at City University London. The whiff of melon is unmistakable; it emerged from a tiny device clipped to an iPhone and was triggered by Cheok standing on the other side of the room.
In the shorter term, the applications of these devices seem slightly frivolous. “Fortunately, or unfortunately,” he says, “that’s where they’ve decided that the money is.
“But we need to explore the boundaries of how these things can be used, because scientists and inventors can’t think of all the possibilities.”
Our transition to an internet of all the senses is evidently dependent on the breadth of information that can be conveyed from one person to another as a series of zeroes and ones. “You have to find a way of, say, transmitting smell digitally, without using a sachet,” says Cheok. Few of us can conceive of the pace with which technological power is developing. Ray Kurzweil(author, futurist, and a director of engineering at Google) predicts that, by 2025, we’ll have a computer which has the processing power of the human brain. Cheok sees this as a hugely important tipping-point for society.
Society will also have to work out how it’s going to handle the hyper-connectivity of a multisensory internet – bearing in mind that we can already become deeply frustrated by the few kilobytes of information contained within the average overloaded email inbox.
“Our brains haven’t changed to cope with infinite communication,” says Cheok. “We don’t have a mechanism for knowing when there’s too much, in the way that we do when we’ve eaten too much.
“Communication is not just a desire, it’s a basic need – but we’ve gone from being hunter-gatherers in groups of 20, or 30, to being in a world of infinite data. We could literally gorge on communication and be unable to stop.”
Marshall McLuhan, the Canadian philosopher of communication theory, famously used the term “global village” to describe the effect of connected media upon the world’s population.
Cheok believes that new sensory-communication channels will demonstrate how prescient that prediction was.
“For most of human history, we didn’t have privacy,” he says. “Everyone knew who was doing what. And these developments will mean that we become more and more open; almost bringing us back to the way that life used to be in hunter-gatherer times. Except, of course, it’s now global.”
The implications of the work of Cheok and his contemporaries seem to sit midway between exciting and terrifying, but in the shorter term it’s about focusing on relatively mundane objectives, such as emitting multiple odours from a smartphone.
“People will get used to this new mode of communication,” says Cheok, “and develop new languages.
“We don’t yet have a language of smell, or touch. But, combined with emotion and the subconscious, it’ll bring a heightened sense of presence.
“I’ve no idea what that will feel like, but I’ve always believed that human communication goes far beyond the logical.”