THIS WAS THE FUTURE

Kurzweil AI| November 1, 2011

The Advanced Virtuality Lab (AVL) at the Interdisciplinary Center Israel, is developing a system for controlling a virtual or physical body using only the mind, Israeli Innovation News reports.

The VERE (Virtual Embodiment and Robotic Re-embodiment) project is one of the first to use an fMRI brain scanner to control a computer application interactively in real time, — an innovation which could help severely disabled patients communicate better, says AVL head Dr. Doron Friedman.

“You could control an avatar just by thinking about it and activating the correct areas in your brain,” he said.

Another focus of the AVL is telepresence. The BEAMING (Being in Augmented Multi-modal Naturally-networked Gatherings) project aims to produce the feeling of a live interaction using mediated technologies such as surround video conference, virtual and augmented reality, virtual sense of touch (haptics), and spatialized audio and robotics.

New Scientist | October 12, 2011

A paralysed man has high-fived his girlfriend using a robotic arm controlled only by his thoughts (see video above).

Tim Hemmes, who was paralysed in a motorcycle accident seven years ago, is the first participant in a clinical trial testing a brain implant that directs movement of an external device.

Neurosurgeons at the University of Pittsburgh School of Medicine in Pennsylvania implanted a grid of electrodes, about the size of a large postage stamp, on top of Hemmes’s brain over an area of neurons that fire when he imagines moving his right arm. They threaded wires from the implant underneath the skin of his neck and pulled the ends out of his body near his chest.

The team then connected the implant to a computer that converts specific brainwaves into particular actions.

As shown in this video, Hemmes first practices controlling a dot on a TV screen with his mind. The dot moves right when he imagines bending his elbow. Thinking about wiggling his thumb makes the dot slide left.

With practice, Hemmes learned to move the cursor just by visualizing the motion, rather than concentrating on specific arm movements, says neurosurgeon Elizabeth Tyler-Kabara of the University of Pittsburgh in Pennsylvania, who implanted the electrodes.

After this initial training, Hemmes navigated a ball through a 3D virtual world and eventually controlled the robotic arm, all with his mind. The electrode grid was removed after the 30-day trial.

The team is now recruiting people for a trial of a more sensitive electrode grid that detects messages from individual neurons, rather than a group. They plan to implant two electrode patches, one to control arm movements and another for fine hand motion. The ultimate goal is to allow paralysed people to move individual fingers on a robotic hand.

If you enjoyed this video, watch the first practical demonstration of a mind-controlled robot arm, used by a monkey to feed itself marshmallows.

Wired: Danger Room | October 27, 2010

Controlling robotic limbs with your brain is just step one. The Pentagon eventually wants artificial arms and legs to feel and perform just the same as naturally grown ones. Which means step two is hooking up those prosthetics directly into severed nerves. That’ll allow the wearer to detect subtle sensations, respond to the brain’s neural signals, move with unprecedented agility, and “incorporat[e] the limb into the sense-of-self.”

Over the last decade, the Pentagon has made remarkable progress in creating life-like prosthetic devices. And most of the advances are because of programs funded by Darpa, the far-out military research agency that’s also behind this latest project, called Reliable Peripheral Interfaces (RPI).

Already, Darpa has funded ventures like the DEKA Arm, which relies on a joystick-style interface, and used “targeted muscle reinnervation surgery” for prosthetics that transmit neural signals from a bundle of nerves in the chest. Darpa-funded researchers at Johns Hopkins have even started human trials on their Modular Prosthetic Limb, which transmits cues to an artificial limb using brain-implanted micro-arrays.

But the RPI program taps into key shortcomings that persist in even the most sophisticated prosthetic devices. Existing neural-prosthetic interfaces aren’t sensitive enough to provide myriad signals — prototypes currently transmit around 500 events a second — or offer users a robust degree of freedom. Not to mention that current neural platforms have short life spans and are tough to repair without invasive surgery, making them ill-suited to troops and vets in their 20s.

So Darpa’s after a prosthetic that can record motor-sensory signals right from peripheral nerves (those that are severed when a limb is lost) and then transmit responding feedback signals from the brain. That means an incredibly sensitive platform, “capable of detecting sufficiently strong motor-control signals and distinguishing them from sensory signals and other confounding signals,” in a region packed tightly with nerves. Once signals are detected, they’ll be decoded by algorithms and transmitted to the brain, where a user’s intended movements would be recoded and transmitted back to the prosthetic.

The end result would be a prosthetic that acts as a veritable extension of one’s own body. And a platform capable of accurately distinguishing between, and interpreting, different sensory signals — temperature, pressure, motion — would “allow the incorporation of the limb into the sense-of-self” and offer unprecedented freedom of movement for a prosthetic wearer.

The agency also wants an ultra-reliable platform, with an error rate of less than 0.1 percent and a lifespan of around 70 years. By comparison, current neural-recording interfaces last around two years before they need to be replaced. Sounds far-fetched, but Darpa’s already got one major lead: The agency’s new Neurophotonics Research Center will investigate fiber-optic prosthetic interfaces that can incorporate thousands of sensors into a single filament.

New Scientist | October 28, 2010

IMAGINE being able to manipulate images on a screen by thought alone. That’s the tantalising prospect raised by a brain-machine interface that lets you control which of two competing images you can see on a screen.

Moran Cerf at the California Institute of Technology, Pasadena, and colleagues recruited 12 volunteers who had electrodes implanted in their brains to record epileptic seizures. That meant the team could record activity in the normally inaccessible medial temporal lobe. The MTL houses the hippocampus and amygdala, which are involved in memory and emotions.

Cerf first talked to each person about their interests and recent experiences – the concepts making up their recent memories.

His team then created a database of images to correspond to those concepts, such as a picture of the Eiffel Tower for a person who recently visited Paris, France. Next each person had their brain activity recorded as they looked at 100 of those pictures six times. The team could then identify the individual neurons that fired in response to each image.

The team then chose two images for each person that caused firing in different individual neurons. The two pictures were superimposed, and the person asked to control the strength of each image by focusing on the concepts related to each picture. A brain-machine interface translated the neural activity related to an image so that it was portrayed more or less vividly on the screen. In 70 per cent of tests, volunteers were able to bring one image to the fore (Nature, DOI: 10.1038/nature09510).

Although humans and monkeys can already control the movement of a robotic arm via the brain’s motor cortex, this is the first time anyone has been able to tap into the neurons associated with individual concepts.

Fabien Lotte at the Institute for Infocomm Research in Singapore says the findings could be useful clinically. “A paralysed person could increase the activity in a neuron that corresponds to the person they want to call, for example,” Lotte says.

New Scientist | November 3, 2010

Groups in Germany and the US have been testing electronic implants aimed at restoring vision to people with retinal dystrophy. The condition is hereditary or age-related, and causes degeneration of the photoreceptors – light-sensitive cells in the retina – leading to blindness. It affects 15 million people worldwide.

Eberthart Zrenner and colleagues at the University of Tübingen in Germany have developed a microchip carrying 1500 photosensitive diodes that slides into the retina where the photoreceptors would normally be. The diodes respond to light, and when connected to an outside power source through a wire into the eye, can stimulate the nearby nerves that normally pass signals to the brain, mimicking healthy photoreceptors.

The team reports that their first three volunteers could all locate bright objects. One could recognise normal objects and read large words.

Nerves in the eye normally adapt to visual input and stop transmitting signals after a short time. Tiny movements of the eye overcome this by constantly projecting the image back and forth between neighbouring nerve cells so that each has time to recover and resume transmitting signals. Because the implant is inside the eye, this mechanism worked normally in the trials. Another device being tested sends images from a head-mounted camera to ocular nerves, but as the image forms outside the eye the tiny movements cannot maintain it and patients must rapidly shake their head instead.

As a safety precaution, the implants in this first pilot study were removed after several weeks, says Walter Wrobel, head of Retina-Implant, a company based in Reutlingen, Germany, formed by the researchers to eventually market the implant. “Based on the results of this study, we have designed a new system, which is being implanted permanently, or as long as patients like it.”

In the new system, the power source connects to the retinal implant via a mechanical coupling through intact skin, not via a wire through an incision in the skin as the earlier system did. “That means they can shower easily, leave the hospital and go around town on their own,” says Zrenner. “They can go out for a meal, and really see things, like a nice glass of beer.”

The Australian | February 12, 2010

A ROBOT hand that could defuse bombs and luminous goo that flows around soldiers’ moving bodies but hardens against bullets sound like they should have been dreamt up for a James Bond film.

Among the companies to have received funding is d3o Lab, which has developed an “intelligent” liquid polymer that is easily malleable when moved slowly but whose molecules lock together and absorb force when hit by a projectile.

To demonstrate its properties, which have uses in a new generation of helmets and types of body armour, members of the d3o team wrapped the luminous orange goo round their hands and then struck them repeatedly with hammers. “The way that the material responds to your body movements, you get a duality of flexibility and protection,” Floria Antolini said.

Other innovations on show included the “Little Owl”, a lightweight drone that its developers hope will carry 20kg of surveillance equipment and power itself on solar energy for up to three months. The project has received £44,000 of funding for development.

Intelligent Textiles received £49,500 to help it to develop an army uniform that conducts electricity and computer data through internally woven “conductive yarns”. It allows troops to attach electrical equipment to powerpoints on their uniform, and to run internal heaters to keep them warm. The uniform runs off a central battery pack.

The MoD believes that it will remove 2kg from the weight of equipment for troops in combat and is also being considered by the Canadian military. Even if the material is pierced, officials claim, it will still conduct electricity around the damaged area without loss of power.

Crib Gogh, which makes extreme survival equipment, has already developed a durable solar-panel mat that folds into a backpack pouch and delivers enough power to run a computer. It is to be delivered to forward bases in Afghanistan.

Currently the MoD runs small generators at such bases whose fuel costs are 17 times the market price because of transport costs.

The MoD also spends millions of pounds on the purchase and transport of batteries for soldiers who use an average of eight AA batteries a day.

In a corner of the event, Sergeant Alex Simpson, 26, a bomb disposal expert with 11 Explosive Ordnance Disposal Regiment, the unit that tackles Taleban roadside bombs, tested a robotic hand developed by the Shadow Robot Company.

The hand mimics the movements of a controller wearing a sensory glove and can be used at a range of hundreds of metres. “Without a shadow of a doubt this could be used in the bomb disposal world,” he said, “and it would obviously be a massive leap forward from what we have at the moment.”

“Thanks man,” said the hand’s dreadlocked designer, Rich Walker, 39, a self-confessed Dungeons and Dragons fan. The company is also developing robotic limbs for use in radioactive environments. He expects the hand to be picking apart bombs on the battlefield within two years.

New Scientist | 7 April, 2009

Canadian filmmaker Rob Spence damaged his right eye in a childhood accident and was later given a prosthetic replacement. Like any other false eye, it was designed to be purely an aesthetic replacement, but he realised that the vacant bit of face real estate could be put to better use in his art.

Now Spence is attempting to build a wireless video camera into his synthetic eye, turning himself into a self-proclaimed “Eyeborg“.

The camera will record anything and anyone that enters Spence’s field of vision and relay the footage back to a computer. That video will provide a unique perspective on the way video surveillance is becoming more popular in western societies, he told New Scientist.

This week, Spence and engineer Kosta Grammatis have succeeded in placing a working red LED in Spence’s eye (see image, above right), giving him a look not dissimilar from Arnold Schwarzenegger’s cyborg in The Terminator.

Although this is a diversion from the main goal, it is the first time the team has fitted a working electronic device, complete with power source, into the eye socket, says Spence.

An LED may even be fitted alongside the camera in the final Eyeborg prosthetic if the battery can spare enough power, he adds. Apart from the aesthetic value, it could provide lighting in dark conditions.

Earlier I posted an article about the University of Tsukuba’s robotic exoskeleton designed to assist elderly peeps with strength and mobility. I’ve tracked down the completed project which has been officially dubbed HAL (Hybrid Assistive Limb). Now, I’m not sure what frightens me more:

a) The idea of supersuit alà Ironman that increases its host’s strength by 5 fold

b) That the motors apparently respond faster to signals from the wearer’s brain than their own muscles… which by default means the natural body is the disadvantaged, i.e. the disabled, or

c) That they gave it the same god damn name as a robot famous for undermining its human hosts and who tried to kill them all

My irony implant just exploded.

To look into the face of certain doom see below:

Exoskeleton for grannies

Finding ways to assist and care for the growing elderly population in many developed countries is a growing problem. One challenge is to work out how to improve the strength and utility of ageing limbs.

Yoshiyuki Sankai at the University of Tsukuba near Tokyo, has developed an exoskeleton for a single arm that can do just that.

The device consists of a tabard worn over the shoulders with a motorised exoskeleton for one arm attached. The exoskeleton senses the angle, torque and nerve impulses in the arm and then assists the user to move his or her shoulder and elbow joints accordingly.

Read the full arm exoskeleton patent application.

Robo-monkey uses brain power to feed itself

Newscientist|28 May 2008

Look Mum, no hands! Two monkeys have managed to use brain power to control a robotic arm to feed themselves. The feat marks the first time a brain-controlled prosthetic limb has been wielded to perform a practical task.

Previous demonstrations in monkeys and humans have tapped into the brain to control computer cursors and virtual worlds, and even to clench a robot hand. But complicated physical activities like eating are “a completely different ball game”, says Andrew Schwartz, a neurological engineer at the University of Pittsburgh, who led the new research.

Tests with humans are being prepared in numerous labs, but experts caution that brain-controlled robotic limbs are far from freeing paraplegics from their wheelchairs or giving amputees their limbs back.

Wired for action

Most people who become paralysed or lose limbs retain the mental dexterity to perform physical actions. And by tapping into a region of the brain responsible for movement – the motor cortex – researchers can decode a person’s intentions and translate them into action with a prosthetic.

This had been done mostly with monkeys and in virtual worlds or with simple movements, such as reaching out a hand. But two years ago, an American team hacked into the brain of a patient with no control over his arms to direct a computer cursor and a simple robotic arm.

Schwartz’s team extracted even more complicated information from the brains of two rhesus macaques by reading the electrical pulses of about 100 brain cells. Normally, millions of neurons fire when we lift an arm or grab a snack, but the signals from a handful of cells are enough to capture the basics, Schwartz says.

Up for grabs

His macaques controlled a robotic arm that moved at the shoulder and elbow and could clench and open its hand.

To train the monkeys, the researchers first recorded their brain activity as they controlled the robotic arm with a joystick. Once the monkeys had learned to feed themselves in this way, Schwartz’s team secured their arms and made them rely on controlling the robot with their brain.

To avoid frustrating the animals during their first attempts, the researchers partially guided the robot themselves. Gradually, these training aids were dispensed with, and after three weeks the monkeys had mastered the robotic arm.

In tests where a monkey had to grab marshmallows or grapes and feed himself, one monkey succeeded 61% of the time, often reaching for another treat while still chewing on the last one. The animals manoeuvred the arm around obstacles and readjusted its path when researchers moved the food.

Thought control

“It’s impressive how naturally the animal interacts with the robot,” says John Kalaska, a neuroscientist at the University of Montreal. “It’s a natural extension of their own body because they control it so easily just by thinking.”

He says Schwartz’s gradual and assisted approach to training the monkeys is likely to work with neural prosthetics in human patients.

However, treatments for people with disabilities are still years away. Schwartz’s robotic arm requires computers, bulky equipment and a technician, and brain-implanted electrodes may not last a lifetime. And ideally, a prosthetic would send tactile sensations back to the brain – a challenge scientists are only beginning to tackle.

“It’s a long way yet to go,” Kalaska says.