Key challenges of the fusion between neuroscience and robotics

Combining neuroscience and robotic research has gained impressive results in the rehabilitation of paraplegic patients. A research team led by Prof. Gordon Cheng from the Technical University of Munich (TUM) was able to show that exoskeleton training not only helped patients to walk, but also stimulated their healing process.

With these findings in mind, Prof. Cheng wants to take the fusion of robotics and neuroscience to the next level.

Prof. Cheng, by training a paraplegic patient with the exoskeleton within your sensational study under the “Walk Again” project (LINK!), you found that patients regained a certain degree of control over the movement of their legs. Back then, this came as a complete surprise to you …

… and it somehow still is. Even though we had this breakthrough four years ago, this was only the beginning. To my regret, none of these patients is walking around freely and unaided yet. We have only touched the tip of the iceberg. To develop better medical devices, we need to dig deeper in understanding how the brain works and how to translate this into robotics.

In your paper published in Science Robotics this month, you and your colleague Prof. Nicolelis, a leading expert in neuroscience and in particular in the area of the human-machine interface, argue that some key challenges in the fusion of neuroscience and robotics need to be overcome in order to take the next steps.

One of them is to “close the loop between the brain and the machine” – what do you mean by that?

The idea behind this is that the coupling between the brain and the machine should work in a way where the brain thinks of the machine as an extension of the body. Let’s take driving as an example.

While driving a car, you don’t think about your moves, do you? But we still don’t know how this really works. My theory is that the brain somehow adapts to the car as if it is a part of the body. With this general idea in mind, it would be great to have an exoskeleton that would be embraced by the brain in the same way.

How could this be achieved in practice?

The exoskeleton that we were using for our research so far is actually just a big chunk of metal and thus rather cumbersome for the wearer.

I want to develop a “soft” exoskeleton – something that you can just wear like a piece of clothing that can both sense the user’s movement intentions and provide instantaneous feedback. Integrating this with recent advances in brain-machine interfaces that allow real-time measurement of brain responses enables the seamless adaptation of such exoskeletons to the needs of individual users.

Given the recent technological advances and better understanding of how to decode the user’s momentary brain activity, the time is ripe for their integration into more human-centered or, better “brain-centered” solutions.

What other pieces are still missing? You talked about providing a “more realistic functional model” for both disciplines.

We have to facilitate the transfer through new developments, for example robots that are closer to human behaviour and the construction of the human body and thus lower the threshold for the use of robots in neuroscience.

This is why we need more realistic functional models, which means that robots should be able to mimic human characteristics. Let’s take the example of a humanoid robot actuated with artificial muscles.

This natural construction mimicking muscles instead of the traditional motorized actuation would provide neuroscientists with a more realistic model for their studies. We think of this as a win-win situation to facilitate better cooperation between neuroscience and robotics in the future.

You are not alone in the mission of overcoming these challenges. In your Elite Graduate Program in Neuroengineering, the first and only one of its kind in Germany combining experimental and theoretical neuroscience with in-depth training in engineering, you are bringing together the best students in the field.

As described above, combining the two disciplines of robotics and neuroscience is a tough exercise, and therefore one of the main reasons why I created this master’s program in Munich. To me, it is important to teach the students to think more broadly and across disciplines, to find previously unimagined solutions.

This is why lecturers from various fields, for example hospitals or the sports department, are teaching our students. We need to create a new community and a new culture in the field of engineering. From my standpoint, education is the key factor.

---

Bionic limbs terminology, existing solutions and current pains
The term ‘bionics’ was first used by Jack E. Steele in the US TV show – ‘The Six Million Dollar Man and Bionic Woman’, in which superpowers were imparted to the protagonists by electromechanical implants. Afterwards, this term earned widespread use in literature and television.34 In current terminology, it mainly addresses devices that make a direct connection with the residual nervous or muscular system of the impaired individuals.

There is a difference in the role and, consequently, construction of bionic limbs for the upper (including hand) and the lower extremities. The functions of the upper limbs (UL) and the lower limbs (LL) differ, and the role of and need for limb replacement in these cases are different; therefore, careful evaluation of the needs and the remaining capacity of patients must be considered during the construction of aprobable bionic limb. The situation in upper limb amputation, hand or forearm, is the most complex, since the hand represents the highest level of evolution with sophisticated and unique functions.

Its control of 40 muscles and the involvement of a large surface of the brain cortex, suggests its significant role and importance in human performance. Present commercial prostheses are failing in replicating such control of the actuation or sensing capability.3,4,35,36 Conversely, the LLs are used for standing, walking, ensuring stability and balance.

This is made possible after a transtibial amputation, using the modern below-the-knee prostheses.6,25 Such patients can walk, dance, and play sports at near-normal levels. However, those undergoing high transfemoral (thigh-level) amputations do not regain normal gait and balance, and are at risk of falling and overloading the opposite, healthy leg. Several long-term problems, including osteoporosis, arthritis, back pain, and increased metabolic consumption (with possible disastrous outcomes) frequently occur in these patients.37,38

Bidirectional control
Many efforts have been made to solve a number of technical problems, which were present in prosthetic devices. Batteries are today long-lasting and energy consumption for these limbs is lower.3,39,40 Biological residuum and electronic devices interface through the placement of parts of a machine in direct connection with the human body in order to enable bidirectional communication between the electronic signals and ionic currents within the living organism.34

Actually, a bionic limb is denominated as such, thanks to the inclusion of the hardware that acts as an interface between the residual human nervous system and the device (such as a robotic hand or leg). Novel surgical techniques have improved the efficacy of these technologies interfacing them with several muscular 41–44 and nervous structures,2,8,9,27,28–31,45,46 in a more intimate way.

These include the muscle direct approach through the injections of small implants,42,47 nerve rerouting for the muscular reinnervation,18,41,43,44,48–50 and nerve interfacing around,2,9 or within8,27,28–31,45 the fascicular structures.

A bionic limb is controlled by the electric signals from the muscle and/or nerves above the level of the amputation. Bidirectional control is then completed via sensation restoration through the connection of the remaining nerves or muscles above the level of amputation to the prosthetic device sensors.

Therefore these devices enable both intuitive control and natural flow of sensation from the artificial device to the user (Fig. 2). The first successful proof of concept was achieved with bionic hands.7–9,41,42,45,48 Modern hand prostheses are actuated by advanced motors, enabling the restoration of sophisticated hand movements, through the connection with direct muscular signals.31,42,50

In order to restore the sensory information flow, the signals from tactile and position sensors embedded in the prosthesis are converted into electrical impulses by sensory encoding algorithms implemented on a system controller.8,9,28,30,46 Then, the stimulation trains are delivered to the nerve, using a neural stimulator by means of microelectrode implants previously fixed into or around the somatosensory nerves. In this way, users could perceive sensations directly on the phantom limb according to the interactions between prosthesis and external world, being again masters of the space around them.

Implants, nerve and muscles transferring
The neural signal pick-up can be achieved by exploiting the natural nerve motor signal amplification that is obtained on the neuromuscular junction, therefore placing the recording electrode on the surface of the muscle50,51 (surface electromyography (EMG)) or inside the muscle42,52 (intramuscular EMG). Moreover, the motor intentions potentially could be also recorded directly from the peripheral nerves27,31,53 (electoroneurography (ENG)) in order to be more selective. Yet the last approach suffers from difficulties with long-term stability and reliability.4,5,27,31

Bionic limbs can be divided into three main groups, according to the implant used, the type and the tissue interfaced:

1. Nerve and muscle transferring
2. Direct muscleinterfacing
3. Direct nerveinterfacing

Nerve and muscle transferring
Targeted muscular reinnervation (TMR) invented by Kuiken41,43 involves the transfer (rerouting) of the remaining nerves (e.g. median and ulnar nerve) of the amputated stump to the available muscles (e.g. chest muscles), thus amplifying neural control signals via their natural muscular amplifier.

Those signals are then registered by the electrodes and transferred to the prosthesis to control its action.41,43 Indeed, when a subject thinks of moving his/her missing hand, the reinnervated chest muscles are stimulated and the signal is then captured by recording electrodes and used to drive the movement of the robotic arm.41,50 The same approach was also applied to lower limb amputees.44 Additionally, tactile stimulation over the reinnervated areas (e.g. chest) can induce the sense of touch of the missing arm/fingers.18,48

However, when trying to implement real bidirectional control, it is yet impossible to record the signals from the innervated muscle and, at the same time, implement the sensory touch-feedback, since the same area needs to be approached, possibly due to the sensory gating problem.52 Recently, this issue was tackled, achieving reinnervation of separated motor and sensory fascicles over different muscles.18

This approach shows promise for the success of such a bidirectional system. The targeted muscle reinnervation approach is an excellent solution, especially for very high amputees (e.g. shoulder disarticulation or transhumeral amputation of the arm).

Recently, an elective amputation, combined with the techniques of selective nerve and muscle transfers and prosthetic rehabilitation to regain hand function, have also been proposed in three patients with brachial plexus injuries.7 On a similar track, Herr and colleagues6, recently proposed the so-called agonist-antagonist myoneural interface (AMI).

AMI is a new idea encompassing a surgical construct made up of two muscle tendons – an agonist and an antagonist – surgically connected in a series so that contraction of one muscle stretches the other.

The idea of the AMI is to recreate the dynamic muscle relationship that existed within the pre-amputation anatomy, thereby allowing proprioceptive signals from both muscles to be transferred to the central nervous system. Herr and his team surgically constructed two AMIs within the residual limb of a subject with a transtibial amputation, achieving very promising results.6 Such an elegant surgical approach appears to be very promising in transtibial amputees, while it could be more difficult to apply in transfemoral patients.

Direct muscleinterfacing
In the second type of bionic limbs, the approach to the control signal captured from the residual muscular tissue is made through direct intramuscular implants.42,47 Intramuscular implant-based control consists of small recording devices implanted into the residual muscle to record muscular contractions, which are then wirelessly captured through a coil integrated in the socket. Muscular contractions then actuate the prosthesis movement.

In the case of upper limb amputees,42 control has been achieved over simultaneous grasp and wrist movements; whereas a previously unseen, voluntary control of the ankle motion has been achieved in lower limb amputees.47 Yet, sensory feedback is not available with this solution.

The drawback is that this approach can work better in the case of more distal amputations (low transradial or transtibial), when many of the extrinsic muscles have been preserved, while in more proximal amputations (were muscles are missing) it would be difficult to implement. However, in higher (more proximal) amputations it could possibly be combined with the surgical techniques described above.

Direct nerveinterfacing
The third option involves the direct interfacing of the residual nerves using implantable peripheral neural interfaces.35,36 This may be achieved by means of the neural electrodes going around or through the nerve. It is thus possible to enable control of the device41 or to impart a sensation from the device.8,9,54,55 Actually, transformed electric signals from prosthetic sensors stimulate the nerves in the stump, restoring sensation in the phantom limb, and thus allowing the patient to ‘feel’ once more.27,45

The third group of bionic limbs incorporates the sense of the absent extremity via electrodes implanted surgically in the residual nerves, which innervate the UL or LL. To regain and improve bionic limb sensibility28–31,56 the electrodes are introduced and placed intraneurally through the fascicles,5,8,28–31,45 or around the nerves by means of an epineural cuff.2,9,46 This has its rationale, since the peripheral nerve is positioned transversally from the topographic aspect, thus enabling different structures to be successfully stimulated through the device pinching the nerve transversally.

Investigations suggest that intraneural stimulation can revive neural paths and improve control of an artificial limb through very short learning and training processes.8,28,30

This is achieved by the process of decoding motor intention from the remaining muscles and encoding the sensation with electric nerve stimulation through the electrodes,8,28 which are placed through the nerve during the intraoperative procedure.8,27,28

In specific studies,8,28–30 the intraneural implants (two in each median and ulnar nerve) bear external wires that are connected to the artificial touch sensors and a neural stimulator of the bionic limb.

This enables them to send impulses to the brain by a process of mapping what patients feel and detect when touch is executed over a certain area of the sensorized prosthesis. These patients exhibit remarkable dexterity8,28–30 and even texture recognition.57 Simultaneously, due to the physiologically plausible afferent drive restoration, phantom pain decreased.28,56,58–60

Preliminary trials seeking to combine osteointegration and neural interfacing into a fully portable and self-contained bionic device have also been performed.2

Surgical procedures
Correct interfacing of residual nerves (Fig. 3) is critical. In such case, the surgeon must take extreme care to do the following:

(a) Target the proximal nerve area, free of any neuro degeneration (e.g. the valerian nerve).

(b) Competently place and fix the interface and cables,while retaining movement of the arm/leg and nerve.

(c) Avoid excessive neural damage.

The surgical procedure for electrode implantation is performed in a limited number of cases.2,8,9,27,28,31 We have trained in the implantation of TIMEs61 (transversal intra-fascicular multichannel electrodes) in the median and ulnar nerve of the upper and sciatic nerve of the lower limb of cadavers. The surgical approach to the both UL and LL nerves is direct. The nerves of interest are the median and the ulnar nerve of UL and the sciatic nerve (tibial nerve) for LL. Skin incision and separation of muscles from other soft tissues should be gentle in order to prevent scarring and fibrosis.

Haemostasis must be meticulous to reduce interference with electronic signalling, oedema and infection. Also, special attention must be paid to nerve preparation. As it is crucial to preserve the epineural tissue and fine vascular structure, electrodes must be placed only after mapping the fascicular structure. After a gentle opening of an external neural sheet, it is advised to access fascicular structures (Fig. 4a).

Electrodes should be perpendicularly inserted into the nerve through as many fascicules as possible to obtain contact with the active sites of the electrodes (Fig. 4b). By pulling the straight needle with an 8-0 suture the electrode could be placed into the nerve. Then the electrode is fixed with sutures through the fixation tabs with holes to the surrounding epineural tissue (Fig. 4c). The electrode structure is fragile and breakage must be avoided so technique must be meticulous. After electrodes are placed and secured at three levels, a subcutaneous tunnel should be created for the cable and connector towards the neurostimulator. This surgical procedure is a demanding one, and requires an experienced microsurgeon to perform it properly. It enables stable fixation of electrodes and cables, and is suitable therefore for long-term use.

reference link: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7047902/

---

Source:TUM