A Novel Functional Electrical Stimulation-Induced Cycling Controller Using Reinforcement Learning to Optimize Online Muscle Activation Pattern

Sensors, 2022 · DOI: 10.3390/s22239126 · Published: November 24, 2022

Assistive Technology Bioinformatics Biomedical

Simple Explanation

This study introduces a new controller that uses Reinforcement Learning (RL) to adjust how muscles are stimulated during FES-cycling in real-time. The system learns by trial and error to change the electrical charge applied to muscles, following a plan and keeping track of pedaling speed. The goal is to adjust the electrical charge to match the changing needs of the muscles, rather than using a fixed stimulation pattern.

Study Duration

Not specified

Participants

One participant with complete spinal cord injury (SCI T8, ASI A)

Evidence Level

Not specified

Key Findings

1
The participant with spinal cord injury was able to pedal overground for distances over 3.5 km using the developed system.
2
The RL agent learned to modify the stimulation pattern as intended while simultaneously maintaining a desired pedaling cadence.
3
The method can be used to reduce the time needed to define stimulation patterns for FES-cycling.

Research Summary

This study introduces a novel controller based on a Reinforcement Learning (RL) algorithm for real-time adaptation of the stimulation pattern during FES-cycling. The participant was able to pedal overground for distances over 3.5 km, and the results evidenced the RL agent learned to modify the stimulation pattern according to the predeﬁned policy and was simultaneously able to track a predeﬁned pedaling cadence. Our results suggest interesting research possibilities to be explored in the future to improve cycling performance since more efﬁcient stimulation cost dynamics can be explored and implemented for the agent to learn.

Practical Implications

Reduced Implementation Time

The use of learning methods would facilitate the implementation of FES-assisted modalities as the complexity of the process of defining the initial stimulation parameters is reduced.

Optimized Electrical Charge Injection

The method has the potential to create interesting research possibilities, such as for example, optimizing the injection of electrical charge to make more efficient the stimulation cost in order to delay the muscle fatigue process.

Personalized Stimulation Patterns

The system can adapt stimulation parameters independently for each session, adjusting for the physiological characteristics of that moment.

Study Limitations

1
Muscle fatigue is yet a limitation to the use of electrical stimulation to assist functional activities and consequently a limitation to the session duration.
2
The time it takes the agent to learn the policy is related to the algorithm settings and to the number of actions available for him to learn.
3
It was necessary to evaluate different parameter settings for the RL algorithm (number of interactions, episode duration, exploration rate, time evaluating rewards, step size for pulse width and amplitude etc.) and policies for the reward-interaction dynamic between the agent and the environment, which is part of the process of tuning the algorithm.

Your Feedback

Was this summary helpful?

Back to Assistive Technology