Our Project

4DAssist will lighten up lives

There are approximately 55 million family members and loved ones around the world currently suffering from dementia – and these numbers are growing.

There is presently no cure for dementia and the majority of these individuals need a lot of daily help from caregivers for basic activities of daily living. Since there’s no pharmacological silver bullet to cure dementia, there’s growing interest and success in using non-pharmacological methods to at least slow down dementia’s rate of progression.

 

Recent research has demonstrated that engaging in “task-oriented training” or “activity-based therapy,” which involves following a series of steps to complete a task, can enhance cognitive function in individuals with dementia. Additionally, this type of task-oriented activity can help maintain the ability to perform these tasks independently for a longer period of time.

 

4DAssist aligns perfectly with this approach and aims to enhance the quality of life for both individuals with dementia and their loved ones.

The Price is Right

Augmented Reality smartglasses up to now haven’t been used to assist in this field for two main reasons: they haven’t been in existence very long, and the capabilities and features required to help individuals with dementia have only been available in the very expensive AR headsets such as the Microsoft Hololens and the Magic Leap One.

 

Fortunately, the consumer-priced Nreal Light AR smartglasses became available in 2020 and they have the right set of features that makes 4DAssist possible. It’s very encouraging that the Nreal Light is approximately only 10% of the cost of the earlier AR headsets!

What will the User see?

User selects an Activity to be guided through (by voice command):

Activities of daily living (ADL) are the fundamental tasks required to independently care for oneself. These include grooming, dressing, toileting, mobility, feeding, and bathing. 4DAssist, however, will be providing guidance for much more than just the basic ADLs and so we will be referring to these as simply ‘Activities’.

 

Once the 4DAssist app has been redeveloped for the consumer–priced Nreal Light smartglasses, the ongoing creation of Activities will be the major task for the 4DAssist team. We’ll be creating new activities in coordination with our Dementia specialists and choosing the highest priority activities needed. And, we’ll be listening to you for specific requests for your loved ones.

The Augmented Reality smartglasses are light and comfortable:

The Nreal Light is a lightweight pair of Augmented Reality (AR) smartglasses that are comfortable to wear and designed to give you a true 3D AR multimedia experience. Weighing only 88 grams (3.1 oz.), they can be worn for hours without the typical fatigue that’s common with VR headsets.

 

The Nreal Light has built-in speakers in the frames close to the ears and microphones as well. A built-in USB cable is used to plug into an Android phone that provides the power and processing capabilities for the device. The glasses are designed to accept prescription lenses can can easily be inserted behind the main lenses.

The AR smartglasses are the enabling component of 4DAssist:

The Nreal Light AR Smartglasses allow hands-free operation for the user as they accomplish any of the steps in a particular Activity. 4DAssist uses voice recognition and very simple voice commands to select activities and steps, and to control its operation.

 

And since they’re powered by an Android smartphone, the phone can be worn in any convenient shirt pocket, sweater or pants pocket and so it will not interfere with performing any of the steps in an activity.

An Activity can have as many steps as needed for the best possible guidance:

When an Activity is selected, the first step of that Activity is displayed and an optional automatic voice narration is started. Each step has images associated with the particular step as well as a video that clearly shows how to perform the step.

 

An exciting planned feature is using an Avatar who will in turn perform the step. The user will be able to pause and walk around and get as close to the Avatar from any angle to hone into a particular view. This is a remarkable feature of 4DAssist and the ADL’s we’re crafting with the experienced dementia care team will greatly simplify the performance of any Activity.

Example sequence of steps for the ‘Making a cup of Tea’ Activity:

Example Step

The activity ingredients have been put on the table in a previous step. The user is  instructed to get the box of  tea.

Performing the Step

The user performs the step of getting the box of tea, then moves to the next step by saying “Next Step”.

Next Step

This process is repeated until the activity is completed.

4DAssist can conveniently access the plethora of helpful content on the Internet:

There’s a plethora of meaningful activities available on the YouTube for people dealing with dementia – early, middle, and late stages. And there’s community centers with caregivers that explain each step of an activity and share their experience in helping your loved one.

 

There’s also a wide variety of card games, puzzles, and other types of games that are designed for older and dementia-troubled individuals and these can be accessed directly with 4DAssist.

 

So the added benefit is that individuals using 4DAssist AR Smartglasses can engage in these outside activities, as well as watch YouTube videos and listen to music, without having to remove the glasses and use another device.

Caregivers will benefit from the user’s immersive experience:

People with dementia generally require high levels of care, most of which is provided by informal or family caregivers. Individuals with dementia would have a poorer quality of life and would need institutional care more quickly without caregivers.

 

Unfortunately, this support comes at a cost of caregiver distress and poorer quality of their life. Caregivers face many obstacles as they balance care-giving with other demands, including child rearing, career, and relationships. Fortunately, there are numerous communities and resources available to assist with the caregiver’s needs.

 

4DAssist helps by engaging the dementia individual with immersive guided activities that will consequently give the caregiver much needed breaks.

Some operational and tech details:

4DAssist’s Activities consist of a varying number of steps, which may vary for each Activity. Each Activity follows the same internal structure, which includes the following elements for each step:

  • Text: a description of the step for the user

  • Audio: an audio recording of the text

  • Image: an image showing the completed step

  • Video: a visual demonstration of how to perform the step

  • Avatar animation: motion capture of the avatar’s movements

Voice commands are available to interact with the activity, including:

  • ‘Show Image’: displays the image for the current step

  • ‘Show Video’: displays the video for the current step

  • ‘Show Avatar’: displays the avatar and animation for the current step

  • ‘Next Step’: advances to the next step

  • ‘Repeat’: repeats the current step

  • ‘Pause’: pauses the video or avatar animation

  • ‘Rewind’: moves backward in the current step

  • ‘Play’: plays or resumes the video or avatar animation

  • ‘Finish’: ends the activity and returns to the main menu

  • ‘Main Menu’: returns to the main menu

  • ‘Exit’: closes the 4DAssist app

Logging information is recorded for each step, including the date and time of the step, the time spent on the step, and whether the image, video, and avatar were viewed. This information can be displayed and exported.

 

Additional Activities will be available for purchase, with nominal prices to be determined. The 4DAssist team is actively working to define and create new Activities, and we welcome requests for specific Activities. Simply contact us with your request.