Touchscreen is extremely difficult to type. Since we cannot feel the keys, we rely on our sense of sight to move our fingers to the right places and check for errors, a combination of tasks that are difficult to accomplish simultaneously.

To really understand how people type on a touchscreen, researchers at Alto University and the Finnish Center for Artificial Intelligence (FCAI) have created the first artificial intelligence model that predicts how people move their eyes and fingers while typing Shakes.

The AI ​​model can simulate how a human user would type a sentence on any keyboard design. It makes errors, detects them – though not always immediately – and corrects them just as humans do. The simulation also predicts how people adapt to alternative situations, such as how their writing style changes when they start using a new auto-correction system or keyboard design.

“Previously, touchscreen typing was primarily understood from the perspective of how our fingers move. AI-based methods have helped shed new light on these movements: what we have discovered is the importance of deciding whether When and where to look. Now, we can better predict how people type on their phones or tablets, “Dr. Says Jussi Jokinan, who led the work.

The study, presented at the ACM CHI on 12 May, for example, lays the foundation for developing better and even personalized text entry solutions.

“Now that we have a realistic simulation of how humans type on a touchscreen, it should be much easier to customize the keyboard design for better typing – meaning fewer errors, faster typing, and, most importantly, less for me Disappointment, ”Jokinan explains.

In addition to predicting how a normal person would type, the model also accounts for a wide variety of users, such as those with motor impairments, and can be used to develop typing aids or interfaces taking into account these groups is. For those not facing a particular challenge, it can cut down on personal writing styles — for example, taking into account repeated mistakes in text and email — what kind of keyboard, Or auto-correction system, will serve the user best.

The novel approach is based on the group’s earlier empirical research, which provided the basis for a cognitive model of the type of man. The researchers then designed a generative model capable of typing independently. Interactive AI was worked on as part of a larger project at the Finnish Center for Artificial Intelligence.

The results are underlined by reinforcement learning, a classic machine learning method that researchers extended to simulate people. Reinforcement learning is commonly used to teach robots to solve tasks by trial and error; The team discovered a new way of using this method to generate behavior that closely matches humans – mistakes, corrections and all.

“We gave the model the same capabilities and limitations that we have as humans. When we asked to type it efficiently, it turned out how to make the best use of these capabilities. The end result is very similar to that of humans. How to type, teach models with no human data, ”says Jokinan.

Comparison of data for human typing confirmed that the model’s predictions were accurate. In the future, the team hopes to emulate slow and fast typing techniques, for example, designing learning modules useful for those who want to improve their typing.

The paper, “Touchscreen typing as optimal supervisory control,” will be presented at the ACM CHI conference on 12 May 2021.

Leave a Reply

Your email address will not be published. Required fields are marked *