Online court hearings are becoming increasingly important. Since the Corona crisis, courts have been increasingly working on the introduction of digital court processes. The advantages: Court proceedings can be carried out more quickly and the parties involved no longer have to travel long distances. In addition to the smoother and more efficient court operations, cost and time savings speak in favor of the virtual processes.

Critics, on the other hand, point to the necessary technology that is not yet mature and to procedural questions: how can flawless video transmissions be guaranteed? How can the process with its detailed rules of conduct be transferred to the digital world?

Researchers at Fraunhofer Austria Research GmbH are addressing these questions. They are being supported, among others, by Prof. David Tait, a researcher at Western Sydney University.With the Virtual Court system, they are developing software with which the use of video technology in civil justice can be optimized and expanded. The setting in virtual courtrooms is initially intended for minor crimes such as neighborhood disputes.

While many video conferencing applications use VR glasses to give users the feeling of being in the same place, the Fraunhofer researchers are not relying on VR in this project. Rather, the image is output via the monitor. Those involved in the process communicate with the help of avatars - 3D graphic figures that represent the real person - in virtual space. Before the start of the hearing in the virtual courtroom, the participants choose the role specification that suits them, such as judge, defense attorney, prosecutor, witness or accused. A webcam records the face. The software, which is based on a 3D graphics engine, uses eye tracking to record the direction in which the user is looking. This look is translated into a head movement of the avatar - a direct eye contact between the conversation partners is simulated.

Video conferencing at eye level

No complicated technical equipment is required for the virtual court hearings with our system. Those involved in the process sit in front of the laptop with a webcam either in a public building, a police station or in their home office. That's enough to take part in the Virtual Court," says Dr.Volker Settgast, scientist in the Visual Computing division at Fraunhofer Austria in Graz. In the virtual representation of the courtroom, all users see each other and, thanks to eye tracking, recognize who is looking at them. The webcam not only captures eye movements, but also facial expressions.

"Because the webcam records mouth movements and facial expressions, conclusions can be drawn about the state of mind of the participants," says Settgast.

In classic video conferences, the video image is transmittedthis does not apply to our system through the use of avatars. Only the audio stream and the data resulting from eye tracking and facial expression recognition are transmitted. The data stream to be transferred is therefore reduced.

It is planned to adapt the virtual courtrooms to specific countries, and the avatar animation is also to be improved. For example, Settgast and his team want to integrate the participants' hands into gesture recognition.In the future, the software will also run in the browser. The first workshops and user tests of the beta version have already taken place at the Visualization Research and Teaching Laboratory at Harvard University, Department of Earth & Planetary Sciences and at the Université de Montréal's Cyberjustice Laboratory.

"Ultimately, we want to completely relocate the court setting to virtual space and implement video conferences at eye level for civil courts. Since eye movements, facial expressions and gestures are recognizable, court proceedings with a personal character can also be conducted digitally in the future," says the researcher.

This preliminary research was funded as part of a project grant from the Austrian Federal Ministry of Labor and Economics.

Article part of Fraunhofer press release in German

Photo ©Fraunhofer Austria