Nagoya University and Fujitsu Develop World's First Technology to Detect Overtrust Situations Based on Voice Pitch and Level : Fujitsu Global

March 19, 2012
Nagoya University
Fujitsu Limited
Japan Science and Technology Agency (JST)

Nagoya University and Fujitsu Develop World's First Technology to Detect Overtrust Situations Based on Voice Pitch and Level

Analysis of phone conversations enables detection of overtrust situations such as remittance-soliciting phone phishing scams and other scams with 90% accuracy

Tokyo and Nagoya, March 19, 2012 — Nagoya University and Fujitsu today announced their successful development of the world's first technology to analyze phone conversations to automatically detect situations in which one party might "over trust" the other party. Specifically, this is the kind of situation in which an individual may have a diminished capacity to objectively evaluate an explanation being given by the other party.

By combining this technology for detecting situations of overtrust from voices over the phone with the detection of characteristic keywords, Nagoya University and Fujitsu have also developed basic technology for detecting remittance-soliciting phone phishing scams. Starting this month, a prototype of this technology deployed in mobile phones will undergo verification testing in collaboration with National Police Agency of Japan (the National Police Academy) and The Bank of Nagoya, Ltd.

Research was conducted as part of the "Modeling and Detecting Overtrust from Behavior Signals" research area led by Kazuya Takeda in the research project "Creation of Human-Harmonized Information Technology for Convivial Society" under the direction of Yoichi Tokura, Research Supervisor, in the Core Research of Evolutional Science and Technology (CREST) program of the Japan Science and Technology Agency (JST), for which a press release was issued in November 2009⁽¹⁾.

Background

There are limits to human powers of perception and judgment. When overwhelmed with information that may be distressing, some individuals, without knowing it, may have a diminished capacity to objectively evaluate information provided by another party—a situation known as "overtrust." In situations of overtrust, there is the risk of believing everything another person is saying, even in cases of remittance-soliciting phone phishing scams, for example. In order to prevent such scams, there is a need to detect such situations and provide appropriate support.

Technological Issues

To be able to detect a situation of overtrust that results with distressing information from another party, speech recognition systems have been used with technology that is able to detect when special keywords are included in a conversation. When a person is subject to psychological stress, however, the person's voice can become indistinct, making methods that rely on keyword detection alone insufficient in terms of detection accuracy. Developing a way to detect situations of overtrust with a high degree of accuracy even under these types of circumstances has, therefore, posed a challenge.

Newly Developed Technology

When a person is subject to psychological stress because the other party is conveying distressing information, situations of overtrust can occur. To address this issue, Nagoya University and Fujitsu developed the world's first technology that can infer situations of overtrust by detecting changes in voice pitch and level, as illustrated in figure 1.

Figure 1. Example of changes in voice pitch and level

Results

In a test using this technology to detect remittance-solicitation phone phishing scams, it was demonstrated that situations of overtrust could be detected with over 90% accuracy, resulting in the impending creation of basic technology for detecting situations of overtrust.

Use in Detecting Remittance-Solicitation Phone Phishing Scams

In preventing remittance-solicitation fraud, a serious social problem, effective methods are believed to entail support in detecting remittance-solicitation phone phishing scams and alerting the intended victim as well as alerting the intended victim's family or other objective third-party member, as illustrated in figure 2.

Figure 2. Detecting remittance-solicitation phone phishing scams, offering support

The detection process, as illustrated in figure 3, involves inferring situations of overtrust and then detecting certain keywords that indicate whether the conversation potentially involves an attempted fraud. These results are then integrated to detect remittance-solicitation phone scams.

Operation of this technology is checked by using a keyword list provided by the National Police Academy and recordings of actual remittance-solicitation phone scams.

1. Inferring situations of overtrust from the intended victim's voice

Nagoya University and Fujitsu created a function that, by detecting changes in voice pitch and level, as shown in figure 1, is able to infer situations of overtrust on the part of the intended victim when overwhelmed by distressing information from the suspected perpetrator.

2. Detecting special keywords from the suspected perpetrator's voice

In cases of remittance-solicitation phone phishing scams, the suspected perpetrator will often pretend to be an acquaintance of the victim or a person in a position of authority, such as a police officer or lawyer. They will then convey distressing information, such as relating to a scandal or crime supposedly committed by an acquaintance of the intended victim. Nagoya University and Fujitsu developed a function using word spotting voice recognition technology to identify when the suspected perpetrator uses special keywords from a pre-registered list, such as "indebtedness" or "compensation." The function using word spotting voice recognition technology, which ignores everything except the keywords on the pre-registered list, detects the number of times keywords relating to remittance-solicitation scams are spoken.

Figure 3. Technology for detecting remittance-solicitation phone scams

Future Plans

Using mobile phones containing this technology for detecting remittance-solicitation phone scams, Nagoya University and Fujitsu will move forward with verification testing of the technology, in collaboration with the National Police Academy and The Bank of Nagoya. The aim is to evaluate the practical utility of the technology in a variety of actual cases of remittance-solicitation fraud, identifying any potential problems, and devising solutions to overcome the problems.

---

Glossary and Notes

1 Press release:

On November 13, 2009, Fujitsu issued a press release (in Japanese only) announcing the start of research on "Modeling and Detecting Overtrust from Behavior Signals."

About Fujitsu

Fujitsu is the leading Japanese information and communication technology (ICT) company offering a full range of technology products, solutions and services. Over 170,000 Fujitsu people support customers in more than 100 countries. We use our experience and the power of ICT to shape the future of society with our customers. Fujitsu Limited (TSE:6702) reported consolidated revenues of 4.5 trillion yen (US$55 billion) for the fiscal year ended March 31, 2011. For more information, please see http://www.fujitsu.com

Press Contacts

Fujitsu Limited
Public and Investor Relations Division

Inquiries

via fujitsu.com