1. Could you briefly introduce yourself (and your University/Lab)?

I am an Associate Professor at the Department of Electrical Engineering with joint appointment at the Institute of Communication Engineering and the Department of Sports Science of the National Tsing Hua University (NTHU), Taiwan. I received my B.S. degree and Ph.D. degree in Electrical Engineering under supervision of Prof. Shri Narayanan from the University of Southern California, USA in 2007 and 2012 and has served as a consultant for E.Sun Bank and Allianz Taiwan. My research interests are in speech and language, affect, behavior signals, health analytics, and multimedia. I am an IEEE senior member and an ACM and ISCA member. I am an associate editor for the IEEE Transaction on Affective Computing (2020-), the IEEE Transaction on Multimedia (2019-2020), and a TPC member for APSIPA IVM and MLDA committee.

I am the director of the BIIC lab at NTHU, Taiwan. BIIC Lab conducts advanced research at the intersection between signal processing, and machine learning with across multiple disciplinary, such as psychology, mental health, clinical applications, and so on. Our students have received multiple best student papers at conferences of INTERSPEECH and IEEE EMBC, and have won the INTERSPEECH paralinguistic challenges twice. BIIC Lab focuses on fostering future leaders in the human-centered computing community, and by working closely with relevant stakeholders, our research works contribute both to enable next-generation human-centered technology and to enhance our scientific understanding of humans.

2. What have been your most significant research contributions up to now?

My research interests are in speech and language, affect, behavior signals, health analytics, and multimedia

My research focuses on the development of multi-modal signal processing and machine learning technology processing to optimize Human Centered Intelligent Decision Analytics. Specifically, my research focuses on spoken language processing, affective multimedia, behavior computing and health analytics. The following briefly describes the three main research and innovation directions and application areas.

Multimodal Affective Computing

Globally, industry applications of emotion recognition mostly focus on a single behavior modality or a single culture (language). We have long focused on the development of multi-modal, robust, privacy sensitive machine learning and deep learning algorithm. Multi-modality (such as: voice, text, body, facial expression, physiological signal) signal analysis to optimize the accuracy of emotion recognition, robustness is achieved by transfer learning, few shot labeled data learning, handling acquisition and label variability to achieve the model robustness, and finally the proliferation of AI algorithms requires privacy sensitive representation to achieve wide adoption.

Multi-Person Conversational Analytics

Different personal traits and states revealed by modeling conversational behaviors has produced a considerable amount of research and development with applications in business decisions, human resources, and customer service. However, most of the core algorithms can only handle the context of the two-person conversation, and they lack the ability to adapt to different languages ​​and cultures. We extend these two-person conversational AI analysis from dyadic interactions to a more challenging setting of 3-6 people’s interaction situations. This kind of analysis of multi-person behavior interaction provides new application technologies for related applications such as education, gaming, product user research, organizational communication optimization, etc.

Health Analytics for Medical Applications

The application of AI technology in the medical application is a very popular topic in recent years, but most research and development focus on the technology of medical imaging for clinical diagnosis. Aiming at the development of medical-related AI, we extend to the entire patient journey application: prediction, diagnosis, prognosis, and focus on multi-modal medical data (clinical medical examination data, medical behavior, medical claim data, etc.). We also develop applications from cancer (blood cancer, lung cancer), psychiatric care (autism, ADHD), to elderly degeneration (dementia, Parkinson’s disease), multi-faceted development of algorithms with clinical value.

3. What problems in your research field deserve more attention (or what problems will you like to solve) in the next few years, and why?

Being a relatively young research domain globally with the complication and challenges in modeling human behaviors, there remains a need to continuously develop AI-enabled behavior analytics. Specifically, the challenges lie in the complexities of modeling human behaviors – from typical to distressed and disordered manifestations – computationally with AI-enabled algorithms and in the contextualization of such analytics in their relevant realm of application domains. The complexities are centered on the issue of heterogeneity of human behaviors. Sources of variability in human behaviors originate from the differences in mechanisms of information encoding (behavior or physio generation) and decoding (behavior perception). An additional layer of complexity exists because human behaviors occur largely during interactions with the environment and agents therein. This interplay, which causes a coupling effect between inter-human behaviors, is the essence of unique dynamics that has been at the core of human communications. This dependency-induced dynamic creates intricate variabilities along with variable time scales, and across interaction contexts. Lastly, much of the research effort needs to be contextualized in a meaningful and domain-aware manner. This involves translating the knowledge into a range of domains (e.g., the arts, education, and healthcare) in order to create a tangible impact.

4. What advice would you like to give to the young generation of researchers/engineers?

I would advise to be open-minded. Look around your daily life, take a critical angle in thinking about each problem and issues faced and treat those as a scientifically relevant technology development. Learn fast and try to approach relevant experts soon. My personal experience tells them there are many excellent professors that are willing to help and share their valuable knowledge. By properly integrating knowledge with techniques and by asking the right questions, your research will grow and make a substantial impact.