首页 文献索引 SCI期刊 AI助手
登录 注册
首页 正文

Speech communication. 2021 Oct:133:41-61. doi: 10.1016/j.specom.2021.07.010 Q32.42024

Analysis of acoustic and voice quality features for the classification of infant and mother vocalizations

婴儿与母亲发声的声学和音质特征分析分类 翻译改进

Jialu Li  1  2, Mark Hasegawa-Johnson  1  2, Nancy L McElwain  1  3

作者单位 +展开

作者单位

  • 1 Beckman Institute, University of Illinois, Urbana, IL 61801, USA.
  • 2 Department of Electrical and Computer Engineering, USA.
  • 3 Department of Human Development and Family Studies, USA.
  • DOI: 10.1016/j.specom.2021.07.010 PMID: 36062214

    摘要 翻译

    Classification of infant and parent vocalizations, particularly emotional vocalizations, is critical to understanding how infants learn to regulate emotions in social dyadic processes. This work is an experimental study of classifiers, features, and data augmentation strategies applied to the task of classifying infant and parent vocalization types. Our data were recorded both in the home and in the laboratory. Infant vocalizations were manually labeled as cry, fus (fuss), lau (laugh), bab (babble) or scr (screech), while parent (mostly mother) vocalizations were labeled as ids (infant-directed speech), ads (adult-directed speech), pla (playful), rhy (rhythmic speech or singing), lau (laugh) or whi (whisper). Linear discriminant analysis (LDA) was selected as a baseline classifier, because it gave the highest accuracy in a previously published study covering part of this corpus. LDA was compared to two neural network architectures: a two-layer fully-connected network (FCN), and a convolutional neural network with self-attention (CNSA). Baseline features extracted using the OpenSMILE toolkit were augmented by extra voice quality, phonetic, and prosodic features, each targeting perceptual features of one or more of the vocalization types. Three web data augmentation and transfer learning methods were tested: pre-training of network weights for a related task (adult emotion classification), augmentation of under-represented classes using data uniformly sampled from other corpora, and augmentation of under-represented classes using data selected by a minimum cross-corpus information difference criterion. Feature selection using Fisher scores and experiments of using weighted and unweighted samplers were also tested. Two datasets were evaluated: a benchmark dataset (CRIED) and our own corpus. In terms of unweighted-average recall of CRIED dataset, the CNSA achieved the best UAR compared with previous studies. In terms of classification accuracy, weighted F1, and macro F1 of our own dataset, the neural networks both significantly outperformed LDA; the FCN slightly (but not significantly) outperformed the CNSA. Cross-examining features selected by different feature selection algorithms permits a type of post-hoc feature analysis, in which the most important acoustic features for each binary type discrimination are listed. Examples of each vocalization type of overlapped features were selected, and their spectrograms are presented, and discussed with respect to the type-discriminative acoustic features selected by various algorithms. MFCC, log Mel Frequency Band Energy, LSP frequency, and F1 are found to be the most important spectral envelope features; F0 is found to be the most important prosodic feature.

    Keywords: Convolutional neural networks; Emotion classifier; Feature selection; Global feature; Infant vocalizations; Infant-directed speech; Self-attention.

    Keywords:acoustic features; voice quality; infant vocalizations; mother vocalizations

    Copyright © Speech communication. 中文内容为AI机器翻译,仅供参考!

    相关内容

    期刊名:Speech communication

    缩写:SPEECH COMMUN

    ISSN:0167-6393

    e-ISSN:1872-7182

    IF/分区:2.4/Q3

    文章目录 更多期刊信息

    全文链接
    引文链接
    复制
    已复制!
    推荐内容
    Analysis of acoustic and voice quality features for the classification of infant and mother vocalizations