Computer games are widespread nowadays and enjoyed by people of all ages. But when it comes to kids, playing these games can be more than just fun, it is a way for them to develop important skills and build emotional intelligence. Facial expressions and sounds that kids produce during gameplay reflect their feelings, thoughts, and moods. In this paper, we propose a novel framework that integrates a fuzzy approach for the recognition of emotions through the analysis of audio and video data. Our focus lies within the specific context of computer games tailored for children, aiming to enhance their overall user experience. We use the FER dataset to detect facial emotions in video frames recorded from the screen during the game. For the audio emotion recognition of sounds a kid produces during the game, we use CREMA-D, TESS, RAVDESS, and Savee datasets. Next, a fuzzy inference system is used for the fusion of results. Besides this, our system can detect emotion stability and emotion diversity during gameplay, which, together with prevailing emotion report, can serve as valuable information for parents worrying about the effect of certain games on their kids. The proposed approach has shown promising results in the preliminary experiments we conducted, involving 3 different video games, namely fighting, racing, and logic games, and providing emotion-tracking results for kids in each game. Our study can contribute to the advancement of child-oriented game development, which is not only engaging but also accounts for children's cognitive and emotional states.
This paper presents a cross-national investigation into the soundscape perceptual attributes translation from English into European and Brazilian Portuguese. It is a study within the scope of a larger project – the Soundscape Attributes Translation Project (SATP) – that involves the translation of the same eight Perceived Affective Qualities (PAQs) into different languages through standard procedures. The structure of the paper reflects two separate moments of data collection and analysis. In the first part, the work done outlines the translation process and the resulting Portuguese-language expressions for the eight PAQs. The translation used a mixed-method approach that involved expert panels and online questionnaires. The second part of the paper outlines the impact of cultural and linguistic factors on soundscape perception and evaluation. This assessment analyzed how Portuguese speakers perceive the eight PAQs of the SATP project and compared it with the results of the English speakers. Additionally, investigations occurred on the differences between the perception of European Portuguese and Brazilian Portuguese speakers. The study found differences in how individuals from Portuguese-speaking countries describe sounds compared to the original English circumplex soundscape model. The Vibrant, Monotonous, Annoying, and Chaotic PAQs did not correlate as in the original English model, indicating the need for further investigation. However, the Pleasant and Calm PAQs had no issues in all analyses and shall remain with their initial translations. These findings have important implications for soundscape developments related to standardization tools, research methods, and test design in Portuguese-speaking contexts, as well as contributions to the growing body of knowledge on the role of culture and language in soundscape perception and evaluation.
The use of questionnaires for soundscape evaluation is a key aspect of soundscape research. Since standards and protocols mainly exist in English, using an appropriate translation and cross-cultural adaptation (CCA) methodology is necessary to maintain content equivalence between source and target language. However, many examples can be found in the literature where no appropriate methodology was applied. This study addresses the neglected aspect of the translation and CCA process in soundscape research by selecting, applying and evaluating an appropriate methodology. After a survey of the relevant literature, an approach based on a combined technique of the forward translation, synthesis, back translation, pre-test and a committee approach was selected. Additional translation guidelines drawn from the literature are suggested and implemented. For the case study of the Greek translation of ISO/TS 12913–2:2018 attributes (Method A), the steps of the methodology were applied by four independent translation groups with different compositions according to the biculturalism and bilingualism of the group members. A method for categorization of bilinguals according to the literature is proposed. In order to compare and validate the results, translated and original attributes were used in listening tests with Greek and English participants respectively, and principal component analysis (PCA) was applied. The most important findings of this research are: the results of the bicultural translation group were closer to the PCA results of the English participants for every attribute, translation of bilingual groups may not be always sufficient, translation errors may be misinterpreted for cross-cultural differences without proper application of a translation methodology and the process of back translation can be effective, especially in cases where there are not corresponding words in the target language. Finally, PCA can be used as a validation methodology for comparison of different translations.
To increase the availability and adoption of the soundscape standard, a low-cost calibration procedure for reproduction of audio stimuli over headphones was proposed as part of the global ``Soundscape Attributes Translation Project'' (SATP) for validating ISO/TS~12913-2:2018 perceived affective quality (PAQ) attribute translations. A previous preliminary study revealed significant deviations from the intended equivalent continuous A-weighted sound pressure levels ($L_{\text{A,eq}}$) using the open-circuit voltage (OCV) calibration procedure. For a more holistic human-centric perspective, the OCV method is further investigated here in terms of psychoacoustic parameters, including relevant exceedance levels to account for temporal effects on the same 27 stimuli from the SATP. Moreover, a within-subjects experiment with 36 participants was conducted to examine the effects of OCV calibration on the PAQ attributes in ISO/TS~12913-2:2018. Bland-Altman analysis of the objective indicators revealed large biases in the OCV method across all weighted sound level and loudness indicators; and roughness indicators at \SI{5}{\%} and \SI{10}{\%} exceedance levels. Significant perceptual differences due to the OCV method were observed in about \SI{20}{\%} of the stimuli, which did not correspond clearly with the biased acoustic indicators. A cautioned interpretation of the objective and perceptual differences due to small and unpaired samples nevertheless provide grounds for further investigation.
Due to the rapid advancements of sensory and computing technology, multi-modal data sources that represent the same pattern or phenomenon have attracted growing attention. As a result, finding means to explore useful information from these multi-modal data sources has quickly become a necessity. In this paper, a discriminative vectorial framework is proposed for multi-modal feature representation in knowledge discovery by employing multi-modal hashing (MH) and discriminative correlation maximization (DCM) analysis. Specifically, the proposed framework is capable of minimizing the semantic similarity among different modalities by MH and exacting intrinsic discriminative representations across multiple data sources by DCM analysis jointly, enabling a novel vectorial framework of multi-modal feature representation. Moreover, the proposed feature representation strategy is analyzed and further optimized based on canonical and non-canonical cases, respectively. Consequently, the generated feature representation leads to effective utilization of the input data sources of high quality, producing improved, sometimes quite impressive, results in various applications. The effectiveness and generality of the proposed framework are demonstrated by utilizing classical features and deep neural network (DNN) based features with applications to image and multimedia analysis and recognition tasks, including data visualization, face recognition, object recognition; cross-modal (text-image) recognition and audio emotion recognition. Experimental results show that the proposed solutions are superior to state-of-the-art statistical machine learning (SML) and DNN algorithms.
As a general means of expression, audio analysis and recognition has attracted much attentions for its wide applications in real-life world. Audio emotion recognition (AER) attempts to understand emotional states of human with the given utterance signals, and has been studied abroad for its further development on friendly human-machine interfaces. Distinguish from other existing works, the person-dependent patterns of audio emotions are conducted, and fractal dimension features are calculated for acoustic feature extraction. Furthermore, it is able to efficiently learn intrinsic characteristics of auditory emotions, while the utterance features are learned from fractal dimensions of each sub-bands. Experimental results show the proposed method is able to provide comparative performance for audio emotion recognition.