Perturbation Profiles of Scanpath Classifiers: What Reading, Code-Comprehension and Video Decoders Are Sensitive To

원문 초록

When a scanpath descriptor discriminates a task, it is rarely clear what the discrimination rests on. We propose a perturbation profile for trial-level descriptors. Five transformations each destroy one stated property while preserving a listed set of invariants: fixation order, duration-position coupling, the duration distribution, orientation and handedness, and fine spatial structure. The classifier is refit, and the drop in participant-held-out AUC is the pipeline's sensitivity to that property. Because refitting lets the learner use redundant information, this describes the refitted pipeline, not what a fitted classifier used. Intervals come from a participant bootstrap that refits; in simulations at the sample sizes analysed, where the target was zero they excluded zero at a pooled rate of at most 2.5%, and they covered the target in 99.6% of 2720 cases. We profiled seven descriptors on twelve binary problems from three public datasets: reading (SB-SAT, 95 participants), program comprehension (EMIP, up to 159) and video and game viewing (GazeBase, 60). Permuting fixation order raised the AUC of a descriptor built on mean saccade amplitude on three problems (by up to 0.23), because permutation turns amplitude into a measure of spatial spread. A "comprehension" contrast that pooled two programs reached AUC 0.70 with a spread-only baseline and fell to 0.47 within one program: the profile located the confound in stimulus layout. On native versus non-native readers, recurrence was sensitive to order and to jitter at the scale of text lines but not of characters, while a two-feature duration and amplitude descriptor reached the same AUC (0.65). A graph descriptor built on the code's tokens was the strongest non-baseline descriptor on the program contrast (0.78; 0.87 with boosted trees). The interpreted results held under longer runs and three seeds; under a second learner 15 of 16 kept t