Procedure Step Recognition

Procedure Step Recognition (PSR) focuses on recognizing the correct completion and order of procedural steps. Unlike traditional action recognition, which lacks a measure of success for actions, PSR aims to provide a meaningful understanding for procedural videos: recognizing the outcome (complete? correct?) of a procedural step is often more relevant than recognizing (partial) execution of an action.

Therefore, the objective of PSR is to extract an estimate of all procedure steps correctly performed by a person up to time $t$, based on sensory inputs $X_t=(x_t, x_{t-1}, \dots, x_{t-h})$ and a descriptive set of the procedural actions to be performed $\mathcal{P}=\{a_0, a_1, \dots, a_n\}$. Here, $h$ is the observation horizon and $n+1$ the total number of actions~$a_i\in \mathcal{P}$ covered in the procedure.