Chained Deep Learning Using Generalized Cross-Entropy for Multiple Annotators Classification

Supervised learning requires the accurate labeling of instances, usually provided by an expert. Crowdsourcing platforms offer a practical and cost-effective alternative for large datasets when individual annotation is impractical. In addition, these platforms gather labels from multiple labelers. Still, traditional multiple-annotator methods must account for the varying levels of expertise and the noise introduced by unreliable outputs, resulting in decreased performance. In addition, they assume a homogeneous behavior of the labelers across the input feature space, and independence constraints are imposed on outputs. We propose a Generalized Cross-Entropy-based framework using Chained Deep Learning (GCECDL) to code each annotator’s non-stationary patterns regarding the input space while preserving the inter-dependencies among experts through a chained deep learning approach. Experimental results devoted to multiple-annotator classification tasks on several well-known datasets demonstrate that our GCECDL can achieve robust predictive properties, outperforming state-of-the-art algorithms by combining the power of deep learning with a noise-robust loss function to deal with noisy labels. Moreover, network self-regularization is achieved by estimating each labeler’s reliability within the chained approach. Lastly, visual inspection and relevance analysis experiments are conducted to reveal the non-stationary coding of our method. In a nutshell, GCEDL weights reliable labelers as a function of each input sample and achieves suitable discrimination performance with preserved interpretability regarding each annotator’s trustworthiness estimation.

• Autores: Gustavo Martinez Villalobos; Gil-González, Julian; Fernandez-Gallego, Jose A.; Álvarez-Meza, Andrés Marino; Castellanos-Dominguez, Cesar German
• Tipo de recurso: Article of investigation
• Fecha de publicación: 2023
• Institución: Universidad de Ibagué
• Repositorio: Repositorio Universidad de Ibagué
• Idioma: eng
• OAI Identifier: oai:repositorio.unibague.edu.co:20.500.12313/5585
• Acceso en línea: https://hdl.handle.net/20.500.12313/5585
• Palabra clave: Entropía cruzada generalizada; Múltiples anotadores; Chained approach; Classification; Deep learning; Generalized cross-entropy; Multiple annotators
• Rights: openAccess
• License: © 2023 by the authors.