Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad
ilustraciones
- Autores:
-
Bermúdez García, Andrés Julián
- Tipo de recurso:
- Fecha de publicación:
- 2023
- Institución:
- Universidad Nacional de Colombia
- Repositorio:
- Universidad Nacional de Colombia
- Idioma:
- spa
- OAI Identifier:
- oai:repositorio.unal.edu.co:unal/84030
- Palabra clave:
- 510 - Matemáticas::519 - Probabilidades y matemáticas aplicadas
000 - Ciencias de la computación, información y obras generales::001 - Conocimiento
Sentimientos
Percepción de Seguridad (PoS)
Interpretabilidad Local
Interpretabilidad Categórica
Procesamiento de Lenguaje Natural (NLP)
LIME
Perception of Security (PoS)
Local and Categorical interpretability
Natural Language Processing (NPL)
LIME
- Rights
- openAccess
- License
- Atribución-NoComercial-CompartirIgual 4.0 Internacional
id |
UNACIONAL2_627b6a661d81fde7e4ad99eda13bd8aa |
---|---|
oai_identifier_str |
oai:repositorio.unal.edu.co:unal/84030 |
network_acronym_str |
UNACIONAL2 |
network_name_str |
Universidad Nacional de Colombia |
repository_id_str |
|
dc.title.spa.fl_str_mv |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
dc.title.translated.eng.fl_str_mv |
Categorical interpretability of automatic classifiers on content related to the perception of security |
dc.title.translated.por.fl_str_mv |
Interpretabilidade categórica de classificadores automáticos sobre conteúdo relacionado à percepção de segurança. |
title |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
spellingShingle |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad 510 - Matemáticas::519 - Probabilidades y matemáticas aplicadas 000 - Ciencias de la computación, información y obras generales::001 - Conocimiento Sentimientos Percepción de Seguridad (PoS) Interpretabilidad Local Interpretabilidad Categórica Procesamiento de Lenguaje Natural (NLP) LIME Perception of Security (PoS) Local and Categorical interpretability Natural Language Processing (NPL) LIME |
title_short |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
title_full |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
title_fullStr |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
title_full_unstemmed |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
title_sort |
Interpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridad |
dc.creator.fl_str_mv |
Bermúdez García, Andrés Julián |
dc.contributor.advisor.none.fl_str_mv |
Gómez Jaramillo, Francisco Albeiro |
dc.contributor.author.none.fl_str_mv |
Bermúdez García, Andrés Julián |
dc.contributor.researchgroup.spa.fl_str_mv |
Computational Modeling of Biological Systems Research Group - COMBIOS |
dc.contributor.subjectmatterexpert.none.fl_str_mv |
Chaparro , Luisa Fernanda |
dc.subject.ddc.spa.fl_str_mv |
510 - Matemáticas::519 - Probabilidades y matemáticas aplicadas 000 - Ciencias de la computación, información y obras generales::001 - Conocimiento |
topic |
510 - Matemáticas::519 - Probabilidades y matemáticas aplicadas 000 - Ciencias de la computación, información y obras generales::001 - Conocimiento Sentimientos Percepción de Seguridad (PoS) Interpretabilidad Local Interpretabilidad Categórica Procesamiento de Lenguaje Natural (NLP) LIME Perception of Security (PoS) Local and Categorical interpretability Natural Language Processing (NPL) LIME |
dc.subject.lemb.none.fl_str_mv |
Sentimientos |
dc.subject.proposal.spa.fl_str_mv |
Percepción de Seguridad (PoS) Interpretabilidad Local Interpretabilidad Categórica Procesamiento de Lenguaje Natural (NLP) LIME |
dc.subject.proposal.eng.fl_str_mv |
Perception of Security (PoS) Local and Categorical interpretability Natural Language Processing (NPL) LIME |
description |
ilustraciones |
publishDate |
2023 |
dc.date.accessioned.none.fl_str_mv |
2023-06-20T16:36:28Z |
dc.date.available.none.fl_str_mv |
2023-06-20T16:36:28Z |
dc.date.issued.none.fl_str_mv |
2023-06-06 |
dc.type.spa.fl_str_mv |
Trabajo de grado - Maestría |
dc.type.driver.spa.fl_str_mv |
info:eu-repo/semantics/masterThesis |
dc.type.version.spa.fl_str_mv |
info:eu-repo/semantics/acceptedVersion |
dc.type.content.spa.fl_str_mv |
Text |
dc.type.redcol.spa.fl_str_mv |
http://purl.org/redcol/resource_type/TM |
status_str |
acceptedVersion |
dc.identifier.uri.none.fl_str_mv |
https://repositorio.unal.edu.co/handle/unal/84030 |
dc.identifier.instname.spa.fl_str_mv |
Universidad Nacional de Colombia |
dc.identifier.reponame.spa.fl_str_mv |
Repositorio Institucional Universidad Nacional de Colombia |
dc.identifier.repourl.spa.fl_str_mv |
https://repositorio.unal.edu.co/ |
url |
https://repositorio.unal.edu.co/handle/unal/84030 https://repositorio.unal.edu.co/ |
identifier_str_mv |
Universidad Nacional de Colombia Repositorio Institucional Universidad Nacional de Colombia |
dc.language.iso.spa.fl_str_mv |
spa |
language |
spa |
dc.relation.references.spa.fl_str_mv |
Zschech, P., Weinzierl, S., Hambauer, N., Zilker, S., and Kraus, M. (2022). Gam (e) changer or not? an evaluation o interpretable machine learning models based on additive model constraints. arXiv preprint arXiv:2204.09123. Zhu, Q. (2020). On the performance of matthews correlation coefficient (mcc) for imbalanced dataset. Pattern Recognition Letters, 136:71–80. Zhang, Y., Chen, M., and Liu, L. (2015). A review on text mining. In 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS), pages 681–685. IEEE. Yadav, S. and Sarkar, M. (2018). Enhancing sentiment analysis using domain-specific lexicon: A case study on gst. In 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 1109–1114. IEEE. Yadav, R. and Sheoran, S. K. (2018). Crime prediction using auto regression techniques for time series data. In 2018 3rd International Conference and Workshops on Recent Advances and Innovations in Engineering (ICRAIE), pages 1–5. IEEE. Wang, Z. J., Kale, A., Nori, H., Stella, P., Nunnally, M., Chau, D. H., Vorvoreanu, M., Vaughan, J. W., and Caruana, R. (2021). Gam changer: Editing generalized additive models with interactive visualization. arXiv preprint arXiv:2112.03245. Wang, X., Gerber, M. S., and Brown, D. E. (2012). Automatic crime prediction using events extracted from twitter posts. In International conference on social computing, behavioral-cultural modeling, and prediction, pages 231–238. Springer. Villegas, A.J.R., Pabón, J.S.M., Rubio, M.D., Quintero, S., Vargas, J.G., and García, H. (2022). Spatio-temporal sparsity in homicide prediction models. IEEE Access, 10:14359–14367. Victorino, J., Rudas, J., Reyes, A. M., Pulido, C., Chaparro, L. F., Narváez, L. A., Martinez, D., and Gómez, F. (2020). Spatial-temporal patterns of aggressive behaviors: A case study Bogotá, Colombia. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 685–691. IEEE. Soumya George, K. and Joseph, S. (2014). Text classification by augmenting bag of words (bow) representation with co-occurrence feature. IOSR Journal of Computer Engineering, 16(1):34–38. Schultz-Jones, B. (2009). Examining information behavior through social networks: An interdisciplinary review. Journal of Documentation. Sánchez, M. (2008). La percepción de seguridad y la realidad social. Cuadernos de seguridad, 219. Rutjens, B. T. and Brandt, M. J. (2019). Belief systems and the perception of reality. Routledge London, UK. Rundmo, T. r. and Moen, B. r.-E. (2006). Risk perception and demand for risk mitigation in transport: A comparison of lay people, politicians and experts. Journal of Risk research, 9(6):623–640. Ribeiro, M. T., Singh, S., and Guestrin, C. (2016b). “why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135– 1144. Ribeiro, M. T., Singh, S., and Guestrin, C. (2016a). ”why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135–1144. Reyes, A. M., Rudas, J., Pulido, C., Victorino, J., Martínez, D., Narváez, L. Á., and Gómez, F. (2020). Characterization of temporal patterns in the occurrence of aggressive behaviors in Bogotá (Colombia). In 2020 7th International conference on behavioural and social computing (BESC), pages 1–4. IEEE. Refaeilzadeh, P., Tang, L., and Liu, H. (2009). Cross-validation. Encyclopedia of database systems, 5:532–538. Qi, P., Zhang, Y., Zhang, Y., Bolton, J., and Manning, C. D. (2020). Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Pulido, C., Prieto, J., and Gómez, F. (2019). How the social interactions in communities affect the fear of crime. Systems Research and Behavioral Science, 36(6):789–798. Pulido, C., Chaparro, L. F., Rudas, J., Reyes, A. M., Victorino, J., Narváez, L. Á., Martínez, D., and Gómez, F. (2021). Data filtering and classification for the identification of texts related to security in Bogotá Colombia. 7th International Conference on Computational Social Science IC2S2 2021. Prieto Curiel, R., Cresci, S., Muntean, C. I., and Bishop, S. R. (2020). Crime and its fear in social media. Palgrave Communications, 6(1):1–12. Prieto Curiel, R. and Bishop, S. R. (2016). A metric of the difference between perception of security and victimisation rates. Crime Science, 5(1):1–15. Prathap, B. R. and Ramesha, K. (2018). Twitter sentiment for analysing different types of crimes. In 2018 International Conference on Communication, Computing and Internet of Things (IC3IoT), pages 483–488. IEEE. Porter, M. F. (2001). Snowball: A language for stemming algorithms. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830. Pabón, J. S. M., Rubio, M. D., Castaño, Y., Riascos, A. J., and Díaz, P. R. (2020). A manifold learning data enrichment methodology for homicide prediction. In 2020 7th International Conference on Behavioural and Social Computing (BESC), pages 1–4. IEEE. Ordoñez-Eraso, H.-A., Pardo-Calvache, C.-J., and Cobos-Lozada, C.-A. (2020). Detection of homicide trends in Colombia using machine learning. Learning, 29(54):e11740. Nocedal, J. and Wright, S. J. (2006). Numerical optimization 2nd edition. Naser, M. and Alavi, A. (2020). Insights into performance fitness and error metrics for machine learning. arXiv preprint arXiv:2006.00887. Molnar, C. (2022). Interpretable Machine Learning. 2 edition. Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38. Messalas, A., Kanellopoulos, Y., and Makris, C. (2019). Modelagnostic interpretability with shapley values. In 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), pages 1–7. IEEE. Matthews, B. W. (1975). Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442–451. Malleson, N. and Andresen, M. A. (2015). The impact of using social media data in crime rate calculations: shifting hot spots and changing spatial patterns. Cartography and Geographic Information Science, 42(2):112–121. Madasamy, K. and Ramaswami, M. (2017). Data imbalance and classifiers: Impact and solutions from a big data perspective. International Journal of Computational Intelligence Research, 13(9):2267–2281. Luque, C., Luna, J. M., Luque, M., and Ventura, S. (2019). An advanced review on text mining in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(3):e1302. Le, Q. and Mikolov, T. (2014). Distributed representations of sentences and documents. In International conference on machine learning, pages 1188–1196. PMLR. Latané, B. (1981). The psychology of social impact. American psychologist, 36(4). Ladani, D. J. and Desai, N. P. (2020). Stopword identification and removal techniques on tc and ir applications: A survey. In 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), pages 466– 472. IEEE. Kounadi, O., Lampoltshammer, T. J., Groff, E., Sitko, I., and Leitner, M. (2015). Exploring twitter to analyze the public’s reaction patterns to recently reported homicides in london. PloS one, 10(3):e0121848. Kleck, G. and Barnes, J. C. (2014). Do more police lead to more crime deterrence? Crime & Delinquency, 60(5):716–738. Kim, B., Khanna, R., and Koyejo, O. O. (2016). Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29. Ketkar, N. (2017). Stochastic gradient descent. In Deep learning with Python, pages 113–132. Springer. Kaur, J. and Buttar, P. K. (2018). A systematic review on stopword removal algorithms. International Journal on Future Revolution in Computer Science & Communication Engineering, 4(4):207–210. Jurman, G., Riccadonna, S., and Furlanello, C. (2012). A comparison of MCC and CEN error measures in multi-class prediction. Java, A., Song, X., Finin, T., and Tseng, B. (2007). Why we twitter: An analysis of a microblogging community. In International Workshop on Social Network Mining and Analysis, pages 118–138. Springer. Hummelsheim, D., Hirtenlehner, H., Jackson, J., and Oberwittler, D. (2011). Social insecurities and fear of crime: A cross-national study on the impact of welfare state policies on crime-related anxieties. European sociological review, 27(3):327–345. Hooker, G. (2007). Generalized functional anova diagnostics for highdimensional functions of dependent variables. Journal of Computational and Graphical Statistics, 16(3):709–732. Hooker, G. (2004). Discovering additive structure in black box functions. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 575–580. Hollis, M. E., Downey, S., Del Carmen, A., and Dobbs, R. R. (2017). The relationship between media portrayals and crime: perceptions of fear of crime among citizens. Crime prevention and community safety, 19(1):46–60. Hirschberg, J. and Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245):261–266. Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756. Ghosh, S., Roy, S., and Bandyopadhyay, S. K. (2012). A tutorial review on text mining algorithms. International Journal of Advanced Research in Computer and Communication Engineering, 1(4):7. Gerber, M. S. (2014). Predicting crime using twitter and kernel density estimation. Decision Support Systems, 61:115–125. Gaisbauer, F., Pournaki, A., Banisch, S., and Olbrich, E. (2021). Ideological differences in engagement in public debate on twitter. Plos one, 16(3):e0249241. Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. The Annals of statistics, 32(2):407–499. Drakulich, K. M. (2015). Social capital, information, and perceived safety from crime: The differential effects of reassuring social connections and vicarious victimization. Social Science Quarterly, 96(1):176–190. Doshi-Velez, F. and Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608. Da Silva, N. F., Hruschka, E. R., and Hruschka Jr, E. R. (2014). Tweet sentiment analysis with classifier ensembles. Decision support systems, 66:170–179. Cvetojevic, S. and Hochmair, H. H. (2018). Analyzing the spread of tweets in response to paris attacks. Computers, Environment and Urban Systems, 71:14–26. Curiel, R. and Bishop, S. (2017). Modelling the fear of crime. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Science, 473:20170156. Cruz, F. L., Troyano, J. A., Pontes, B., and Ortega, F. J. (2014). Mlsenticon: Un lexicón multilingüe de polaridades semánticas a nivel de lemas. Procesamiento del Lenguaje Natural, 53:113–120. Christopher, D. M., Prabhakar, R., and Hinrich, S. (2008). Introduction to information retrieval. Chowdhury, G. G. and Chowdhury, S. (2003). Introduction to digital libraries. Facet publishing. Chowdhary, K. (2020). Natural language processing. Fundamentals of artificial intelligence, pages 603–649. Chicco, D. and Jurman, G. (2020). The advantages of the matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC genomics, 21(1):1–13. Chen, X., Cho, Y., and Jang, S. Y. (2015). Crime prediction using twitter sentiment and weather. In 2015 systems and information engineering design symposium, pages 63–68. IEEE. Chaparro, L. F., Pulido, C., Rudas, J., Victorino, J., Reyes, A. M., Estrada, C., Narvaez, L. A., and Gómez, F. (2021b). Quantifying perception of security through social media and its relationship with crime. IEEE Access, 9:139201–139213. Chaparro, L. F., Pulido, C., Rudas, J., Reyes, A. M., Victorino, J., Narv´aez, L., Martinez, D., and Gómez, F. (2021a). Interpretability of the perception of security based on tweets content. In 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pages 1–6. IEEE. Chaparro, L. F., Pulido, C., Rudas, J., Reyes, A., Victorino, J., Narváez, L. a., Gómez, F., and Martinez, D. (2020). Sentiment analysis of social network content to characterize the perception of security. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 685– 691. IEEE. Carvalho, D. V., Pereira, E. M., and Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832. Cambria, E., Xing, F., Thelwall, M., andWelsch, R. (2022). Sentiment analysis as a multidisciplinary research area. IEEE Transactions on Artificial Intelligence, 3(2):1–3. Camargo, J. E., Torres, C. A., Martínez, O. H., and Gómez, F. A. (2016). A big data analytics system to analyze citizens’ perception of security. In 2016 IEEE International Smart Cities Conference (ISC2), pages 1–5. IEEE. Cámara de Comercio de Bogotá . (2022). Encuesta de percepción y victimización de Bogotá-2021. Brown, M. E., Dustman, P. A., and Barthelemy, J. J. (2021). Twitter impact on a community trauma: An examination of who, what, and why it radiated. Journal of community psychology, 49(3):838–853. Brooker, R. G. and Schaefer, T. (2015). Methods of measuring public opinion. Public opinion in the 21st century. https://www. uky.edu/AS/PoliSci/Peffley/pdf/473Measuring% 20Public% 20Opinion. pdf. Bottou, L. (2010). Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010, pages 177–186. Springer. Bokinsky, H., McKenzie, A., Bayoumi, A., McCaslin, R., Patterson, A., Matthews, M., Schmidley, J., and Eisner, L. (2013). Application of natural language processing techniques to marine v-22 maintenance data for populating a cbm-oriented database. In AHS Airworthiness, CBM, and HUMS Specialists’ Meeting, Huntsville, AL. Bishop, C. M. and Nasrabadi, N. M. (2006). Pattern recognition and machine learning, volume 4. Springer. Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pages 69–72. Bhagvat, S. (2011). Clustering of twitter technology tweets and the impact of stopwords on clusters. Bendler, J., Brandt, T., Wagner, S., and Neumann, D. (2014). Investigating crime-to-twitter relationships in urban environments-facilitating a virtual neighborhood watch. Association for Information Systems (AIS) eLibrary. Barreras, F., Diaz, C., Riascos, A., and Ribero, M. (2016). Comparison of different crime prediction models in bogot´a. 2016. Balakrishnan, V. and Lloyd-Yemoh, E. (2014). Stemming and lemmatization: A comparison of retrieval performances. Ayodele, T. O. (2010). Types of machine learning algorithms in new advances in machine learning. Croatia, Rijeka. Anguiano-Hernández, E. (2009). Naive bayes multinomial para clasificación de texto usando un esquema de pesado por clases. Agirre, E., Alegria, I., Arregi, X., Artola, X., de Ilarraza, A. D., Maritxalar, M., Sarasola, K., and Urkia, M. (1992). Xuxen: A spelling checker/corrector for basque based on two-level morphology. In Third Conference on Applied Natural Language Processing, pages 119–125. |
dc.rights.coar.fl_str_mv |
http://purl.org/coar/access_right/c_abf2 |
dc.rights.license.spa.fl_str_mv |
Atribución-NoComercial-CompartirIgual 4.0 Internacional |
dc.rights.uri.spa.fl_str_mv |
http://creativecommons.org/licenses/by-nc-sa/4.0/ |
dc.rights.accessrights.spa.fl_str_mv |
info:eu-repo/semantics/openAccess |
rights_invalid_str_mv |
Atribución-NoComercial-CompartirIgual 4.0 Internacional http://creativecommons.org/licenses/by-nc-sa/4.0/ http://purl.org/coar/access_right/c_abf2 |
eu_rights_str_mv |
openAccess |
dc.format.extent.spa.fl_str_mv |
xiv, 70 páginas |
dc.format.mimetype.spa.fl_str_mv |
application/pdf |
dc.publisher.spa.fl_str_mv |
Universidad Nacional de Colombia |
dc.publisher.program.spa.fl_str_mv |
Bogotá - Ciencias - Maestría en Ciencias - Matemática Aplicada |
dc.publisher.faculty.spa.fl_str_mv |
Facultad de Ciencias |
dc.publisher.place.spa.fl_str_mv |
Bogotá,Colombia |
dc.publisher.branch.spa.fl_str_mv |
Universidad Nacional de Colombia - Sede Bogotá |
institution |
Universidad Nacional de Colombia |
bitstream.url.fl_str_mv |
https://repositorio.unal.edu.co/bitstream/unal/84030/1/license.txt https://repositorio.unal.edu.co/bitstream/unal/84030/3/1010217881.2023.pdf https://repositorio.unal.edu.co/bitstream/unal/84030/4/1010217881.2023.pdf.jpg |
bitstream.checksum.fl_str_mv |
eb34b1cf90b7e1103fc9dfd26be24b4a d4b8ef110bd4c89dd8b6359708bc8bae 712860d7950ae70a941f5348eba6c63a |
bitstream.checksumAlgorithm.fl_str_mv |
MD5 MD5 MD5 |
repository.name.fl_str_mv |
Repositorio Institucional Universidad Nacional de Colombia |
repository.mail.fl_str_mv |
repositorio_nal@unal.edu.co |
_version_ |
1814089768777023488 |
spelling |
Atribución-NoComercial-CompartirIgual 4.0 Internacionalhttp://creativecommons.org/licenses/by-nc-sa/4.0/info:eu-repo/semantics/openAccesshttp://purl.org/coar/access_right/c_abf2Gómez Jaramillo, Francisco Albeiro415aa92d5615e8a2fa29cfa0a28ec210Bermúdez García, Andrés Julián33ccc8775f8162142559ba12f0e07f64Computational Modeling of Biological Systems Research Group - COMBIOSChaparro , Luisa Fernanda2023-06-20T16:36:28Z2023-06-20T16:36:28Z2023-06-06https://repositorio.unal.edu.co/handle/unal/84030Universidad Nacional de ColombiaRepositorio Institucional Universidad Nacional de Colombiahttps://repositorio.unal.edu.co/ilustracionesLa percepción de la seguridad está relacionada con los sentimientos de los ciudadanos ante el riesgo asociado a los sucesos de seguridad y la magnitud de sus consecuencias. Debido a esta naturaleza subjetiva, es un tema complejo de cuantificar. Por ello, las redes sociales surgieron como una alternativa para cuantificar estas opiniones. Recientemente, se han utilizado métodos de aprendizaje automático supervisado multiclase para cuantificar distintos niveles de percepción de la seguridad. Sin embargo, estos métodos carecen de interpretabilidad sobre por qué un grupo de tweets clasifica en el mismo nivel de percepción de seguridad. En este trabajo, se propone una estrategia novedosa de interpretabilidad categórica y selección agnóstica al modelo para un grupo de predicciones relacionadas con el mismo nivel de percepción de la seguridad. Los resultados sugieren que el modelo propuesto presenta altos niveles de interpretabilidad para las diferentes categorías de percepción de seguridad. Adicionalmente, las métricas de interpretabilidad introducidas mejoran el proceso de selección de los modelos. (Texto tomado de la fuente)The perception of security relates to citizens’ feelings in the face of risk associated with security events and the magnitude of its consequences. Because of this subjective nature, it is a complex subject to quantify. Therefore, social networks emerged as an alternative to quantifying these opinions. Recently, multiclass supervised machine learning methods quantified different levels of security perception. However, these methods lack interpretability about why a group of tweets classifies in the same level of perception of security. This work proposes a novel strategy of categorical interpretability and model-agnostic selection for a group of predictions related to the same level of perception of security. The results suggest that the proposed model presents high levels of interpretability for the different PoS categories. Additionally, the introduced interpretability metrics improve the model selection process.MaestríaMagíster en Ciencias - Matemática AplicadaInterpretabilidad en aprendizaje automático.xiv, 70 páginasapplication/pdfspaUniversidad Nacional de ColombiaBogotá - Ciencias - Maestría en Ciencias - Matemática AplicadaFacultad de CienciasBogotá,ColombiaUniversidad Nacional de Colombia - Sede Bogotá510 - Matemáticas::519 - Probabilidades y matemáticas aplicadas000 - Ciencias de la computación, información y obras generales::001 - ConocimientoSentimientosPercepción de Seguridad (PoS)Interpretabilidad LocalInterpretabilidad CategóricaProcesamiento de Lenguaje Natural (NLP)LIMEPerception of Security (PoS)Local and Categorical interpretabilityNatural Language Processing (NPL)LIMEInterpretabilidad categórica de clasificadores automáticos sobre contenido relacionado a la percepción de la seguridadCategorical interpretability of automatic classifiers on content related to the perception of securityInterpretabilidade categórica de classificadores automáticos sobre conteúdo relacionado à percepção de segurança.Trabajo de grado - Maestríainfo:eu-repo/semantics/masterThesisinfo:eu-repo/semantics/acceptedVersionTexthttp://purl.org/redcol/resource_type/TMZschech, P., Weinzierl, S., Hambauer, N., Zilker, S., and Kraus, M. (2022). Gam (e) changer or not? an evaluation o interpretable machine learning models based on additive model constraints. arXiv preprint arXiv:2204.09123.Zhu, Q. (2020). On the performance of matthews correlation coefficient (mcc) for imbalanced dataset. Pattern Recognition Letters, 136:71–80.Zhang, Y., Chen, M., and Liu, L. (2015). A review on text mining. In 2015 6th IEEE International Conference on Software Engineering and Service Science (ICSESS), pages 681–685. IEEE.Yadav, S. and Sarkar, M. (2018). Enhancing sentiment analysis using domain-specific lexicon: A case study on gst. In 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pages 1109–1114. IEEE.Yadav, R. and Sheoran, S. K. (2018). Crime prediction using auto regression techniques for time series data. In 2018 3rd International Conference and Workshops on Recent Advances and Innovations in Engineering (ICRAIE), pages 1–5. IEEE.Wang, Z. J., Kale, A., Nori, H., Stella, P., Nunnally, M., Chau, D. H., Vorvoreanu, M., Vaughan, J. W., and Caruana, R. (2021). Gam changer: Editing generalized additive models with interactive visualization. arXiv preprint arXiv:2112.03245.Wang, X., Gerber, M. S., and Brown, D. E. (2012). Automatic crime prediction using events extracted from twitter posts. In International conference on social computing, behavioral-cultural modeling, and prediction, pages 231–238. Springer.Villegas, A.J.R., Pabón, J.S.M., Rubio, M.D., Quintero, S., Vargas, J.G., and García, H. (2022). Spatio-temporal sparsity in homicide prediction models. IEEE Access, 10:14359–14367.Victorino, J., Rudas, J., Reyes, A. M., Pulido, C., Chaparro, L. F., Narváez, L. A., Martinez, D., and Gómez, F. (2020). Spatial-temporal patterns of aggressive behaviors: A case study Bogotá, Colombia. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 685–691. IEEE.Soumya George, K. and Joseph, S. (2014). Text classification by augmenting bag of words (bow) representation with co-occurrence feature. IOSR Journal of Computer Engineering, 16(1):34–38.Schultz-Jones, B. (2009). Examining information behavior through social networks: An interdisciplinary review. Journal of Documentation.Sánchez, M. (2008). La percepción de seguridad y la realidad social. Cuadernos de seguridad, 219.Rutjens, B. T. and Brandt, M. J. (2019). Belief systems and the perception of reality. Routledge London, UK.Rundmo, T. r. and Moen, B. r.-E. (2006). Risk perception and demand for risk mitigation in transport: A comparison of lay people, politicians and experts. Journal of Risk research, 9(6):623–640.Ribeiro, M. T., Singh, S., and Guestrin, C. (2016b). “why should i trust you?” explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135– 1144.Ribeiro, M. T., Singh, S., and Guestrin, C. (2016a). ”why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA, August 13-17, 2016, pages 1135–1144.Reyes, A. M., Rudas, J., Pulido, C., Victorino, J., Martínez, D., Narváez, L. Á., and Gómez, F. (2020). Characterization of temporal patterns in the occurrence of aggressive behaviors in Bogotá (Colombia). In 2020 7th International conference on behavioural and social computing (BESC), pages 1–4. IEEE.Refaeilzadeh, P., Tang, L., and Liu, H. (2009). Cross-validation. Encyclopedia of database systems, 5:532–538.Qi, P., Zhang, Y., Zhang, Y., Bolton, J., and Manning, C. D. (2020). Stanza: A Python natural language processing toolkit for many human languages. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations.Pulido, C., Prieto, J., and Gómez, F. (2019). How the social interactions in communities affect the fear of crime. Systems Research and Behavioral Science, 36(6):789–798.Pulido, C., Chaparro, L. F., Rudas, J., Reyes, A. M., Victorino, J., Narváez, L. Á., Martínez, D., and Gómez, F. (2021). Data filtering and classification for the identification of texts related to security in Bogotá Colombia. 7th International Conference on Computational Social Science IC2S2 2021.Prieto Curiel, R., Cresci, S., Muntean, C. I., and Bishop, S. R. (2020). Crime and its fear in social media. Palgrave Communications, 6(1):1–12.Prieto Curiel, R. and Bishop, S. R. (2016). A metric of the difference between perception of security and victimisation rates. Crime Science, 5(1):1–15.Prathap, B. R. and Ramesha, K. (2018). Twitter sentiment for analysing different types of crimes. In 2018 International Conference on Communication, Computing and Internet of Things (IC3IoT), pages 483–488. IEEE.Porter, M. F. (2001). Snowball: A language for stemming algorithms.Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830.Pabón, J. S. M., Rubio, M. D., Castaño, Y., Riascos, A. J., and Díaz, P. R. (2020). A manifold learning data enrichment methodology for homicide prediction. In 2020 7th International Conference on Behavioural and Social Computing (BESC), pages 1–4. IEEE.Ordoñez-Eraso, H.-A., Pardo-Calvache, C.-J., and Cobos-Lozada, C.-A. (2020). Detection of homicide trends in Colombia using machine learning. Learning, 29(54):e11740.Nocedal, J. and Wright, S. J. (2006). Numerical optimization 2nd edition.Naser, M. and Alavi, A. (2020). Insights into performance fitness and error metrics for machine learning. arXiv preprint arXiv:2006.00887.Molnar, C. (2022). Interpretable Machine Learning. 2 edition.Explanation in artificial intelligence: Insights from the social sciences. Artificial intelligence, 267:1–38.Messalas, A., Kanellopoulos, Y., and Makris, C. (2019). Modelagnostic interpretability with shapley values. In 2019 10th International Conference on Information, Intelligence, Systems and Applications (IISA), pages 1–7. IEEE.Matthews, B. W. (1975). Comparison of the predicted and observed secondary structure of t4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Structure, 405(2):442–451.Malleson, N. and Andresen, M. A. (2015). The impact of using social media data in crime rate calculations: shifting hot spots and changing spatial patterns. Cartography and Geographic Information Science, 42(2):112–121.Madasamy, K. and Ramaswami, M. (2017). Data imbalance and classifiers: Impact and solutions from a big data perspective. International Journal of Computational Intelligence Research, 13(9):2267–2281.Luque, C., Luna, J. M., Luque, M., and Ventura, S. (2019). An advanced review on text mining in medicine. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 9(3):e1302.Le, Q. and Mikolov, T. (2014). Distributed representations of sentences and documents. In International conference on machine learning, pages 1188–1196. PMLR.Latané, B. (1981). The psychology of social impact. American psychologist, 36(4).Ladani, D. J. and Desai, N. P. (2020). Stopword identification and removal techniques on tc and ir applications: A survey. In 2020 6th International Conference on Advanced Computing and Communication Systems (ICACCS), pages 466– 472. IEEE.Kounadi, O., Lampoltshammer, T. J., Groff, E., Sitko, I., and Leitner, M. (2015). Exploring twitter to analyze the public’s reaction patterns to recently reported homicides in london. PloS one, 10(3):e0121848.Kleck, G. and Barnes, J. C. (2014). Do more police lead to more crime deterrence? Crime & Delinquency, 60(5):716–738.Kim, B., Khanna, R., and Koyejo, O. O. (2016). Examples are not enough, learn to criticize! criticism for interpretability. Advances in neural information processing systems, 29.Ketkar, N. (2017). Stochastic gradient descent. In Deep learning with Python, pages 113–132. Springer.Kaur, J. and Buttar, P. K. (2018). A systematic review on stopword removal algorithms. International Journal on Future Revolution in Computer Science & Communication Engineering, 4(4):207–210.Jurman, G., Riccadonna, S., and Furlanello, C. (2012). A comparison of MCC and CEN error measures in multi-class prediction.Java, A., Song, X., Finin, T., and Tseng, B. (2007). Why we twitter: An analysis of a microblogging community. In International Workshop on Social Network Mining and Analysis, pages 118–138. Springer.Hummelsheim, D., Hirtenlehner, H., Jackson, J., and Oberwittler, D. (2011). Social insecurities and fear of crime: A cross-national study on the impact of welfare state policies on crime-related anxieties. European sociological review, 27(3):327–345.Hooker, G. (2007). Generalized functional anova diagnostics for highdimensional functions of dependent variables. Journal of Computational and Graphical Statistics, 16(3):709–732.Hooker, G. (2004). Discovering additive structure in black box functions. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 575–580.Hollis, M. E., Downey, S., Del Carmen, A., and Dobbs, R. R. (2017). The relationship between media portrayals and crime: perceptions of fear of crime among citizens. Crime prevention and community safety, 19(1):46–60.Hirschberg, J. and Manning, C. D. (2015). Advances in natural language processing. Science, 349(6245):261–266.Grandini, M., Bagli, E., and Visani, G. (2020). Metrics for multi-class classification: an overview. arXiv preprint arXiv:2008.05756.Ghosh, S., Roy, S., and Bandyopadhyay, S. K. (2012). A tutorial review on text mining algorithms. International Journal of Advanced Research in Computer and Communication Engineering, 1(4):7.Gerber, M. S. (2014). Predicting crime using twitter and kernel density estimation. Decision Support Systems, 61:115–125.Gaisbauer, F., Pournaki, A., Banisch, S., and Olbrich, E. (2021). Ideological differences in engagement in public debate on twitter. Plos one, 16(3):e0249241.Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. (2004). Least angle regression. The Annals of statistics, 32(2):407–499.Drakulich, K. M. (2015). Social capital, information, and perceived safety from crime: The differential effects of reassuring social connections and vicarious victimization. Social Science Quarterly, 96(1):176–190.Doshi-Velez, F. and Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608.Da Silva, N. F., Hruschka, E. R., and Hruschka Jr, E. R. (2014). Tweet sentiment analysis with classifier ensembles. Decision support systems, 66:170–179.Cvetojevic, S. and Hochmair, H. H. (2018). Analyzing the spread of tweets in response to paris attacks. Computers, Environment and Urban Systems, 71:14–26.Curiel, R. and Bishop, S. (2017). Modelling the fear of crime. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Science, 473:20170156.Cruz, F. L., Troyano, J. A., Pontes, B., and Ortega, F. J. (2014). Mlsenticon: Un lexicón multilingüe de polaridades semánticas a nivel de lemas. Procesamiento del Lenguaje Natural, 53:113–120.Christopher, D. M., Prabhakar, R., and Hinrich, S. (2008). Introduction to information retrieval.Chowdhury, G. G. and Chowdhury, S. (2003). Introduction to digital libraries. Facet publishing.Chowdhary, K. (2020). Natural language processing. Fundamentals of artificial intelligence, pages 603–649.Chicco, D. and Jurman, G. (2020). The advantages of the matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC genomics, 21(1):1–13.Chen, X., Cho, Y., and Jang, S. Y. (2015). Crime prediction using twitter sentiment and weather. In 2015 systems and information engineering design symposium, pages 63–68. IEEE.Chaparro, L. F., Pulido, C., Rudas, J., Victorino, J., Reyes, A. M., Estrada, C., Narvaez, L. A., and Gómez, F. (2021b). Quantifying perception of security through social media and its relationship with crime. IEEE Access, 9:139201–139213.Chaparro, L. F., Pulido, C., Rudas, J., Reyes, A. M., Victorino, J., Narv´aez, L., Martinez, D., and Gómez, F. (2021a). Interpretability of the perception of security based on tweets content. In 2021 International Conference on Applied Artificial Intelligence (ICAPAI), pages 1–6. IEEE.Chaparro, L. F., Pulido, C., Rudas, J., Reyes, A., Victorino, J., Narváez, L. a., Gómez, F., and Martinez, D. (2020). Sentiment analysis of social network content to characterize the perception of security. In 2020 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), pages 685– 691. IEEE.Carvalho, D. V., Pereira, E. M., and Cardoso, J. S. (2019). Machine learning interpretability: A survey on methods and metrics. Electronics, 8(8):832.Cambria, E., Xing, F., Thelwall, M., andWelsch, R. (2022). Sentiment analysis as a multidisciplinary research area. IEEE Transactions on Artificial Intelligence, 3(2):1–3.Camargo, J. E., Torres, C. A., Martínez, O. H., and Gómez, F. A. (2016). A big data analytics system to analyze citizens’ perception of security. In 2016 IEEE International Smart Cities Conference (ISC2), pages 1–5. IEEE.Cámara de Comercio de Bogotá . (2022). Encuesta de percepción y victimización de Bogotá-2021.Brown, M. E., Dustman, P. A., and Barthelemy, J. J. (2021). Twitter impact on a community trauma: An examination of who, what, and why it radiated. Journal of community psychology, 49(3):838–853.Brooker, R. G. and Schaefer, T. (2015). Methods of measuring public opinion. Public opinion in the 21st century. https://www. uky.edu/AS/PoliSci/Peffley/pdf/473Measuring% 20Public% 20Opinion. pdf.Bottou, L. (2010). Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT’2010, pages 177–186. Springer.Bokinsky, H., McKenzie, A., Bayoumi, A., McCaslin, R., Patterson, A., Matthews, M., Schmidley, J., and Eisner, L. (2013). Application of natural language processing techniques to marine v-22 maintenance data for populating a cbm-oriented database. In AHS Airworthiness, CBM, and HUMS Specialists’ Meeting, Huntsville, AL.Bishop, C. M. and Nasrabadi, N. M. (2006). Pattern recognition and machine learning, volume 4. Springer.Nltk: the natural language toolkit. In Proceedings of the COLING/ACL 2006 Interactive Presentation Sessions, pages 69–72.Bhagvat, S. (2011). Clustering of twitter technology tweets and the impact of stopwords on clusters.Bendler, J., Brandt, T., Wagner, S., and Neumann, D. (2014). Investigating crime-to-twitter relationships in urban environments-facilitating a virtual neighborhood watch. Association for Information Systems (AIS) eLibrary.Barreras, F., Diaz, C., Riascos, A., and Ribero, M. (2016). Comparison of different crime prediction models in bogot´a. 2016.Balakrishnan, V. and Lloyd-Yemoh, E. (2014). Stemming and lemmatization: A comparison of retrieval performances.Ayodele, T. O. (2010). Types of machine learning algorithms in new advances in machine learning. Croatia, Rijeka.Anguiano-Hernández, E. (2009). Naive bayes multinomial para clasificación de texto usando un esquema de pesado por clases.Agirre, E., Alegria, I., Arregi, X., Artola, X., de Ilarraza, A. D., Maritxalar, M., Sarasola, K., and Urkia, M. (1992). Xuxen: A spelling checker/corrector for basque based on two-level morphology. In Third Conference on Applied Natural Language Processing, pages 119–125.InvestigadoresLICENSElicense.txtlicense.txttext/plain; charset=utf-85879https://repositorio.unal.edu.co/bitstream/unal/84030/1/license.txteb34b1cf90b7e1103fc9dfd26be24b4aMD51ORIGINAL1010217881.2023.pdf1010217881.2023.pdfTesis de Maestría en Ciencias - Matemática Aplicadaapplication/pdf3164404https://repositorio.unal.edu.co/bitstream/unal/84030/3/1010217881.2023.pdfd4b8ef110bd4c89dd8b6359708bc8baeMD53THUMBNAIL1010217881.2023.pdf.jpg1010217881.2023.pdf.jpgGenerated Thumbnailimage/jpeg4598https://repositorio.unal.edu.co/bitstream/unal/84030/4/1010217881.2023.pdf.jpg712860d7950ae70a941f5348eba6c63aMD54unal/84030oai:repositorio.unal.edu.co:unal/840302023-08-09 23:04:10.846Repositorio Institucional Universidad Nacional de Colombiarepositorio_nal@unal.edu.coUEFSVEUgMS4gVMOJUk1JTk9TIERFIExBIExJQ0VOQ0lBIFBBUkEgUFVCTElDQUNJw5NOIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KCkxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgbG9zIGRlcmVjaG9zIHBhdHJpbW9uaWFsZXMgZGUgYXV0b3IsIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEsIGxpbWl0YWRhIHkgZ3JhdHVpdGEgc29icmUgbGEgb2JyYSBxdWUgc2UgaW50ZWdyYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsLCBiYWpvIGxvcyBzaWd1aWVudGVzIHTDqXJtaW5vczoKCgphKQlMb3MgYXV0b3JlcyB5L28gbG9zIHRpdHVsYXJlcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBzb2JyZSBsYSBvYnJhIGNvbmZpZXJlbiBhIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHVuYSBsaWNlbmNpYSBubyBleGNsdXNpdmEgcGFyYSByZWFsaXphciBsb3Mgc2lndWllbnRlcyBhY3RvcyBzb2JyZSBsYSBvYnJhOiBpKSByZXByb2R1Y2lyIGxhIG9icmEgZGUgbWFuZXJhIGRpZ2l0YWwsIHBlcm1hbmVudGUgbyB0ZW1wb3JhbCwgaW5jbHV5ZW5kbyBlbCBhbG1hY2VuYW1pZW50byBlbGVjdHLDs25pY28sIGFzw60gY29tbyBjb252ZXJ0aXIgZWwgZG9jdW1lbnRvIGVuIGVsIGN1YWwgc2UgZW5jdWVudHJhIGNvbnRlbmlkYSBsYSBvYnJhIGEgY3VhbHF1aWVyIG1lZGlvIG8gZm9ybWF0byBleGlzdGVudGUgYSBsYSBmZWNoYSBkZSBsYSBzdXNjcmlwY2nDs24gZGUgbGEgcHJlc2VudGUgbGljZW5jaWEsIHkgaWkpIGNvbXVuaWNhciBhbCBww7pibGljbyBsYSBvYnJhIHBvciBjdWFscXVpZXIgbWVkaW8gbyBwcm9jZWRpbWllbnRvLCBlbiBtZWRpb3MgYWzDoW1icmljb3MgbyBpbmFsw6FtYnJpY29zLCBpbmNsdXllbmRvIGxhIHB1ZXN0YSBhIGRpc3Bvc2ljacOzbiBlbiBhY2Nlc28gYWJpZXJ0by4gQWRpY2lvbmFsIGEgbG8gYW50ZXJpb3IsIGVsIGF1dG9yIHkvbyB0aXR1bGFyIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgcGFyYSBxdWUsIGVuIGxhIHJlcHJvZHVjY2nDs24geSBjb211bmljYWNpw7NuIGFsIHDDumJsaWNvIHF1ZSBsYSBVbml2ZXJzaWRhZCByZWFsaWNlIHNvYnJlIGxhIG9icmEsIGhhZ2EgbWVuY2nDs24gZGUgbWFuZXJhIGV4cHJlc2EgYWwgdGlwbyBkZSBsaWNlbmNpYSBDcmVhdGl2ZSBDb21tb25zIGJham8gbGEgY3VhbCBlbCBhdXRvciB5L28gdGl0dWxhciBkZXNlYSBvZnJlY2VyIHN1IG9icmEgYSBsb3MgdGVyY2Vyb3MgcXVlIGFjY2VkYW4gYSBkaWNoYSBvYnJhIGEgdHJhdsOpcyBkZWwgUmVwb3NpdG9yaW8gSW5zdGl0dWNpb25hbCwgY3VhbmRvIHNlYSBlbCBjYXNvLiBFbCBhdXRvciB5L28gdGl0dWxhciBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBwb2Ryw6EgZGFyIHBvciB0ZXJtaW5hZGEgbGEgcHJlc2VudGUgbGljZW5jaWEgbWVkaWFudGUgc29saWNpdHVkIGVsZXZhZGEgYSBsYSBEaXJlY2Npw7NuIE5hY2lvbmFsIGRlIEJpYmxpb3RlY2FzIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLiAKCmIpIAlMb3MgYXV0b3JlcyB5L28gdGl0dWxhcmVzIGRlIGxvcyBkZXJlY2hvcyBwYXRyaW1vbmlhbGVzIGRlIGF1dG9yIHNvYnJlIGxhIG9icmEgY29uZmllcmVuIGxhIGxpY2VuY2lhIHNlw7FhbGFkYSBlbiBlbCBsaXRlcmFsIGEpIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8gcG9yIGVsIHRpZW1wbyBkZSBwcm90ZWNjacOzbiBkZSBsYSBvYnJhIGVuIHRvZG9zIGxvcyBwYcOtc2VzIGRlbCBtdW5kbywgZXN0byBlcywgc2luIGxpbWl0YWNpw7NuIHRlcnJpdG9yaWFsIGFsZ3VuYS4KCmMpCUxvcyBhdXRvcmVzIHkvbyB0aXR1bGFyZXMgZGUgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciBtYW5pZmllc3RhbiBlc3RhciBkZSBhY3VlcmRvIGNvbiBxdWUgbGEgcHJlc2VudGUgbGljZW5jaWEgc2Ugb3RvcmdhIGEgdMOtdHVsbyBncmF0dWl0bywgcG9yIGxvIHRhbnRvLCByZW51bmNpYW4gYSByZWNpYmlyIGN1YWxxdWllciByZXRyaWJ1Y2nDs24gZWNvbsOzbWljYSBvIGVtb2x1bWVudG8gYWxndW5vIHBvciBsYSBwdWJsaWNhY2nDs24sIGRpc3RyaWJ1Y2nDs24sIGNvbXVuaWNhY2nDs24gcMO6YmxpY2EgeSBjdWFscXVpZXIgb3RybyB1c28gcXVlIHNlIGhhZ2EgZW4gbG9zIHTDqXJtaW5vcyBkZSBsYSBwcmVzZW50ZSBsaWNlbmNpYSB5IGRlIGxhIGxpY2VuY2lhIENyZWF0aXZlIENvbW1vbnMgY29uIHF1ZSBzZSBwdWJsaWNhLgoKZCkJUXVpZW5lcyBmaXJtYW4gZWwgcHJlc2VudGUgZG9jdW1lbnRvIGRlY2xhcmFuIHF1ZSBwYXJhIGxhIGNyZWFjacOzbiBkZSBsYSBvYnJhLCBubyBzZSBoYW4gdnVsbmVyYWRvIGxvcyBkZXJlY2hvcyBkZSBwcm9waWVkYWQgaW50ZWxlY3R1YWwsIGluZHVzdHJpYWwsIG1vcmFsZXMgeSBwYXRyaW1vbmlhbGVzIGRlIHRlcmNlcm9zLiBEZSBvdHJhIHBhcnRlLCAgcmVjb25vY2VuIHF1ZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhY3TDumEgY29tbyB1biB0ZXJjZXJvIGRlIGJ1ZW5hIGZlIHkgc2UgZW5jdWVudHJhIGV4ZW50YSBkZSBjdWxwYSBlbiBjYXNvIGRlIHByZXNlbnRhcnNlIGFsZ8O6biB0aXBvIGRlIHJlY2xhbWFjacOzbiBlbiBtYXRlcmlhIGRlIGRlcmVjaG9zIGRlIGF1dG9yIG8gcHJvcGllZGFkIGludGVsZWN0dWFsIGVuIGdlbmVyYWwuIFBvciBsbyB0YW50bywgbG9zIGZpcm1hbnRlcyAgYWNlcHRhbiBxdWUgY29tbyB0aXR1bGFyZXMgw7puaWNvcyBkZSBsb3MgZGVyZWNob3MgcGF0cmltb25pYWxlcyBkZSBhdXRvciwgYXN1bWlyw6FuIHRvZGEgbGEgcmVzcG9uc2FiaWxpZGFkIGNpdmlsLCBhZG1pbmlzdHJhdGl2YSB5L28gcGVuYWwgcXVlIHB1ZWRhIGRlcml2YXJzZSBkZSBsYSBwdWJsaWNhY2nDs24gZGUgbGEgb2JyYS4gIAoKZikJQXV0b3JpemFuIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgaW5jbHVpciBsYSBvYnJhIGVuIGxvcyBhZ3JlZ2Fkb3JlcyBkZSBjb250ZW5pZG9zLCBidXNjYWRvcmVzIGFjYWTDqW1pY29zLCBtZXRhYnVzY2Fkb3Jlcywgw61uZGljZXMgeSBkZW3DoXMgbWVkaW9zIHF1ZSBzZSBlc3RpbWVuIG5lY2VzYXJpb3MgcGFyYSBwcm9tb3ZlciBlbCBhY2Nlc28geSBjb25zdWx0YSBkZSBsYSBtaXNtYS4gCgpnKQlFbiBlbCBjYXNvIGRlIGxhcyB0ZXNpcyBjcmVhZGFzIHBhcmEgb3B0YXIgZG9ibGUgdGl0dWxhY2nDs24sIGxvcyBmaXJtYW50ZXMgc2Vyw6FuIGxvcyByZXNwb25zYWJsZXMgZGUgY29tdW5pY2FyIGEgbGFzIGluc3RpdHVjaW9uZXMgbmFjaW9uYWxlcyBvIGV4dHJhbmplcmFzIGVuIGNvbnZlbmlvLCBsYXMgbGljZW5jaWFzIGRlIGFjY2VzbyBhYmllcnRvIENyZWF0aXZlIENvbW1vbnMgeSBhdXRvcml6YWNpb25lcyBhc2lnbmFkYXMgYSBzdSBvYnJhIHBhcmEgbGEgcHVibGljYWNpw7NuIGVuIGVsIFJlcG9zaXRvcmlvIEluc3RpdHVjaW9uYWwgVU5BTCBkZSBhY3VlcmRvIGNvbiBsYXMgZGlyZWN0cmljZXMgZGUgbGEgUG9sw610aWNhIEdlbmVyYWwgZGUgbGEgQmlibGlvdGVjYSBEaWdpdGFsLgoKCmgpCVNlIGF1dG9yaXphIGEgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEgY29tbyByZXNwb25zYWJsZSBkZWwgdHJhdGFtaWVudG8gZGUgZGF0b3MgcGVyc29uYWxlcywgZGUgYWN1ZXJkbyBjb24gbGEgbGV5IDE1ODEgZGUgMjAxMiBlbnRlbmRpZW5kbyBxdWUgc2UgZW5jdWVudHJhbiBiYWpvIG1lZGlkYXMgcXVlIGdhcmFudGl6YW4gbGEgc2VndXJpZGFkLCBjb25maWRlbmNpYWxpZGFkIGUgaW50ZWdyaWRhZCwgeSBzdSB0cmF0YW1pZW50byB0aWVuZSB1bmEgZmluYWxpZGFkIGhpc3TDs3JpY2EsIGVzdGFkw61zdGljYSBvIGNpZW50w61maWNhIHNlZ8O6biBsbyBkaXNwdWVzdG8gZW4gbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMuCgoKClBBUlRFIDIuIEFVVE9SSVpBQ0nDk04gUEFSQSBQVUJMSUNBUiBZIFBFUk1JVElSIExBIENPTlNVTFRBIFkgVVNPIERFIE9CUkFTIEVOIEVMIFJFUE9TSVRPUklPIElOU1RJVFVDSU9OQUwgVU5BTC4KClNlIGF1dG9yaXphIGxhIHB1YmxpY2FjacOzbiBlbGVjdHLDs25pY2EsIGNvbnN1bHRhIHkgdXNvIGRlIGxhIG9icmEgcG9yIHBhcnRlIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgZGUgc3VzIHVzdWFyaW9zIGRlIGxhIHNpZ3VpZW50ZSBtYW5lcmE6CgphLglDb25jZWRvIGxpY2VuY2lhIGVuIGxvcyB0w6lybWlub3Mgc2XDsWFsYWRvcyBlbiBsYSBwYXJ0ZSAxIGRlbCBwcmVzZW50ZSBkb2N1bWVudG8sIGNvbiBlbCBvYmpldGl2byBkZSBxdWUgbGEgb2JyYSBlbnRyZWdhZGEgc2VhIHB1YmxpY2FkYSBlbiBlbCBSZXBvc2l0b3JpbyBJbnN0aXR1Y2lvbmFsIGRlIGxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhIHkgcHVlc3RhIGEgZGlzcG9zaWNpw7NuIGVuIGFjY2VzbyBhYmllcnRvIHBhcmEgc3UgY29uc3VsdGEgcG9yIGxvcyB1c3VhcmlvcyBkZSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSAgYSB0cmF2w6lzIGRlIGludGVybmV0LgoKCgpQQVJURSAzIEFVVE9SSVpBQ0nDk04gREUgVFJBVEFNSUVOVE8gREUgREFUT1MgUEVSU09OQUxFUy4KCkxhIFVuaXZlcnNpZGFkIE5hY2lvbmFsIGRlIENvbG9tYmlhLCBjb21vIHJlc3BvbnNhYmxlIGRlbCBUcmF0YW1pZW50byBkZSBEYXRvcyBQZXJzb25hbGVzLCBpbmZvcm1hIHF1ZSBsb3MgZGF0b3MgZGUgY2Fyw6FjdGVyIHBlcnNvbmFsIHJlY29sZWN0YWRvcyBtZWRpYW50ZSBlc3RlIGZvcm11bGFyaW8sIHNlIGVuY3VlbnRyYW4gYmFqbyBtZWRpZGFzIHF1ZSBnYXJhbnRpemFuIGxhIHNlZ3VyaWRhZCwgY29uZmlkZW5jaWFsaWRhZCBlIGludGVncmlkYWQgeSBzdSB0cmF0YW1pZW50byBzZSByZWFsaXphIGRlIGFjdWVyZG8gYWwgY3VtcGxpbWllbnRvIG5vcm1hdGl2byBkZSBsYSBMZXkgMTU4MSBkZSAyMDEyIHkgZGUgbGEgUG9sw610aWNhIGRlIFRyYXRhbWllbnRvIGRlIERhdG9zIFBlcnNvbmFsZXMgZGUgbGEgVW5pdmVyc2lkYWQgTmFjaW9uYWwgZGUgQ29sb21iaWEuIFB1ZWRlIGVqZXJjZXIgc3VzIGRlcmVjaG9zIGNvbW8gdGl0dWxhciBhIGNvbm9jZXIsIGFjdHVhbGl6YXIsIHJlY3RpZmljYXIgeSByZXZvY2FyIGxhcyBhdXRvcml6YWNpb25lcyBkYWRhcyBhIGxhcyBmaW5hbGlkYWRlcyBhcGxpY2FibGVzIGEgdHJhdsOpcyBkZSBsb3MgY2FuYWxlcyBkaXNwdWVzdG9zIHkgZGlzcG9uaWJsZXMgZW4gd3d3LnVuYWwuZWR1LmNvIG8gZS1tYWlsOiBwcm90ZWNkYXRvc19uYUB1bmFsLmVkdS5jbyIKClRlbmllbmRvIGVuIGN1ZW50YSBsbyBhbnRlcmlvciwgYXV0b3Jpem8gZGUgbWFuZXJhIHZvbHVudGFyaWEsIHByZXZpYSwgZXhwbMOtY2l0YSwgaW5mb3JtYWRhIGUgaW5lcXXDrXZvY2EgYSBsYSBVbml2ZXJzaWRhZCBOYWNpb25hbCBkZSBDb2xvbWJpYSBhIHRyYXRhciBsb3MgZGF0b3MgcGVyc29uYWxlcyBkZSBhY3VlcmRvIGNvbiBsYXMgZmluYWxpZGFkZXMgZXNwZWPDrWZpY2FzIHBhcmEgZWwgZGVzYXJyb2xsbyB5IGVqZXJjaWNpbyBkZSBsYXMgZnVuY2lvbmVzIG1pc2lvbmFsZXMgZGUgZG9jZW5jaWEsIGludmVzdGlnYWNpw7NuIHkgZXh0ZW5zacOzbiwgYXPDrSBjb21vIGxhcyByZWxhY2lvbmVzIGFjYWTDqW1pY2FzLCBsYWJvcmFsZXMsIGNvbnRyYWN0dWFsZXMgeSB0b2RhcyBsYXMgZGVtw6FzIHJlbGFjaW9uYWRhcyBjb24gZWwgb2JqZXRvIHNvY2lhbCBkZSBsYSBVbml2ZXJzaWRhZC4gCgo= |