silp_nlp at LLMs4OL 2024 Tasks A, B, and C: Ontology Learning through Prompts with LLMs

Pankaj Kumar Goyal; Sumit Singh; Uma Shanker Tiwary

doi:10.52825/ocp.v4i.2485

Authors

Pankaj Kumar Goyal Indian Institute of Information Technology Allahabad https://orcid.org/0009-0007-5501-9111
Sumit Singh Indian Institute of Information Technology Allahabad https://orcid.org/0000-0002-3292-4131
Uma Shanker Tiwary Indian Institute of Information Technology Allahabad https://orcid.org/0000-0001-7206-9013

DOI:

https://doi.org/10.52825/ocp.v4i.2485

Keywords:

Large Language Models, LLMs, Ontology Learning, Prompt-based Learning, GPT, Llama

Abstract

Our team, silp_nlp, participated in the LLMs4OL Challenge at ISWC 2024, engaging in all three tasks focused on ontology generation. The tasks include predicting the type of a given term, extracting a hierarchical taxonomy between two terms, and extracting non-taxonomy relations between two terms. To accomplish these tasks, we used machine learning models such as random forest, logistic regression and generative models for the first task and generative models such as llama-3-8b-instruct, mistral 8*7b and GPT-4o-mini for the second and third tasks. Our results showed that generative models performed better for certain domains, such as subtasks A6 and B2. However, for other domains, the prompt-based technique failed to generate promising results. Our team achieved first place in six subtasks and second place in five subtasks, demonstrating our expertise in ontology generation.

Downloads

Download data is not yet available.

References

[1] H. Babaei Giglou, J. D’Souza, and S. Auer, “Llms4ol: Large language models for ontology learning,” in The Semantic Web – ISWC 2023, T. R. Payne, V. Presutti, G. Qi, et al., Eds., Cham: Springer Nature Switzerland, 2023, pp. 408–427, ISBN: 978-3-031-47240-4.

[2] H. Babaei Giglou, J. D’Souza, and S. Auer, “Llms4ol 2024 overview: The 1st large lan- guage models for ontology learning challenge,” Open Conference Proceedings, vol. 4, Oct. 2024.

[3] A. Q. Jiang, A. Sablayrolles, A. Mensch, et al., Mistral 7b, 2023. arXiv: 2310 . 06825 [cs.CL]. [Online]. Available: https://arxiv.org/abs/2310.06825.

[4] A. Dubey, A. Jauhri, A. Pandey, et al., The llama 3 herd of models, 2024. arXiv: 2407. 21783 [cs.AI]. [Online]. Available: https://arxiv.org/abs/2407.21783.

[5] openai. “Gpt-4o.” (2024), [Online]. Available: https://openai.com/index/hello-gpt- 4o/.

[6] A. Konys, “Knowledge repository of ontology learning tools from text,” Procedia Computer Science, vol. 159, pp. 1614–1628, 2019, Knowledge-Based and Intelligent Information Engineering Systems: Proceedings of the 23rd International Conference KES2019, ISSN: 1877-0509. DOI: https://doi.org/10.1016/j.procs.2019.09.332. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1877050919315339.

[7] C. Fellbaum and G. Miller, “Automated discovery of wordnet relations,” in WordNet: An Electronic Lexical Database. 1998, pp. 131–151.

[8] C. H. Hwang, “Incompletely and imprecisely speaking: Using dynamic ontologies for rep- resenting and retrieving information,” in Knowledge Representation Meets Databases, 1999. [Online]. Available: https://api.semanticscholar.org/CorpusID:11502906.

[9] L. Khan and F. Luo, “Ontology construction for information selection,” in 14th IEEE In- ternational Conference on Tools with Artificial Intelligence, 2002. (ICTAI 2002). Proceed- ings., 2002, pp. 122–127. DOI: 10.1109/TAI.2002.1180796.

[10] Z. Akkalyoncu Yilmaz, S. Wang, W. Yang, H. Zhang, and J. Lin, “Applying BERT to docu- ment retrieval with birch,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, S. Pado´ and R. Huang, Eds., Hong Kong, China: Association for Computational Linguistics, Nov. 2019, pp. 19–24. DOI: 10.18653/v1/D19-3004. [Online]. Available: https://aclanthology.org/D19- 3004.

[11] OL’00: Proceedings of the First International Conference on Ontology Learning - Volume 31, Berlin, Germany: CEUR-WS.org, 2000.

[12] F. Dalvi, A. R. Khan, F. Alam, N. Durrani, J. Xu, and H. Sajjad, “Discovering latent con- cepts learned in bert,” ArXiv, vol. abs/2205.07237, 2022. [Online]. Available: https:// api.semanticscholar.org/CorpusID:248810913.

[13] H. Babaei Giglou, J. D’Souza, S. Sadruddin, and S. Auer, “Llms4ol 2024 datasets: Toward ontology learning with large language models,” Open Conference Proceedings, vol. 4, Oct. 2024.

[14] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidi- rectional transformers for language understanding,” in Proceedings of the 2019 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), J. Burstein, C. Do- ran, and T. Solorio, Eds., Minneapolis, Minnesota: Association for Computational Lin- guistics, Jun. 2019, pp. 4171–4186. DOI: 10 . 18653 / v1 / N19 - 1423. [Online]. Available: https://aclanthology.org/N19-1423.