Enhancing the GLANCE Framework for Line-Level Defect Prediction: An Empirical Study of Semantically-Aware Metrics and Non-Linear Classifiers

Zahid Mujaddid, Ema Utami

Abstract


Line-level defect prediction (LLDP) is critical for reducing software maintenance costs, yet its industrial adoption is often hindered by high false alarm rates that erode developer trust. While the state-of-the-art GLANCE-LR framework offers a lightweight solution, it relies on linear classifiers and purely syntactic heuristics, failing to capture the non-linear defect patterns and semantic risks associated with complex code constructs. To bridge the gap between operational efficiency and semantic awareness, this paper proposes GLANCE++, an enhanced framework that integrates a non-linear LightGBM classifier for refined file-level filtering and introduces three semantically-aware line metrics: Cognitive Complexity Score (CCS), API-Weighted Number of Function Calls (AW-NFC), and Variable-Write Count (VWC). These metrics shift the prediction paradigm from counting tokens to modeling "code risk." Empirical evaluation on 19 open-source Java projects (142 releases) reveals that while the non-linear file classifier yields marginal gains, the semantic line-level metrics achieve statistically significant improvements in precision and False Alarm Rate (FAR). However, this increased selectivity introduces a trade-off, resulting in reduced recall compared to the baseline. Our findings demonstrate that improving the semantic intelligence of heuristics yields far greater impact than increasing model complexity. This suggests that future LLDP research should prioritize theoretically grounded risk metrics over computationally expensive deep learning architectures to ensure practical deployment in real-time CI/CD pipelines.

Keywords


Software quality assurance; Line-level defect prediction; Heuristic-based models; Defect localization; GLANCE

Full Text:

Link Download

References


Agrawal, A., Fu, W., Chen, D., Shen, X., & Menzies, T. (2021). How to “DODGE” complex software analytics. IEEE Transactions on Software Engineering, 47(10), 2182–2194.

Ahmad, M. J., Goseva-Popstojanova, K., & Lutz, R. R. (2024). The untold impact of learning approaches on software fault-proneness predictions: An analysis of temporal aspects. Empirical Software Engineering, 29(4), 87.

Assim, M., Obeidat, Q., & Hammad, M. (2020). Software defects prediction using machine learning algorithms. 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI) (pp. 1–6). Presented at the 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy (ICDABI), Sakheer, Bahrain: IEEE. Retrieved August 6, 2025, from https://ieeexplore.ieee.org/document/9325677/

Bagaev, M., Khabibrakhmanova, A., Sabaev, G., & Bugayenko, Y. (2024). The impact of mutability on cyclomatic complexity in java. arXiv. Retrieved December 8, 2025, from https://arxiv.org/abs/2410.10425

Chen, J., Xu, J., Cai, S., Wang, X., Chen, H., & Li, Z. (2024). Software defect prediction approach based on a diversity ensemble combined with neural network. IEEE Transactions on Reliability, 73(3), 1487–1501.

Fakhoury, S., Roy, D., Ma, Y., Arnaoudova, V., & Adesope, O. (2020). Measuring the impact of lexical and structural inconsistencies on developers’ cognitive load during bug localization. Empirical Software Engineering, 25(3), 2140–2178.

Fu, M., & Tantithamthavorn, C. (2022). LineVul: A transformer-based line-level vulnerability prediction. Proceedings of the 19th International Conference on Mining Software Repositories (pp. 608–620). Presented at the MSR ’22: 19th International Conference on Mining Software Repositories, Pittsburgh Pennsylvania: ACM. Retrieved September 28, 2024, from https://dl.acm.org/doi/10.1145/3524842.3528452

Guo, Z., Liu, S., Liu, X., Lai, W., Ma, M., Zhang, X., Ni, C., et al. (2023). Code-line-level bugginess identification: How far have we come, and how far have we yet to go? ACM Transactions on Software Engineering and Methodology, 32(4), 1–55.

Hata, H., Mizuno, O., & Kikuno, T. (2012). Bug prediction based on fine-grained module histories. 2012 34th International Conference on Software Engineering (ICSE) (pp. 200–210). Presented at the 2012 34th International Conference on Software Engineering (ICSE 2012), Zurich: IEEE. Retrieved August 6, 2025, from http://ieeexplore.ieee.org/document/6227193/

Kamei, Y., Matsumoto, S., Monden, A., Matsumoto, K., Adams, B., & Hassan, A. E. (2010). Revisiting common bug prediction findings using effort-aware models. 2010 IEEE International Conference on Software Maintenance (pp. 1–10). Presented at the 2010 IEEE 26th International Conference on Software Maintenance (ICSM), Timi oara, Romania: IEEE. Retrieved August 5, 2025, from http://ieeexplore.ieee.org/document/5609530/

Kamei, Y., Monden, A., Matsumoto, S., Kakimoto, T., & Matsumoto, K. (2007). The effects of over and under sampling on fault-prone module detection. First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) (pp. 196–204). Presented at the First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007), Madrid, Spain: IEEE. Retrieved August 5, 2025, from http://ieeexplore.ieee.org/document/4343747/

Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., et al. (2017). LightGBM: a highly efficient gradient boosting decision tree. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in Neural Information Processing Systems (Vol. 30). Curran Associates, Inc. Retrieved from https://proceedings.neurips.cc/paper_files/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf

Lavazza, L., Abualkishik, A. Z., Liu, G., & Morasca, S. (2023). An empirical evaluation of the “Cognitive Complexity” measure as a predictor of code understandability. Journal of Systems and Software, 197, 111561.

Li, X., Jiang, J., Benton, S., Xiong, Y., & Zhang, L. (2021). A large-scale study on API misuses in the wild. 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST) (pp. 241–252). Presented at the 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), Porto de Galinhas, Brazil: IEEE. Retrieved August 8, 2025, from https://ieeexplore.ieee.org/document/9438601/

Mahbub, P., & Rahman, M. M. (2024). Predicting line-level defects by capturing code contexts with hierarchical transformers. 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 308–319). Presented at the 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Rovaniemi, Finland: IEEE. Retrieved September 28, 2024, from https://ieeexplore.ieee.org/document/10589817/

Mienye, I. D., & Sun, Y. (2022). A survey of ensemble learning: Concepts, algorithms, applications, and prospects. IEEE Access, 10, 99129–99149.

Mujaddid, Z. (2025, December 8). archazid/GLANCE: V1.0.0. Zenodo. Retrieved December 9, 2025, from https://zenodo.org/doi/10.5281/zenodo.17858503

Oueslati, R., Ouertani, M. W., Amdouni, A., & Manita, G. (2025). MS-FSOA-LightGBM: Multi-strategy starfish optimization algorithm with LightGBM for software defect prediction. Procedia Computer Science, 29th International Conference on Knowledge-Based and Intelligent Information & Engineering Systems (KES 2025), 270, 4054–4063.

Pachouly, J., Ahirrao, S., Kotecha, K., Selvachandran, G., & Abraham, A. (2022). A systematic literature review on software defect prediction using artificial intelligence: Datasets, data validation methods, approaches, and tools. Engineering Applications of Artificial Intelligence, 111, 104773.

Parashar, A., Kumar Goyal, R., Kaushal, S., & Kumar Sahana, S. (2022). Machine learning approach for software defect prediction using multi-core parallel computing. Automated Software Engineering, 29(2), 44.

Pascarella, L., Palomba, F., & Bacchelli, A. (2019). Fine-grained just-in-time defect prediction. Journal of Systems and Software, 150, 22–36.

Pereira, J. H. A., Souza, A. L. O. T. D., & Pinto, V. H. S. C. (2021). Cognitive load analyzer: A support tool for cognitive-driven development. Brazilian Symposium on Software Engineering (pp. 468–473). Presented at the SBES ’21: Brazilian Symposium on Software Engineering, Joinville Brazil: ACM. Retrieved December 8, 2025, from https://dl.acm.org/doi/10.1145/3474624.3476011

Pornprasit, C., & Tantithamthavorn, C. K. (2023). DeepLineDP: Towards a deep learning approach for line-level defect prediction. IEEE Transactions on Software Engineering, 49(1), 84–98.

Qiu, Shaojian, Huang, H., Luo, J., Kuang, Y., & Luo, H. (2024). BAFLineDP: Code bilinear attention fusion framework for line-level defect prediction. 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) (pp. 1–12). Presented at the 2024 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), Rovaniemi, Finland: IEEE. Retrieved October 12, 2024, from https://ieeexplore.ieee.org/document/10589784/

Qiu, Shaoming, E, B., He, J., & Liu, L. (2025). Survey of software defect prediction features. Neural Computing and Applications, 37(4), 2113–2144.

Rahman, F., Khatri, S., Barr, E. T., & Devanbu, P. (2014). Comparing static bug finders and statistical prediction. Proceedings of the 36th International Conference on Software Engineering (pp. 424–434). Presented at the ICSE ’14: 36th International Conference on Software Engineering, Hyderabad India: ACM. Retrieved August 8, 2025, from https://dl.acm.org/doi/10.1145/2568225.2568269

Roy, D., Pramanik, R., & Sarkar, R. (2024). Margin-aware adaptive-weighted-loss for deep learning based imbalanced data classification. IEEE Transactions on Artificial Intelligence, 5(2), 776–785.

Segalotto, M., Bolzan, W., & Farias, K. (2023). Effects of modularization on developers’ cognitive effort in code comprehension tasks: A controlled experiment. Proceedings of the XXXVII Brazilian Symposium on Software Engineering (pp. 206–215). Presented at the SBES 2023: XXXVII Brazilian Symposium on Software Engineering, Campo Grande Brazil: ACM. Retrieved December 8, 2025, from https://dl.acm.org/doi/10.1145/3613372.3613387

Sharma, G., & Singh, P. (2025). Comparative study: Word2Vec versus TF-IDF in software defect predictions. In S. Namasudra, N. Kar, S. K. Patra, & D. Taniar (Eds.), Data Science and Network Engineering, Lecture Notes in Networks and Systems (Vol. 1165, pp. 95–107). Singapore: Springer Nature Singapore. Retrieved August 18, 2025, from https://link.springer.com/10.1007/978-981-97-8336-6_8

Stradowski, S., & Madeyski, L. (2023). Industrial applications of software defect prediction using machine learning: A business-driven systematic literature review. Information and Software Technology, 159, 107192.

Szymański, K., & Ochodek, M. (2023). On the applicability of the pareto principle to source-code growth in open source projects (pp. 781–789). Presented at the 18th Conference on Computer Science and Intelligence Systems. Retrieved December 10, 2025, from https://annals-csis.org/Volume_35/drp/5221.html

Wan, Z., Xia, X., Hassan, A. E., Lo, D., Yin, J., & Yang, X. (2020). Perceptions, expectations, and challenges in defect prediction. IEEE Transactions on Software Engineering, 46(11), 1241–1266.

Wang, H., Zhuang, W., & Zhang, X. (2021). Software defect prediction based on gated hierarchical LSTMs. IEEE Transactions on Reliability, 70(2), 711–727.

Wang, S., Huang, L., Gao, A., Ge, J., Zhang, T., Feng, H., Satyarth, I., et al. (2023). Machine/deep learning for software engineering: A systematic literature review. IEEE Transactions on Software Engineering, 49(3), 1188–1231.

Wattanakriengkrai, S., Thongtanunam, P., Tantithamthavorn, C., Hata, H., & Matsumoto, K. (2022). Predicting defective lines using a model-agnostic technique. IEEE Transactions on Software Engineering, 48(5), 1480–1496.

Wohlin, C., Runeson, P., Höst, M., Ohlsson, M. C., Regnell, B., & Wesslén, A. (2024). Experimentation in software engineering. Berlin, Heidelberg: Springer Berlin Heidelberg. Retrieved August 18, 2025, from https://link.springer.com/10.1007/978-3-662-69306-3

Yang, F., Zhong, F., Zeng, G., Xiao, P., & Zheng, W. (2024). LineFlowDP: A deep learning-based two-phase approach for line-level defect prediction. Empirical Software Engineering, 29(2), 50.

Zhang, H., & Cheung, S. C. (2013). A cost-effectiveness criterion for applying software defect prediction models. Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering (pp. 643–646). Presented at the ESEC/FSE’13: Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, Saint Petersburg Russia: ACM. Retrieved August 8, 2025, from https://dl.acm.org/doi/10.1145/2491411.2494581

Zhao, Y., Damevski, K., & Chen, H. (2023). A systematic survey of just-in-time software defect prediction. ACM Computing Surveys, 55(10), 1–35.

Zhong, Y., Song, K., Lv, S., & He, P. (2021). An empirical study of software metrics diversity for cross-project defect prediction. (C. Chai, Ed.)Mathematical Problems in Engineering, 2021, 1–11.

Zhou, Y., Yang, Y., Lu, H., Chen, L., Li, Y., Zhao, Y., Qian, J., et al. (2018). How far we have progressed in the journey? An examination of cross-project defect prediction. ACM Transactions on Software Engineering and Methodology, 27(1), 1–51.

Zhu, J., Huang, Y., Chen, X., Wang, R., & Zheng, Z. (2023). SyntaxLineDP: A line-level software defect prediction model based on extended syntax information. 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS) (pp. 83–94). Presented at the 2023 IEEE 23rd International Conference on Software Quality, Reliability, and Security (QRS), Chiang Mai, Thailand: IEEE. Retrieved October 10, 2024, from https://ieeexplore.ieee.org/document/10366602/




DOI: http://dx.doi.org/10.35671/telematika.v18i2.3196

Refbacks

  • There are currently no refbacks.


 


Indexed by:

   

Telematika
ISSN: 2442-4528 (online) | ISSN: 1979-925X (print)
Published by : Universitas Amikom Purwokerto
Jl. Let. Jend. POL SUMARTO Watumas, Purwonegoro - Purwokerto, Indonesia


Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License .