Grant: Alliance Grants

With recent advancements in Large Language Models (LLMs) designed for coding, Artificial Intelligence for Software Engineering (AI4SE) remains pivotal in enhancing both developer productivity and software quality. Such LLMs pre-trained on vast repositories of SE data have shown strong potential for general-purpose SE tasks like code completion and generation. However, to excel at more specialized SE tasks like code transformation, companies still need to optimize LLMs to their specific context to avoid LLMs' tendency to "hallucinate", i.e., make up certain predictions. This partnership aims to develop advanced LLM-based methods that enable the translation of code across various software frameworks, focusing on detailed method-level and extensive project-aware file-level transformations. Such code transformation enhances the adaptability of complex software systems to changes in both the functional requirements and technological landscape (e.g., moving away from a legacy framework), ensuring that these systems remain efficient, scalable, and up-to-date with the latest industry standards and practices. While achieving this primary goal, we also recognize the value of an important support task: creating tools and methodologies that can improve the efficiency of benchmarking LLMs for code transformation. This involves formulating a conceptual framework that aligns developers' organizational needs with automated code evaluation strategies and an implementation that facilitates easier assessment and comparison of open-source and proprietary LLMs. The utility of this framework and its application will be assessed using open-source data (NSERC component of the project) and a proprietary code base from our partner, with Mitacs internships providing further assistance.