Career Profile

I am a Post-doctoral Research Associate at the University of Bristol. My PhD research focuses on improving the reliability and effectiveness of unit test suites to support the development of secure and trustworthy software systems. In my PhD work, I introduced a comprehensive framework for assessing test suite health, going beyond traditional metrics like code coverage to include factors such as fault detection capability, brittleness, flakiness, and realism—qualities that are essential for identifying vulnerabilities and ensuring software resilience. By developing automated techniques to assess and repair test suites, I aim to strengthen testing infrastructures that help detect regressions and security flaws more effectively. My work bridges automated software testing, empirical software engineering, and software security assurance.

Experiences

Postdoctoral Research Associate

May 2025 - Present
University of Bristol, UK

Software Engineer

April 2019 - December 2020
Oracle Corporation, Malaysia

I worked as a full-stack software developer on Oracle’s next-generation Cruise system, using Oracle JET (Knockout.js/React), Spring REST APIs, and OracleDB, with CI/CD pipelines via Jenkins to streamline deployment and testing. I also maintained the legacy Cruise system built in C#-ASP.Net and VB.Net, resolving critical payment system issues and improving database performance. In an agile, test-driven development environment, I consistently applied a test-first approach, achieving around 85% branch coverage in regression test suites and ensuring thorough validation of all production fixes. Additionally, I contributed to R&D efforts to modernize and standardize the Oracle Hospitality front-end framework, collaborating with the design team to unify UI components across mobile and web platforms.

Research Software Engineer

September 2018 - January 2019
University of Sheffield, UK

I worked with the OAK research group at the University of Sheffield collaborating NHS doctors to develop an online Visual Acuity Test application, working as a full-stack web and Android developer. My contributions focused on implementing real-time video communication using SocketIO and WebRTC, deploying the NodeJS server, and managing data storage with MongoDB. I also maintained regression test suites using the Espresso framework for the Android (Java) app and Jest for the NodeJS backend to ensure reliability across the system.

Internship

May 2016 - August 2016
Cybersecurity Malaysia, Malaysia

I conducted security vulnerability analyses (penetration testing) for one of the Malaysian Government’s back-end systems.

Publications

  • Test Suite Health: Automatically Improving the Reliability and Effectiveness of Test Suites
  • Muhammad Firhard Roslan. Supervised by Prof. Phil McMinn.
    Ph.D. thesis, University of Sheffield, UK, 2025.
  • Beyond Test Flakiness: A Manifesto for a Holistic Approach to Test Suite Health
  • McMinn, P., Roslan, M.F., and Kapfhammer, G.M.
    2025 IEEE/ACM International Flaky Tests Workshop (FTW) Proceedings
  • Viscount: A direct method call coverage tool for Java
  • Roslan, M.F., Rojas, J.M., and McMinn, P.
    2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) — Tool Demo Track
  • Private—Keep Out? Understanding How Developers Account for Code Visibility in Unit Testing
  • Roslan, M.F., Rojas, J.M., and McMinn, P.
    2024 IEEE International Conference on Software Maintenance and Evolution (ICSME) — Research Track
  • Do Automatic Test Generation Tools Generate Flaky Tests?
  • Gruber, M., Roslan, M.F., Parry, O., Scharnböck, F., McMinn, P., and Fraser G.
    2024 IEEE International Conference on Software Engineering (ICSE) — Research Track
  • An empirical comparison of EvoSuite and DSpot for improving developer-written test suites with respect to mutation score
  • Roslan, M.F., Rojas, J.M., and McMinn, P.
    International Symposium on Search Based Software Engineering (SSBSE) 2022

    Projects

    Viscount - A library that identify production methods and its visibility that are being called directly in JUnit tests.
    EvoSuite AMP - An automated approach that improves assertion quality in unit tests by adding assertions that can improve mutation score. This work is based on the EvoSuite framework.
    EvoSuite UTOPIA - An automated technique that refactors unit tests by replacing direct calls to non-public methods with equivalent sequences using only public APIs, improving test maintainability and realism while preserving coverage. This work is based on the EvoSuite framework.