Associate Professor (with Tenure)
Faculty of Information Science and Electrical Engineering
Kyushu University
744 Motooka, Nishi-ku
Fukuoka 819-0395, Japan
Email: malei [at] ait.kyushu-u.ac [dot] jp





img               Bungy@Rotorua, NZ


    We are recruiting now. If you have a strong background in the topics relevant to my research, I strongly encourage you to apply and join us. Currently, we have multiple RA positions for Ph.D. candidates and postdoc positions with competitive salary support! Please send your CV to malei.jobs@gmail.com to apply!


  1. (09/2020) Our paper Watch out! Motion is Blurring the Vision of Your Deep Neural Networks is accepted at NeurIPS 2020. This paper performs intensitive investigation on the impact of potential camera blure effects to the decision of a DNN!
  2. (09/2020) Join AAAI 2021 as PC Member.
  3. (08/2020) Our empirical study on Autonomous Driving Systems and its ML Components is accepted at Industry Track of ESEC/FSE 2020.
  4. (07/2020) Our papers MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems and Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness are accepted at ASE 2020.
  5. (07/2020) Two papers are accepted at ICSME 2020, NIER Track.
  6. (07/2020) Our DeepFake Series works DeepRhythm, DeepSonar, and Amora are accepted at ACM MM 2020.
  7. (07/2020) Our paper SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking is accepted at ECCV 2020. This paper proposes a novel and efficient attack to mislead online trackers along an incorrect or specified trajectory!
  8. (06/2020) Join ESEC/FSE 2021 as PC Member.
  9. (06/2020) Join NeurIPS’20 as a reviewer.
  10. (05/2020) Our paper FakeSpotter is accepted at IJCAI 2020, a previous verison is reported as AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, by the Synced AI Industry Review!
  11. (05/2020) Our paper DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices is available on CoRR arXiv!
  12. (04/2020) Join AISec’20, ASE'20, ICSME'20 as PC member.
  13. (03/2020) Give tutorials regarding Testing and Analysis of Stateful Deep Learning Systems at IVADO Training Program on Software Engineering for Machine Learning Applications, at Montreal, Canada, March 2-6 2020.
  14. (02/2020) Give Invited talk Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning at Polytechnique Montreal, Canada, Feb.6, 2020
  15. (02/2020) Give Invited talk Quality and Reliability Assurance for Deep Learning Systems at School of Computer Science, McGill University, Canada, Feb.5, 2020
  16. (02/2020) Our paper Towards Byzantine-resilient Learning in Decentralized Systems is available on CoRR arXiv!
  17. (02/2020) Our paper ABBA: Saliency-Regularized Motion-Based Adversarial Blur Attack is available on CoRR arXiv!
  18. (01/2020) Our paper FakeLocator: Robust Localization of GAN-Based Face Manipulations via Semantic Segmentation Networks with Bells and Whistles is available on CoRR arXiv!
  19. (12/2019) I am honored to be invited to Dagstuhl Seminar 20091:SE4ML - Software Engineering for AI-ML-based Systems to be held on February 23 – 28, 2020
  20. (12/2019) I am honored to be invited to give lectures at IVADO Training Program on Software Engineering for Machine Learning Applications, to be held at Montreal, Canada, March 2-6 2020.
  21. (12/2019) Our paper Amora: Black-box Adversarial Morphing Attack is available on CoRR arXiv!
  22. (12/2019) Our paper Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty is accepted at ICSE'2020, This work is the very early work that discovers the new uncommon adversarial examples that represent the challenging vulnerability of deep learning software, we are now working to seek new ways to counteract these issues!
  23. (11/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons, on the emerging direction of testing and anlaysis of machine learning is accepted at IEEE Transactions on Software Engineering (TSE), a previous early draft is available at CoRR arXiv, we are working on the camera-ready version and would make it available soon!
  24. (11/2019) Our research framework DeepStudio: a Framework of Securing Deep Learning Systems receives the 2019 CCF Research Prototype Competition Award (Second Prize).
  25. (11/2019) Our papers Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning and Generating Adversarial Examples for Holding Robustness of Source Code Processing Models are accepted at AAAI 2020.
  26. (10/2019) Our paper Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning receives the ACM SIGSOFT Distinguished Paper Award in ASE'19.
  27. (10/2019) Media coverage: AI vs AI: ‘FakeSpotter’ Studies Neurons to Bust DeepFakes, our work FakeSpotter is reported by the media of AI Industry Review!
  28. (10/2019) Our paper Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers is finally accepted at Software Testing, Verification AND Reliability (STVR) Journal after two years continous effort !
  29. (09/2019) Our papers DeepMutation++, DeepHunter, and DeepStellar are accepted and will be demonstrated at ASE 2019 as tool papers !
  30. (09/2019) Our papers Wuji and Deep Learning Platform/Framework Compatibility Study are accepted at ASE 2019 as full research papers !
  31. (08/2019) Our paper CDA: Characterising Deprecated Android APIs is accepted at Empirical Software Engineering (EMSE) Journal !
  32. (07/2019) Our paper An Empirical Study of Common Challenges in Developing Deep Learning Applications is accepted at ISSRE'19 !
  33. (07/2019) Two papers: Safe Inputs Generation for Black-box Systems and MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform are accepted at ICECCS 2019!
  34. (06/2019) Our survey paper Machine Learning Testing: Survey, Landscapes and Horizons is available on CoRR arXiv!
  35. (05/2019) Our paper DiffChaser: Detecting Disagreements for Deep Neural Networks is accepted at IJCAI'19 !
  36. (05/2019) Our paper DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems is accepted at FSE'19 !
  37. (05/2019) I am invited to join AISec'19 Program Committee collocated with CSS'19 this year, please considier to submit your high quality papers!
  38. (05/2019) I am invited to visit and give a talk on Testing Deep Learning System Towards Intelligence of Things at Google Waterloo on June 5!
  39. (05/2019) Our paper DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks is accepted at ISSTA'19 !
  40. (04/2019) I am happy to be invited to join and give a talk on NII Shonan Meeting on Software Engineering for Machine Learning Systems (SEMLS) which will be held on October this year.
  41. (04/2019) I am invited by Machine Learning and Software Engineering Community of Japan (MLSE) to give tutorial on "Software Testing of Deep Learning System: from the Past to the Future"
  42. (04/2019) I am co-organizing the first MLST Workshop Collocated with ICST!
  43. (03/2019) I am happy to be invited to join and give a talk on The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal which will be held on May this year.
  44. (02/2019) I am co-organizing the first AI4Mobile Workshop Collocated with SANER!
  45. (12/2018) Our paper VULTRON: Catching Vulnerable Smart Contracts Once and for All ICSE'19 as an NIER Track Paper!
  46. (12/2018) Our paper DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems SANER'19 as an ERA Track Paper!
  47. (09/2018) Our paper Practical Fault Attack on Deep Neural Networks is accepted at CCS'18 as a Poster Paper!
  48. (09/2018) Our paper Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer is available on CoRR arXiv. This is the very first work towards in-depth analysis on the robustness of Differtentiable Neural Computer!
  49. (08/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems receive the ACM SIGSOFT Distinguished Paper Award in ASE'18.
  50. (08/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is accepted at ISSRE'18
  51. (07/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is accepted at ASE'18
  52. (06/2018) Our paper Combinatorial Testing for Deep Learning Systems is available on CoRR arXiv. This is the first combinatorial testing for Deep Neural Networks!
  53. (06/2018) Our paper Practical Fault Attack on Deep Neural Networks is available on CoRR arXiv. This is the first practical attacks on Deep Neural Networks using laser technology!
  54. (05/2018) Our paper Characterising Deprecated Android APIs won the FOSS Impact Paper Award in MSR'18.
  55. (05/2018) Our paper DeepMutation: Mutation Testing of Deep Learning Systems is available on CoRR arXiv. This is the first mutation testing technique for Deep Neural Networks!
  56. (03/2018) Our paper DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems is available on CoRR arXiv. This paper proposes a set of testing critiera with multi-granuality for Deep Neural Networks!
  57. (01/2018) Our paper Characterising Deprecated Android APIs has been accepted to appear in MSR'18.
  58. (03/2017) Our paper Model-based API Testing of Apache ZooKeeper won the IEEE Best Presentation Award at ICST'17.
  59. (07/2016) Our paper Runtime monitoring for concurrent systems has been accepted to appear in RV'16.
  60. (09/2016) Our paper An empirical study on the effects of code visibility on program testability has been accepted to appear in Springer Software Quliaty Journal (SQJ).
  61. (03/2016) Our paper Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs has been nominated for the IEEE Best Candidate Paper Award in SANER'16.
  62. (11/2015) Our paper GRT: Program-analysis-guided Random Testing won the ACM SIGSOFT Distinguished Paper Award at ASE'15.
  63. (11/2015) Our paper Incremental Web Application Testing using Page Object won the IEEE Best Paper Award at HotWeb'15.

About Me

I am an Associate professor (with Tenure) and co-leading a research group PANGU Lab at Kyushu University, My current research mainly focuses on the interdisciplinary research fields of Software Engineering, Security and Artificial Intelligence, and especially on proposing quality assurance and security solutions for machine learning engineering. I received a B.E. degree from Department of Computer Science and Engineering of Shanghai Jiaotong Unviersity in 2009, M.E. and Ph.D degrees from The University of Tokyo in 2011 and 2014, respectively. During my Ph.D. program, I studied half a year in the Mathematics and Computer Science department of Technische Universität München(TUM). After receiving the Ph.D., I worked as a Research Fellow in the joint projects by The University of Tokyo, Chiba University, Advanced Industrial Science and Technology (AIST/ITRI) and National Institute of Informatics, and later joined in Harbin Institute of Technology as an Associate Professor in 2016, during which in 2018 I got the chance to visit Prof. Liu Yang's group at NTU, together, we made a series of work and progress in testing and analysis of deep learning systems. In 2019, I joined Kyushu University and now is an Associate Professor.

I love theories, but I like even more to transfer theories into practice!
I have several research interests and working directions. My current major research interest mainly focuses on general purpose security and quality assurance for deep learning systems. I believe the robust deep learning system would be the key driving force for novel technology of the future. It does not solely rely on the foundational research from the machine learning community, it also highly requires the contribution from software engineering community, security community, etc; the novel interdisciplinary direction so be called Machine Learning Engineering, an extremely exciting and demanding area of the future! We have conducted consecutive works along the general purpose security and quality assurance of DL systems (i.e., testing, analysis, verification, attacks, defenses), to bridge vast demands from industry application and the academic research. Exciting research results would be continuously updated!

In the past, my research interest mainly lies around software engineering and programming language, in particular automated software testing, verification, analysis, evolution, mining, etc. I also worked on green computation and optimization solutions for cloud and big data centers. In addition, I am also interested in genomic data analsyis to uncover life mystery and to search for health solutions through deep learning techniques.

    See also at Google Scholar or DBLP.

  1. Qing Gu, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Jian Wang, Bing Yu, Wei Feng, Yang Liu
    Watch out! Motion is Blurring the Vision of Your Deep Neural Networks
    Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020, CORE Rank A*)
  2. Zi Peng, Jinqiu Yang, Tse-Hsun Chen, Lei Ma
    A First Look at the Integration of Machine Learning Models in Complex Autonomous Driving Systems - A Case Study on Apollo
    The 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Industry Track (ESEC/FSE 2020, CORE Rank A*/CCF A)
  3. Xiaoning Du, Yi Li, Xiaofei Xie, Lei Ma, Yang Liu, Jianjun Zhao
    MARBLE: Model-Based Robustness Analysis of Stateful Deep Learning Systems
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  4. David Berend, Xiaofei Xie, Lei Ma, Lingjun Zhou, Yang Liu, Chi Xu, Jianjun Zhao
    Cats Are Not Fish: Deep Learning Testing Calls for Out-Of-Distribution Awareness
    The 35th IEEE/ACM International Conference on Automated Software Engineering. (ASE 2020, CORE Rank A/CCF A)
  5. Xuhong Ren, Bing Yu, Hua Qi, Felix Juefei-Xu, Zhuo Li, Wanli Xue, Lei Ma and Jianjun Zhao
    Few-Shot Guided Mix for DNN Repairing
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  6. Yuechen Wu, Yingfeng Chen, Xiaofei Xie, Bing Yu, Changjie Fan and Lei Ma
    Regression Testing of Massively Multiplayer Online Role-Playing Games
    The 36th IEEE International Conference on Software Maintenance and Evolution, NIER Track (ICSME 2020, CORE Rank A)
  7. Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Wei Feng, Yang Liu, Jianjun Zhao
    DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  8. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    DeepSonar: Towards Effective and Robust Detection of AI-Synthesized Fake Voices
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  9. Run Wang,Felix Juefei-Xu, Yihao Huang, Qing Guo, Xiaofei Xie, Lei Ma, Yang Liu
    Amora: Black-box Adversarial Morphing Attack
    The 28th ACM International Conference on Multimedia. (ACM MM 2020, CORE Rank A*/CCF A)
  10. Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, Yang Liu
    SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking
    The 16th European Conference on Computer Vision (ECCV 2020, CORE Rank A)
  11. Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, Yang Liu
    FakeSpotter: A Simple Baseline for Spotting AI-Synthesized Fake Faces
    The 30th International Joint Conference on Artificial Intelligence - Paciffic Rim International Conference on Artificial Intelligence, 7 pages, Japan, July 2020. (IJCAI 2020, CORE Rank A*/CCF A) (Media Coverage by Synced AI Industry Review)
  12. Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun
    Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
    The 42nd International Conference on Software Engineering, 12 pages, 23-29 May 2020, Seoul, South Korea (ICSE’20, CORE Rank A*/CCF A)
  13. Jie M. Zhang, Mark Harman, Lei Ma, Yang Liu
    Machine Learning Testing: Survey, Landscapes and Horizons
    IEEE Transactions on Software Engineering, 2020. (TSE, CCF Rank A, SCI-indexed, Impact Factor:4.778)
  14. Huangzhao Zhang, Zhuo Li, Ge Li, Lei Ma, Yang Liu, Zhi Jin
    Generating Adversarial Examples for Holding Robustness of Source Code Processing Models
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  15. Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu
    Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning.
    The 34th AAAI Conference on Artificial Intelligence, New York, USA, Feb 7-12, 2020. (AAAI'20, CCF Rank A)
  16. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    CDA: Characterising Deprecated Android APIs
    Empirical Software Engineering Journal, 2020 (EMSE, SCI-indexed, Impact Factor: 4.457)
  17. Cyrille Artho, Kazuuki Banzai, Quentin Gros, Guillaume Rousset, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based Testing of Apache ZooKeeper: Fundamental API Usage and Watchers
    Software Testing, Verification and Reliability, 2019 (STVR, SCI-indexed, Impact Factor: 1.171)
  18. Yan Zheng, Xiaofei Xie, Ting Su, Lei Ma, Jianye Hao, Zhaopeng Meng, Yang Liu, Ruimin Shen, Yinfeng Chen, Changjie Fan
    Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning
    ACM SIGSOFT Distinguished Paper Award.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  19. Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li
    An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  20. Qiang Hu, Lei Ma, Xiaofei Xie, Bing Yu, Yang Liu, and Jianjun Zhao
    DeepMutation++: a Mutation Testing Framework for Deep Learning Systems
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  21. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao.
    A Quantitative Analysis Framework for Recurrent Neural Network.
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  22. Xiaofei Xie, Hongxu Chen, Yi Li, Lei Ma, Yang Liu, and Jianjun Zhao
    Coverage-guided Fuzzing for Feedforward Neural Networks
    In Proc. 34th IEEE/ACM Conference on Automated Software Engineering (ASE 2019), (Tool Demo Track), San Diego, California, USA, November 11-15, 2019. (ASE'19, CCF Rank A)
  23. Tianyi Zhang, Cuiyun Gao, Lei Ma, Michael R. Lyu and Miryung Kim
    An Empirical Study of Common Challenges in Developing Deep Learning Applications
    The 30th International Symposium on Software Reliability Engineering (ISSRE'19, CORE Rank A)
  24. Bai Xue, Yang Liu, Lei Ma, Xiyue Zhang, Meng Sun and Xiaofei Xie
    Safe Inputs Generation for Black-box Systems
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  25. Ruitao Feng, Sen Chen, Xiaofei Xie, Lei Ma, Shangwei Lin and Yang Liu
    MobiDroid: A Performance-Sensitive Malware Detection System on Mobile Platform
    The 24th International Conference on Engineering of Complex Computer Systems (ICECCS'19, CORE Rank A)
  26. Xiaofei Xie, Lei Ma, Haijun Wang, Yuekang Li, Yang Liu, Xiaohong Li
    DiffChaser: Detecting Disagreements for Deep Neural Networks
    The 29th International Joint Conference on Artificial Intelligence, Macau, China, August 2019 (IJCAI'19, CCF Rank A)
  27. Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Yang Liu and Jianjun Zhao.
    DeepStellar: Model-Based Quantitative Analysis of Stateful Deep Learning Systems
    The 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, Tallinn, Estonia, August 2019 (FSE'19, CCF Rank A)
  28. Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Minhui Xue, Hongxu Chen, Yang Liu, Jianjun Zhao, Bo Li, Jianxiong Yin, Simon See
    DeepHunter: A Coverage-Guided Fuzz Testing Framework for Deep Neural Networks
    The 28th International Symposium on Software Testing and Analysis, Beijing, China, July, 2019 (ISSTA'19, CCF Rank A)
  29. Haijun Wang, Yi Li, Shang-Wei Lin, Lei Ma, Yang Liu
    VULTRON: Catching Vulnerable Smart Contracts Once and for All
    The 41st ACM/IEEE International Conference on Software Engineering, Montreal, QC, Canada, May, 2019, (ICSE'19, CCF Rank A)
  30. Chao Xie, Hua Qi,Lei Ma, Jianjun Zhao
    DeepVisual: A Visual Programming Tool for Deep Learning Systems
    The 27th IEEE/ACM International Conference on Program Comprehension, Montreal, QC, Canada, May, 2019, (ICPC'19, CCF Rank B)
  31. Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, Jianjun Zhao
    DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 614-618, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  32. Yanjie Zhao, Haoyu Wang, Lei Ma, Yuxin Liu, Li Li, John Grundy
    Knowledge Graphing Git Repositories: A Preliminary Study
    2019 IEEE 26th International Conference on Software Analysis, Evolution and Reengineering, pp. 599-603, Hangzhou, China, 2019 (SANER'19, CCF Rank B)
  33. Sen Chen, Minhui Xue, Lingling Fan, Lei Ma,, Yang Liu, Lihua Xu
    How Can We Craft Large-Scale Android Malware? An Automated Poisoning Attack.
    2019 IEEE International Workshop on Artificial Intelligence for Mobile, pp. 21-24, Hangzhou, China, 2019. (AI4Mobile'19)
  34. Sen Chen, Lingling Fan, Ting Su, Lei Ma,, Yang Liu, Lihua Xu
    Automated Cross-Platform GUI Code Generation for Mobile Apps.
    IEEE International Workshop on Artificial Intelligence for Mobile, pp. 13-16, Hangzhou, China, 2019. (AI4Mobile'19)
  35. Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu
    Practical Fault Attack on Deep Neural Networks.
    The 25th ACM Conference on Computer and Communications Security, Poster Paper (CCS'18 CCF Rank A)
  36. Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems. ACM SIGSOFT Distinguished Paper Award.
    The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18, CORE Rank A)
  37. Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang
    DeepMutation: Mutation Testing of Deep Learning Systems.
    The 29th IEEE International Symposium on Software Reliability Engineering (ISSRE'18 CORE Rank A)
  38. Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong
    Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer.
    arXiv:1809.02444
  39. Qiang Hu, Lei Ma, Jianjun Zhao
    DeepGraph: A PyCharm Tool for Visualizing and Understanding Deep Learning Models.
    The 25th Asia-Pacific Software Engineering Conference (APSEC'18 CORE Rank B)
  40. Li Li, Jun Gao, Tegawendé F. Bissyandé, Lei Ma, Xin Xia and Jacques Klein
    Characterising Deprecated Android APIs. ACM FOSS Impact Paper Award.
    The 15th International Conference on Mining Software Repositories, Gothenburg, Sweden (MSR'18, CORE Rank A)
  41. Cyrille Valentin Artho, Quentin Gros, Guillaume Rousset, Kazuaki Banzai, Lei Ma, Takashi Kitamura, Masami Hagiya, Yoshinori Tanabe and Mitsuharu Yamamoto
    Model-based API Testing of Apache ZooKeeper.IEEE Best Presentation Award.
    The 10th IEEE International Conference on Software Testing, Verification and Validation, Tokyo, Japan (ICST'17, CORE Rank A)
  42. Lei Ma,Bing Yu, Hiroyuki Sato and Yadong Wang
    Collaborative Computation Offloading in Heterogeneous Asynchronous Cloud Environment.
    2017 IEEE 41st Annual Computer Software and Applications Conference (COMPSAC'17, CORE Rank B)
  43. Yoriyuki Yamagata, Cyrille Artho, Masami Hagiya, Jun Inoue, Lei Ma, Yoshinori Tanabe, Mitsuharu Yamamoto
    Runtime monitoring for concurrent systems.
    The 16th International Conference in Runtime Verification, Madrid, Spain, September 2016, Springer.(RV'16, CORE Rank C)
  44. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on the effects of code visibility on program testability.
    Software Quality Journal, 2016, Springer.(SQJ, SCI-Indexed, Impact Factor: 2.141)
  45. Lei Ma, Cheng Zhang, Bing Yu, Jianjun Zhao
    Retrofitting Automatic Testing through Library Tests Reusing.
    The IEEE 24th International Conference on Program Comprehension, May 2016, Austin, USA. (ICPC'16, CCF B)
  46. Lei Ma, Cyrille Artho, Masami Hagiya, Yoshinori Tanabe, Yoriyuki Yamagata, Alexander Kohan and Mitsuharu Yamamoto
    Monitoring Distributed Applications with Java Pathfinder.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  47. Alexander Kohan, Mitsuharu Yamamoto, Cyrille Artho, Yoriyuki Yamagata, Lei Ma, Masami Hagiya and Yoshinori Tanabe
    Java Pathfinder on Android Devices.
    2016 JPF Workshop 2016, November 2016, Seattle, USA. (JPF Workshop'16)
  48. Jiabin Ye, Cheng Zhang, Lei Ma, Haibo Yu, Jianjun Zhao
    Efficient and Precise Dynamic Slicing for Client-Side JavaScript Programs. IEEE Best Paper Candidate Award.
    The IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering, March 2016, Osaka, Japan. (SANER'16, CCF B)
  49. Cyrille Artho, Lei Ma
    Classification of Randomly Generated Test Cases.
    The first IEEE International Workshop on Validating Software Tests, March, 2016, Osaka, Japan. (VST'16 colocated with SANER'16)
  50. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: Program-analysis-guided random testing. ACM SIGSOFT Distinguished Paper Award.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
  51. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato, Johannes Gmeiner, Rudolf Ramler
    GRT: An automated test generator using orchestrated program analysis.
    The 30th IEEE/ACM International Conference on Automated Software Engineering, November 2015, Lincoln, USA. (ASE'15, CCF A)
  52. Lei Ma, Cheng Zhang, Bing Yu, Hiroyuki Sato
    An empirical study on effects of code visibility on code coverage of software testing.
    The IEEE/ACM 10th International Workshop on Automation of Software Test, May 2015, Florence, Italy. (AST'15, CORE Rank C)
  53. Lei Ma, Cyrille Artho, Cheng Zhang, Hiroyuki Sato,Masami Hagiya, Yoshinori Tanabe, Mitsuharu Yamamoto
    GRT at the SBST 2015 tool competition. IEEE Best Testing Tool Award.
    The IEEE 8th International Workshop on Search-based Software Testing, May 2015, Florence, Italy.
  54. Bing Yu, Lei Ma, Cheng Zhang
    Incremental Web Application Testing using Page Object.IEEE Best Paper Award.
    The IEEE International Workshop on Hot Topics in Web Systems and Technologies, November 2015, Washington, D.C, USA. (HotWeb'15, CORE Rank B)
  55. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    Efficient testing of software product lines via centralization.
    The ACM International Conference on Generative Programming: Concepts and Experiences, September 2014, Vasteras, Sweden. (GPCE'14, CORE Rank B)
  56. Lei Ma, Cyrille Artho, Cheng Zhang, and Hiroyuki Sato
    When Project Centralization and Random Testing Meet: Efficient Automatic Testing of Multiple Software Product Variants.
    The 2014 JSSST Software Conference, July 2014, Nagoya, Japan. (JSSST'14)
  57. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Managing Product Variants by Project Centralization.
    Lecture Notes on Software Engineering (LNSE), January 2014.
  58. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Project centralization based on graph coloring.
    The ACM 29th International Symposium on Applied Computing, March 2014, Gyeongju, South Korea. (SAC'14, CORE Rank B)
  59. Lei Ma, Cyrille Artho, and Hiroyuki Sato
    Improving automatic centralization by version separation.
    IPSJ Transactions on Programming, 6(4), December 2013.
  60. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: Tangible Devices for Designing Robot Navigation Routes.
    Computer Software Journal, August 2013.
  61. Lei Ma, Cyrille Artho, Hiroyuki Sato
    Analyzing distributed Java applications by automatic centralization.
    The IEEE 37th Annual Computer Software and Applications Conference Workshops, July 2013, Kyoto, Japan.(COMPSACW'13)
  62. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: User-Configurable Device Network for Robot Navigation.
    The 14th IFIP International Conference on Human-Computer Interaction, September 2013, Cape Town, South Africa. (INTERACT'13, CORE Rank A)
  63. Kentaro Ishii, Haipeng Mi, Lei Ma, Natsuda Laokulrat, Masahiko Inami, Takeo Igarashi
    Pebbles: an interactive configuration tool for indoor robot navigation.
    The 25th Annual ACM Symposium on User Interface Software and Technology, October 2012, Cambridge, MA, USA. (UIST'12, CORE Rank A)
  64. Lei Ma, Hiroyuki Sato
    An Annotated Type System for Inlining.
    The 2012 IPSJ Programming Language Workshop (PRO, Oral)

  • Testing and Analysis of Stateful Deep Learning Systems
    Institute for Data Valorisation (IVADO) Training Program on the Software Engineering of Machine Learning-based Systems, Montreal, Canada, 2020
  • Automatic Video Game Testing via Evolutionary Deep Reinforcement Learning
    Polytechnique Montreal, Canada, Feb.6, 2020
  • Quality and Reliability Assurance for Deep Learning Systems
    School of Computer Science, McGill University, Canada, Feb.5, 2020
  • Towards Analyzing Stateful Deep Learning System
    University of British Columbia, Canada, Dec.13, 2019
  • Testing and Analysis of Deep Learning Systems
    National Software Application Conference of China (NASAC'19), Nov. 24, Hangzhou, China, 2019
  • Quality and Reliability for Machine Learning Systems
    Celebration of ASE 2019, 10 Nov. San Diego, California, USA, 2019
  • DeepStellar: Analysis of Stateful Deep Learning Systems
    JST ERATO MMSD (Metamathematics for System Design) Project, Tokyo, Japan, 7 Nov. 2019
  • Towards Quality and Reliability Engineering of Deep Learning Systems
    The 8th Asian Workshop on Advanced Software Engineering (AWASE 2019), Fukuoka, Japan, 28-29 Oct., 2019
  • Towards Testing and Analysis of Deep Learning Systems
    MLSE International Symposium 2019, Tokyo, Japan, 18 Oct., 2019
  • Towards Testing Deep Learning Systems
    Shonan Meeting SEMLS (Software Engineering for Machine Learning Systems), Shonan Village, Japan, 13-17 Oct., 2019
  • Quality Assurance of Deep Learning Engineering
    NAIST Data Science Day, Nara, Japan, 11 Oct., 2019
  • Quality Assurance of Machine Learning Systems
    Intelligent Software Testing Symposium, Nanjing, China, 17 Sep., 2019
  • Testing Deep Learning System Towards Intelligence of Things
    Google Waterloo, Canada, June 5, 2019
  • Towards Quality Assurance of Deep Learning Systems
    The Software Engineering for Machine Learning Applications (SEMLA) International Symposium, Polytechnique Montreal, Canada, May 23 – 24, 2019
  • Software Testing of Deep Learning System: from the Past to the Future
    Japan Machine Learning Systems Engineering Community (MLSE, 日本機械学習工学研究会), Tokyo, Japan, April 26, 2019
  • DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
    2018 National Software Application Conference of China (NASAC'18), Nov. 23, ShenZhen, China, 2018
  • Quality Assurance for Deep Learning System
    KTH Royal Institute of Technology, Stockholm, Sweden, Sep. 13, 2018
  • Deep Learning Quality Assurance in Industrial Scale
    2018 Nvidia NVAITC AI Symposium, Singapore, August 30, 2018
  • Program Analysis Guided Intelligent Automated Testing
    2017 National Software Testing Competition Symposium, Nanjing, China, October 25, 2017
  • A Comprehensive Study on Automated Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 24, 2017
  • Program Analysis Guided Software Verification and Testing
    National Institute of Advanced Industrial Science and Technology (AIST), Osaka, Japan, March 13, 2016
  • Guided Random Testing: When Program Analysis Meets Automatic Test Generation
    Static Analysis Meets Runtime Verification NII Shonan Meeting, Shonan Village Center, Japan, March 15-19, 2015

2021

  • PC member, The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021)
  • PC member, The ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE 2021)

2020

  • PC member, The 13th ACM Workshop on Artificial Intelligence and Security (AISec 2020)
  • Reviewer, The Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS 2020)
  • PC member, The 35rd IEEE/ACM International Conference on Automated Software Engineering (ASE 2020)
  • PC member, The 36th IEEE International Conference on Software Maintenance and Evolution (ICSME’20)
  • PC member, The 21st Genetic and Evolutionary Computation Conference (GECCO 2020)
  • PC member, This 27th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER 2020)
  • Workshop Chair, The 27th Asia-Pacific Software Engineering Conference (APSEC 2020)
  • Doctoral Symposium Co-Chair, THE 22nd International Conference on Formal Engineering Methods (ICFEM 2020)
  • Publicity and Social Media Chair, The 12th Asia-Pacific Symposium on Internetware (Internetware 2020)
  • PC member, The 25th International Conference on Engineering of Complex Computer Systems (ICECCS 2020)
  • PC member, The 8th International Workshop on Realizing Artificial Intelligence Synergies in Software Engineering (RAISE 2020)
  • PC member, The IEEE Intl. Conf. on Softw.Testing, Verification and Validation, Poster Track (ICST 2020)
  • PC member, The 14th International Symposium on Theoretical Aspects of Software Engineering (TASE 2020)
  • PC member, The 43th IEEE Computer Society International Conference on Computers, Software \& Applications (COMPSAC 2020)
  • PC member, The 20th IEEE International Conference on Software Quality, Reliabiilty, and Security & Applications (QRS'2020)
  • PC member, The IEEE International Workshop on Intelligent Bug Fixing (IBF 2020)
  • Reviewer, IEEE Transactions on Software Engineering (TSE)
  • ACM Transactions on Software Engineering and Methodology (TOSEM)
  • Reviewer, Empirical Software Engineering Journal (EMSE)
  • Journal of Systems and Software (JSS)

    • 2019

    • PC member, The 34rd IEEE/ACM International Conference on Automated Software Engineering (ASE'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Machine Learning and Software Testing(MLST'2019)
    • Organizing Comitte Member, The IEEE International Workshop on Artificial Intelligence for Mobile (AI4Mobile)
    • Local Chair & PC member, The 11th Asia-Pacific Symposium on Internetware (Internetware'19)
    • Local Chair, The 8th Asia-Pacific Workshop of Advanced Software Engineering (AWASE'19)
    • PC Member, The 26th IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER'19 ERA)
    • PC member, The 19th IEEE International Conference on Software Quality, Reliabiilty, and Security(QRS'2019)
    • PC member, 19th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'2019)
    • PC member, The 43th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, International Symposium on Computing and Networking (CANDAR'19)
    • PC member, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'19)
    • PC member, The First International Workshop on Intelligent Bug Fixing (IBF'2019))
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'19)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'19)
    • PC Member, The 19th IEEE International Conference on Scalable Computing and Communications (ScalCom'19)
    • Reviewer, IEEE Transactions on Software Engineering (TSE)
    • Reviewer, Journal of Software: Evolution and Process (JSME)
    • Reviewer, Journal of Systems and Software (JSS)
    • Reviewer, The 16th International Conference on Mining Software Repositories (MSR'2019)

    2018

    • Reviewer, The 11th ACM Workshop on Artificial Intelligence and Security (AISec'18)
    • Reviewer, IEEE Transactions on Reliability
    • Reviewer, The 15th International Conference on Managed Languages & Runtimes (ManLang'18)
    • Reviewer, The 25th ACM Conference on Computer and Communications Security (CCS'18)
    • Reviewer, The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18)
    • PC member, The 42th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'18)
    • Reviewer, ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA'18)
    • Reviewer, ETAPS 24th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'18)
    • PC Member, The 18th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'18)
    • PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'18)
    • PC Member, The 18th IEEE International Conference on Scalable Computing and Communications (ScalCom'18)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'18)
    • PC Member, The 26th European Conference On Information Systems (ECIS'18)
    • PC member, The Exellent Ph.D Research Forum (NASAC'2018)
    • Reviewer, Springer Journal of Computer Virology and Hacking Techniques
    • PC member, The 18th International Conference on Advances in System Testing and Validation Lifecycle (VALID'18)
    • PC Member, International Symposium on Computing and Networking (CANDAR'18)

    2017

    • Publicity Chair, The 24th International Conference on Engineering of Complex Computer Systems(ICECCS'17)
    • PC member, The 17th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM'17)
    • Reviewer, Elsevier Journal of Systems and Software (JSS)
    • PC member, The 41th IEEE Computer Society International Conference on Computers, Software & Applications (COMPSAC'17)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'17)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'17)
    • PC Member, International Symposium on Computing and Networking (CANDAR'17)
    • IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER): 2018, 2019

    2016 & 2015

    • External reviewer tool track, The 31st IEEE/ACM International Conference on Automated Software Engineering (ASE'16)
    • PC member, The Eighth International Conference on Advances in System Testing and Validation Lifecycle (VALID'16)
    • PC Member, IEEE International Workshop on Validating Software Tests (VST'16)
    • Publication Chair & PC Member, IEEE International Conference on Mobile Cloud Computing, Services, and Engineering (Mobile Cloud'16)
    • Technical reviewer, International Conference on Formal Aspects of Component Software (FACS'16)
    • Reviewer, Elsevier Science of Computer Programming Journal (FTSCS'16)
    • Tool Panel Member, International Workshop on Search-Based Software Testing (SBST'15)
  • ACM SIGSOFT Distinguished Paper Award (ASE'19)
  • ACM SIGSOFT Distinguished Paper Award (ASE'18)
  • ACM SIGSOFT Distinguished Paper Award (ASE'15)
  • Research Prototype Competition Award (Second Prize, CCF NASAC'19)
  • 2018 Baidu-NASAC Academic Star Award
  • ACM FOSS Impact Paper Award (MSR'18)
  • Leadership Award (ICECCS'17)
  • IEEE Best Presentation Award (ICST'17)
  • IEEE Best Paper Candidate Award (SANER'16)
  • IEEE Best Paper Award (HotWeb'15)
  • IEEE Best Testing Tool Award (SBST'15)
  • JASSO Hornor scholoarship (2011 and 2013)
  • MEXT Scholarship sponsored by Japanese Government (2009)
  • Excellent Academic Scholarship of Shanghai Jiao Tong University (2006)