Project Title: HiFi-Gas: Hierarchical Federated Learning Incentive Mechanism Enhanced Gas Usage Estimation

Keywords: Fairness, Privacy-Preservation, Incentive Mechanism Design, Hierarchical FL

Introduction:
Gas usage estimation plays a critical role in various aspects of the power generation and delivery business, including budgeting, resource planning, and environmental preservation. Federated Learning (FL) has demonstrated its potential in enhancing the accuracy and reliability of gas usage estimation by enabling distributedly owned data to be leveraged, while ensuring privacy and confidentiality. However, to effectively motivate stakeholders to contribute their high-quality local data and computational resources for this purpose, incentive mechanism design is key. In this project, we design the Hierarchical FL Incentive mechanism for the Gas usage estimation (HiFi-Gas) system. It is designed to cater to the unique structure of gas companies and their affiliated heating stations. HiFi-Gas provides effective incentivization in a hierarchical federated learning framework that consists of a horizontal federated learning (HFL) component for effective collaboration among gas companies and multiple vertical federated learning (VFL) components for the gas company and its affiliated heating stations. To motivate active participation and ensure fairness among gas companies and heating stations, we incorporate a multi-dimensional contribution-aware reward distribution function that considers both data quality and model contributions. Since its deployment in the ENN Group in December 2022, HiFi-Gas has successfully provided incentives for gas companies and heating stations to actively participate in FL training, resulting in more than 12% higher average gas usage estimation accuracy and substantial gas procurement cost savings. This implementation marks the first successful deployment of a hierarchical FL incentive approach in the energy industry.

Publications:

1. Hao Sun, Xiaoli Tang, Chengyi Yang, Zhenpeng Yu, Xiuli Wang, Qijie Ding, Zengxiang Li & Han Yu, "HiFi-Gas: Hierarchical Federated Learning Incentive Mechanism Enhanced Gas Usage Estimation," in Proceedings of the 36th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24), pp. 22824-22832, 2024. (Innovative Application of AI Award)

Project Title: Transformer-empowered Multi-modal Item Embedding for Enhanced Image Search in E-Commerce

Keywords: Foundation Models

Introduction:
Over the past decade, significant advances have been made in the field of image search for e-commerce applications. Traditional image-to-image retrieval models, which focus solely on image details such as texture, tend to overlook useful semantic information contained within the images. As a result, the retrieved products might possess similar image details, but fail to fulfil the user's search goals. Moreover, the use of image-to-image retrieval models for products containing multiple images results in significant online product feature storage overhead and complex mapping implementations. In this project, we design the Multi-modal Item EmbeddingModel (MIEM) to address these limitations. It is capable of utilizing both textual information and multiple images about a product to construct meaningful product features. By leveraging semantic information from images, MIEM effectively supplements the image search process, improving the overall accuracy of retrieval results. MIEM has become an integral part of the Shopee image search platform, with its features covering over 400 million products. Since its deployment in March 2023, it has achieved a remarkable 9.90% increase in terms of clicks per user and a 4.23% boost in terms of orders per user for the image search feature on the Shopee e-commerce platform.

Publications:

1. Chang Liu, Peng Hou, Anxiang Zeng & Han Yu, "Transformer-empowered Multi-modal Item Embedding for Enhanced Image Search in E-Commerce," in Proceedings of the 36th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24), pp. 22770-22778, 2024. (Innovative Application of AI Award)

Project Title: IBCA: An Intelligent Platform for Social Insurance Benefit Qualification Status Assessment

Keywords: Fairness, Explainability, Privacy-Preservation

Introduction:
Social insurance benefits qualification assessment is an important task to ensure that retirees enjoy their benefits according to the regulations. It also plays a key role in curbing social security frauds. In this project, we develop the Intelligent Benefit Certification and Analysis (IBCA) platform, an AI-empowered platform for verifying the status of retirees to ensure proper dispursement of funds in Shandong province, China. Based on an improved Gated Recurrent Unit (GRU) neural network, IBCA aggregates missing value interpolation, temporal information, and global and local feature extraction to perform accurate retiree survival rate prediction. Based on the predicted results, a reliability assessment mechanism based on Variational Auto-Encoder (VAE) and Monte-Carlo Dropout (MC Dropout) is executed to perform reliability assessment. Deployed since November 2019, the IBCA platform has been adopted by 12 cities across the Shandong province, handling over 50 terabytes of data. It has empowered human resources and social services, civil affairs, and health care institutions to collaboratively provide highquality public services. Under the IBCA platform, the efficiency of resources utilization as well as the accuracy of benefit qualification assessment have been significantly improved.

Publications:

1. Yuliang Shi, Lin Cheng, Cheng Jiang, Hui Zhang, Guifeng Li, Xiaoli Tang, Han Yu, Zhiqi Shen & Cyril Leung, "IBCA: An Intelligent Platform for Social Insurance Benefit Qualification Status Assessment," in Proceedings of the 36th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-24), pp. 22815-22823, 2024. (Innovative Application of AI Award)

Project Title: FedOBD: Efficient Training of Large-Scale Industrial Fault Diagnostic Models through Federated Opportunistic Block Dropout

Keywords: Robustness, Privacy-Preservation, Large-Scale Models

Introduction:
Artificial intelligence (AI)-empowered industrial fault diagnostics is important in ensuring the safe operation of industrial applications. Since complex industrial systems often involve multiple industrial plants (possibly belonging to different companies or subsidiaries) with sensitive data collected and stored in a distributed manner, collaborative fault diagnostic model training often needs to leverage federated learning (FL). As the scale of the industrial fault diagnostic models are often large and communication channels in such systems are often not exclusively used for FL model training, existing deployed FL model training frameworks cannot train such models efficiently across multiple institutions. In this project, we develop and deploy the Federated Opportunistic Block Dropout (FedOBD) approach for industrial fault diagnostic model training. By decomposing large-scale models into semantic blocks and enabling FL participants to opportunistically upload selected important blocks in a quantized manner, it significantly reduces the communication overhead while maintaining model performance. Since its deployment in ENN Group in February 2022, FedOBD has served two coal chemical plants across two cities in China to build industrial fault prediction models. It helped the company reduce the training communication overhead by over 70% compared to its previous AI Engine, while maintaining model performance at over 85% test F1 score. To our knowledge, it is the first successfully deployed dropout-based FL approach.

Publications:

1. Yuanyuan Chen, Zichen Chen, Pengcheng Wu & Han Yu, "FedOBD: Opportunistic Block Dropout for Efficiently Training Large-scale Neural Networks through Federated Learning," in Proceedings of the 32nd International Joint Conference on Artificial Intelligence (IJCAI'23), pp. 3541-3549, 2023.
2. Yuanyuan Chen, Zichen Chen, Sheng Guo, Yansong Zhao, Zelei Liu, Pengcheng Wu, Chengyi Yang, Zengxiang Li & Han Yu, "Efficient Training of Large-Scale Industrial Fault Diagnostic Models through Federated Opportunistic Block Dropout," in Proceedings of the 35th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-23), pp. 15485-15493, 2023. (Innovative Application of AI Award)

Project Title: HACFL: Hierarchical Auctioning in Crowd-based Federated Learning

Keywords: Fairness, Privacy-Preservation, Data Valuation, Trading

Introduction:
In open collaborative federated learning (FL) taking place within a network of participants, anyone can initiate an FL model training task. Participants can either bid to join an FL task, or help refer others in their own networks. Currently, there is a lack of simulation and benchmarking tools to support research in this domain. In this research, we built Hierarchical Auctioning in Crowd-based Federated Learning (HACFL), a benchmark platform which enables simulations of FL networks with any given topology and reputation-aware hierarchical auction-based FL team formation to support research in this domain. It consists of a configurable back-end simulation system and a web-based interactive user interface, allowing end users and researchers to visualize trust-based open collaborative FL training processes. Results show that leveraging such an ecosystem of FL participants not only improves model performance, but also improves social welfare.

Publications:

1. Yulan Gao, Yansong Zhao & Han Yu, "Multi-Tier Client Selection for Mobile Federated Learning Networks," in Proceedings of the 2023 IEEE International Conference on Multimedia and Expo (ICME'23), pp. 666-671, 2023.

Project Title: Contribution-Aware Federated Learning (CAreFL)

Keywords: Explainability, Fairness, Privacy-Preservation

Introduction:
Artificial intelligence (AI) is a promising technology to transform the healthcare industry. Due to the highly sensitive nature of patient data, federated learning (FL) is often leveraged to build models for smart healthcare applications. Existing deployed FL frameworks cannot address the key issues of varying data quality and heterogeneous data distributions across multiple institutions in this sector. In this project, we design, develop and deploy the Contribution-Aware Federated Learning (CAreFL) framework for smart healthcare. It provides fair and explainable FL participant contribution evaluation in an efficient and privacy-preserving manner, and optimizes the FL model aggregation approach based on the evaluation results. Since its deployment in Yidu Cloud Technology Inc. in 2021, CAreFL has served 8 well-established medical institutions in China to build healthcare decision support models. It can perform contribution evaluations 2.84 times faster than the best existing approach, and has improved the average accuracy of the resulting models by 2.62% compared to the previous system (which is significant in industrial settings). To our knowledge, it is the first contribution-aware federated learning successfully deployed in the healthcare industry.

Publications:

1. Zelei Liu, Yuanyuan Chen, Yansong Zhao, Han Yu, Yang Liu, Renyi Bao, Jinpeng Jiang, Zaiqing Nie, Qian Xu & Qiang Yang, "Contribution-Aware Federated Learning for Smart Healthcare," in Proceedings of the 34th Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-22), pp. 12396-12404, 2022. (Innovative Application of AI Award)
2. Zelei Liu, Yuanyuan Chen, Han Yu, Yang Liu & Lizhen Cui. GTG-Shapley: Efficient and accurate participant contribution evaluation in federated learning. ACM Transactions on Intelligent Systems and Technology, ACM (2022).

Project Title: CrowdFL: A Marketplace for Crowdsourced Federated Learning

Keywords: Privacy-Preservation, Data Valuation, Trading, Crowdsourcing

Introduction:
Amid data privacy concerns, Federated Learning (FL) has emerged as a promising machine learning paradigm that enables privacy-preserving collaborative model training. However, there exists a need for a platform that matches data owners (supply) with model requesters (demand). In this project, we built CrowdFL, a platform to facilitate the crowdsourcing of FL model training. It coordinates client selection, model training, and reputation management, which are essential steps for the FL crowdsourcing operations. By implementing model training on actual mobile devices, we demonstrate that the platform improves model performance and training efficiency. To the best of our knowledge, it is the first platform to support crowdsourcing-based FL on edge devices.

Publications:

1. Daifei Feng, Cicilia Helena, Wei Yang Bryan Lim, Jer Shyuan Ng, Hongchao Jiang, Zehui Xiong, Jiawen Kang, Han Yu, Dusit Niyato & Chunyan Miao, "CrowdFL: A Marketplace for Crowdsourced Federated Learning," in Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI-22), 2022.

Project Title: FedVision: An Online Visual Object Detection Platform Powered by Federated Learning

Keywords: Privacy-Preservation, Visualization, Process Management

Introduction:
Visual object detection is a computer vision-based artificial intelligence (AI) technique which has many practical applications (e.g., fire hazard monitoring). However, due to privacy concerns and the high cost of transmitting video data, it is highly challenging to build object detection models on centrally stored large training datasets following the current approach. Federated learning (FL) is a promising approach to resolve this challenge. Nevertheless, there currently lacks an easy to use tool to enable computer vision application developers who are not experts in federated learning to conveniently leverage this technology and apply it in their systems. In this project, we built FedVision - a machine learning engineering platform to support the development of federated learning powered computer vision applications. The platform has been deployed through a collaboration between WeBank and Extreme Vision to help customers develop computer vision-based safety monitoring solutions in smart city applications. Over four months of usage, it has achieved significant efficiency improvement and cost reduction while removing the need to transmit sensitive data for three major corporate customers. To the best of our knowledge, this is the first real application of FL in computer vision-based tasks.

Publications:

1. Yang Liu, Anbu Huang, Yun Luo, He Huang, Youzhi Liu, Yuanyuan Chen, Lican Feng, Tianjian Chen, Han Yu & Qiang Yang, Federated learning-powered visual object detection for safety monitoring. AI Magazine, vol. 42, no. 2, AAAI Press (2021).
2. Yang Liu, Anbu Huang, Yun Luo, He Huang, Youzhi Liu, Yuanyuan Chen, Lican Feng, Tianjian Chen, Han Yu & Qiang Yang, "FedVision: An Online Visual Object Detection Platform Powered by Federated Learning," in Proceedings of the 32nd Annual Conference on Innovative Applications of Artificial Intelligence (IAAI-20), pp. 13172-13179, 2020. (Innovative Application of AI Award)