Continuously identifying day-to-day mental stress can be realized by accessing wearable devices to measure physiological indicators. However, the nature of bodily signals raises issues of privacy and data heterogeneity. Recent federated learning scheme provides a promising direction to alleviate the privacy concern, but the large inter-client differences can lead to a sub-optimal model performance. In this work, we propose a client-aware aggregation strategy to customize the global model forked by each client to conduct mutual learning in federated setting. Our proposed mixture Federated Mutual Learning (mixFML) weighs the distances of local models to generate a unique mixture of global model per client. We evaluated our method on the public TILES-2018 and an inhouse Firefighters dataset for stress detection using HRV. Our proposed mixFML achieved 8.0% and 1.8% MCC improvement on two datasets compared to federated mutual learning.