It is important to make automatic speech recognition (ASR) be inclusive to all users including disordered population. Besides the model performances, privacy  concerns, such as leakage of medical condition, are severe and detrimental for these already vulnerable population. Hence, developing privacy-preserving machine learning (PPML) algorithms are becoming important. Recent approaches of node cancellation strategies while repeatedly show their privacy protection efficacy, they involve complex multi-branched structures with manually-tuned thresholds. In this work, we focus on the task of learning ASR for dementia patients without revealing their medical condition. Specifically, we present a dementia attribute cancellation strategy (DACS) that involves training a single toggling network in an end-to-end manner that toggle off particular node dimensions at ASR decoding to conceal a subject’s dementia status. Our study shows that using DACS can achieve 33% dementia protection efficacy. By further configuring to higher demands of protection efficacy, we can achieve 45% of dementia protection efficacy with only a slight decrease of 0.004 WER in ASR performances.