Crying is the only communication way from infants due to immature larynx and pharynx. It usually represents the urgent demand from infants. Caregivers and parents need to solve the urgent demand as soon as possible. Unfortunately, they are suffered from not knowing why infants cried and try many methods to comfort infants, in a result of that, there are a lot of material to teach parents how to understand infants' cry sound. However, it didn't work in the baby care center. The law in Taiwan allows caregiver taking care at most 5 infants in a time. If one of infants cried and caregiver could not solve immediately, chain problem will happen such as all infants crying simultaneously. Therefore, cry reason classification are important for the baby care center. Furthermore, event in the center may induce infants cry due to sensitivity to environment of infants. In this paper, we proposed a deep network that learns context information from the cry sound and leverages the event and environment factor to increase the performances of reason classification.