The affective state of children with autism is not always expressed or discernible through observational cues, a phenomenon which is further confounded by vast variability across individuals on the autism spectrum. Electrodermal Activity (EDA) is a physiological signal indicative of a person's arousal and thus affording us new insights into a child's inner affective state. In this work we study EDA cues of children with autism while interacting with an Embodied Conversational Agent (ECA). EDA is affected by both cognitive and social factors. In this paper, we consider the child's verbal response latency as the overt behavioral cue and link it with his/her physiology. A classification experiment was performed to differentiate between physiological cues of high and low verbal response latency intervals, based on the assumption that different kinds of mechanisms are triggered in each case. Our results indicate that physiological patterns between short and long verbal response latencies are more discriminative for some children than others, suggesting the existence of multiple levels of cognitive and social efforts across children. They also show variable levels of arousal response, which can provide a complementary view of the observational cues.