Affective multimedia content has long been used as stimulation to study an individual's personality using physiology. In this work, we propose a novel Siamese Content-Attentive Graph Convolutional Network (SCA-GCN) to learn a discriminative physiology representation jointly guided by the actual video content of the emotional stimuli. The visual content of the stimuli is integrated into learning to weight the importance of physiology in the task of personality recognition. We evaluate our framework on a large public corpus of physiological data. Our method achieves the state of the art unweighted accuracy of 72.1%,69.5%, and 68.2% in a binary classification for dimensions of Openness, Emotion Stability, and Extraversion, which improves over the baseline DNN by 20.4%, 9%, and 13.9%. Further analysis reveals that there indeed exists a substantial effect from the media content in affecting the subject's internal physiological responses that result in an improved personality recognition performances.