We introduce a simple semi-supervised ap-proach to improve implicit discourse relation identification. This approach harnesses large amounts of automatically extracted discourse connectives along with their arguments to con-struct new distributional word representations. Specifically, we represent words in the space of discourse connectives as a way to directly encode their rhetorical function. Experiments on the Penn Discourse Treebank demonstrate the effectiveness of these task-tailored repre-sentations in predicting implicit discourse re-lations. Our results indeed show that, despite their simplicity, these connective-based rep-resentations outperform various off-the-shelf word embeddings, and achieve state-of-the-art performance on this problem.