Attributed graph representation has attracted increasing attention recently due to its broad applications such as node classification and link prediction. Most existing methods adopt Graph Neural Network (GNN) or its variants to propagate the attributes over the structure network. However, the attribute information will be overshadowed by the structure perspective. To address this limitation and build a link between nodes feature and network structure, we aim to learn a holistic representation from two perspectives: topology perspective and feature perspective. To be specific, we separately construct the feature graph and topology graph. Inspired by the network homophily, we argue that there is deep correlation information between the network structure perspective and the node attributes perspective. Attempting to exploit the potential information between them, we extend our approaches by maximizing the consistency between structural perspective and attribute perspective. In addition, an information fusion module is presented to allow flexible information exchange and integration between the two perspectives. Experimental results on four benchmark datasets demonstrate the effectiveness of our proposed method on graph representation learning, compared with several representative baselines.