The methodology of knowledge graph embedding (KGE) tries to find appropriate representations for entities and relations and appropriate mathematical computations between the representations to approximate the symbolic and logical relationships between entities. One major challenge for knowledge graph embedding is that the relations in real- world knowledge bases exhibit very complex behaviors: they can be injective (1-1) or non-injective (1-N, N-1, or N-N), symmetry or skew-symmetry; one relation may be the inversion of another relation; one relation may be the composition of other two relations (where the composition can be either Abelian or non-Abelian). However, to our knowledge, there hasn’t been any theoretical guarantee that these complex behaviors can be modeled by existing KGE methods. In this paper, we propose a method called MQuadE to solve the problem. In MQuadE, we represent a fact triple $(h, r, t)$ (that is, (head entity, relation, tail entity)) in the knowledge graph with a matrix quadruple $(bm{H}, bm{R}, hat{bm{R}}, bm{T})$, where $bm{H}$ and $bm{T}$ are the representations of $h$ and $t$ respectively and $