A conjugacy limit group is the limit of a sequence of conjugates of the positive diagonal Cartan subgroup, C \leq SL(n) in the Chabauty topology. Over R, the group C is naturally associated to a projective n-1 simplex. We can compute the conjugacy limits of C by collapsing the n-1 simplex in different ways. In low dimensions, we enumerate all possible ways of doing this. In higher dimensions we show there are infinitely many non-conjugate limits of C. In the Q_p case, SL(n,Q_p) has an associated p+1 regular affine building. (We'll give a gentle introduction to buildings in the talk). The group C stabilizes an apartment in this building, and limits are contained in the parabolic subgroups stabilizing the facets in the spherical building at infinity. There is a strong interplay between the conjugacy limit groups and the geometry of the building, which we exploit to extend some of the results above. The Q_p part is joint work with Corina Ciobotaru and Alain Valette.