On the automorphism groups of coloured graphs arising from manifolds