Operator product expansion of higher rank Wilson loops from D-branes and matrix models