A worldline approach to colored particles