Applying Multi-armed Bandit Strategies to Change of Collaboration Patterns at Runtime

Autonomic systems have some interesting properties that enable them to self-manage at runtime. Traditionally, they are Self-configuration, Self-healing, Self-optimization, and Self-protection. Another interesting property is Self-expression, i.e. the capability of reconfiguring the collaboration pattern of a system at runtime and in an autonomous way. This re-factoring of behaviours, roles and interactions takes place whenever the environment in which the set of components is inserted experiences some changes, so that in order to adapt to the ever variable external conditions, the system is still able to maximize its utilities. In the past we have motivated the need for such a pattern change, and have proposed some approaches to enact it in situations in which we can associate to each external condition the fittest coordination pattern. In this paper we propose a strategy to decide the change and to choose the next collaboration pattern in situations in which the system components do not have sufficient knowledge and cognition to reason about the varying external conditions. Our approach is based on the algorithms proposed to solve the multi-armed bandit problem, in which a player must choose which slot machine to pull, given a number of them, in order to maximize the reward.