A scalable architecture for real-time monitoring of large information systems

Data centers supporting cloud-based services are characterized by a huge number of hardware and software resources often cooperating in complex and unpredictable ways. Understanding the state of these systems for reasons of management and service level agreement requires scalable monitoring architectures that should gather and evaluate continuosly large flows in almost real-time periods. We propose a novel monitoring architecture that, by combining a hierarchical approach with decentralized monitors, addresses these challenges. In this context, fully centralized systems do not scale to the required number of flows, while pure peer-to-peer architectures cannot provide a global view of the system state. We evaluate the monitoring architecture for computational units of gathering and evaluation in real contexts that demonstrate the scalability potential of the proposed system.