The user profile is one of the main context-information in a wide range of pervasive computing applications. Modern handheld devices provided with localization capabilities could automatically create a diary of user’s whereabouts and use that information as a surrogate (or a complement) of the user profile. The places we go, in fact, reveal also something about us, for example, two persons can be matched as compatible given the fact they visit the same places. Web-retrieved information, and the temporal patterns with which different places are visited, can be used to automatically define meaningful semantic labels to the visited places. In our work we used geocoding and white-pages Web-services to extract information about a place, and Bayesian networks to classify places on the basis of the time in which they have been visited. In this paper we describe the general idea at the basis of the whereabouts diary, discuss our implementation, and present experimental results. Finally, several applications that can exploit the diary are illustrated.