A calibration methodology for building dynamic models

nature is to simplify consumption outputs interpretation even in case of 33 complex buildings. A further innovative consequence of the methodology 34 proposed is its capability to promptly identify ineﬃciencies in the build-35 ing subsystems, i


Introduction
The software for dynamic building simulation represents a powerful tool for the design of new energy efficient buildings as well as the analysis of existing buildings in order to reduce their energy consumption [1].
In fact, most of the simulation software available on the market allows to predict the power and energy consumption outputs resulting in combination with the temperatures and other environmental parameters.In such a way, it is possible to guarantee a consumption reduction without compromising the comfort and, in some cases, even increasing it [2,3].The way this software is able to produce such detailed outputs is through sophisticate simulation engines that run on a meticulous description of the building such as EnergyPlus or TRNSYS [4].For the analysis of existing facilities, the major risk is the wrong definition of some parameters that will inevitably lead to wrong outputs.A further complication is the lack of verifiability that the designer has on some of these parameters.In fact, while the transmittance of walls or windows can be easily assessed by A c c e p t e d M a n u s c r i p t 3 the right tools, other user-dependent parameters cannot be measured so easily, e.g. the air change due to ventilation/leakage or the actual point imposed by the thermostats [5,6,7] .All this, together with contingent variations between the design data and the actual operative conditions of the building, suggests to validate the simulation tool by a calibration of the model [8]..The most commonly used way to proceed is to compare the real natural gas and electric bills with the simulated consumption [9].This approach, however, can be considered effective only for general comparison and validation.In fact, in case of an overall annual (or seasonal) value comparison, all the valuable dynamic data produced by the simulation are cut off.An opposite case is the ASHRAE guidelines, which suggest a comparison done on monthly or hourly basis [10].This comparison requires processing a large amount of data just to outline the values that are out of the maximum error (5 or 10% as suggested by [1]), but no further information is given on how to proceed in order to obtain a better calibration of the model.More complex anayses of model validity can be found in literature, i.e. the work from Manfren et al. is based on a meta-model approach used to outline advantages and the drawbacks of various calibration strategies [11].
The aim of this work is the definition of a calibration approach that takes the dynamic behavior of the building-HVAC systems interaction into account but does not require more data than usually available from A c c e p t e d M a n u s c r i p t 4 billing reports or similar summary information.The proposed method is based on the comparison between the energy signature of the real building with the energy signature of the simulated one.The energy signature is an effective tool able to describe the dynamic behavior of a building thanks to the correlation between the energy consumption and the outside temperature [12]: these data are collected monthly or sometimes weekly, so the signature keeps track of the dynamic phenomena that characterizes the building.In such a way it is possible to calibrate the model and also to outline the reason behind the differences of an incorrect calibration at the same time.Further to the simple calibration, the proposed approach allowed to learn about the building through an indirect process.In fact, values that were difficult to collect in an onfield survey are here identified trough the calibration process.[12,13].
After this it is possible to promptly outline malfunctioning as well as wrong uses of the HVAC systems in terms of ventilation, infiltration and temperature control, even in those cases where there is no time for long on-site measurements and monitoring or there is a lack on information from the survey. .The way to proceed chosen for this work, outlined in Figure 1, is as follows: first a case study building was chosen for the work.It is a retail store building dating back to the 1970s that consists of more than 3544 m 2 of floor surface.Then a field survey was performed to identify all  the parameters required to model the building by Design Builder software, together with the draft of its real energy signature [5].The next step generically consisted in the identification of the uncertain parameters and their respective ranges.After that, a parametric approach for multiple simulations was adopted, focusing on temperature set-point and ventilation air change.By means of the jEPlus, tool [14], the building was simulated under 176 different possible combinations of the uncertain parameters [14].Every simulation resulted in an energy signature that was then compared with the actual one obtained from the case study.
The minimization of the differences between the simulated and real signatures enables the definition of the parameter set that validates the model.
Further results were obtained through a more specific parametric analysis within the neighborhood of the optimum point found in the previous analysis, reported in Section 2.1.A more detailed description of the adopted methodology is reported in the following sections.and outside temperature, or even degree-days.[16,12].In the heating season analysis, the straight line that best fits the point cloud is the energy signature.The energy signature of the superstore investigated in this work is reported in Figure 2. .The average external temperature is recorded at regular intervals.These intervals can be as small as one hour while the most common time steps are the week or the month [15,17].
The analysis proposed for the investigated signature focuses on the win- ter period but the signature can also be extended to the summer period in order to evaluate the behavior of the air cooling systems as reported by Yu and Chan [18].The effectiveness of the energy signature analysis on commercial buildings was extensively proven by Rabl and Rialhe in 1992 when they screened more than 50 commercial buildings using this technique [19].In this work the data collected enables to define the energy signature on a monthly basis.

Use of energy signature and other calibration methods
A model calibration leads to many benefits, for example identifying and evaluating savings or finding input parameter errors [20].Liu et al.
show a methodology for the rapid calibration of energy consumption simulations for commercial buildings based on the use of "calibration signatures", which characterize the difference between measured and simulated performance [21].Thanks to simulation programs the characteristic cal-  method for calibrating simplified building energy simulation models of commonly used HVAC systems, using an office building as a case-study [22].Their model is calibrated using two weeks of measured heating and cooling data.In this way, the root mean squared error RMSE (used as parameter to evaluate the reliability of the model) is significantly reduced.Reddy et al. [23] studied a method for calibrating building energy models to monthly measured data.After completing an audit of the building, a "base-case" model of the building is created and then, by means of a parametric optimization analysis, a number of calibrated models for the building is determined.Raftery et al. [20] focus their study on the necessity to bring the principle of evidence-based decisionmaking to the calibration method.They argue that, in order to improve the effectiveness of calibration, it is extremely important to change the

Calculation
In the following paragraph the case study and the definition of the model are described.The energy signature together with specific indexes, based on the differences between the real and simulated data, are effective tools to verify the congruence of the modeling process.

Model costruction and calibration
The construction of the building model is realized trough the Design-Builder dynamic modeling software, a Graphical User Interface (GUI) of EnergyPlus [5].In order to validate the model, a comparison between A c c e p t e d M a n u s c r i p t

Model costruction and calibration 9
the real and simulated consumption data is performed.Particular attention is paid to the thermal energy consumed by the building.Natural gas consumption is easily evaluated going through the bills.Then, the energy used can be evaluated considering the efficiency of the natural gas boiler, here considered 0.84 as reported by the manufacturer.The bills are provided on a monthly basis, forcing the calibration process to use the same time step.This work focuses on the calibration process acting on those operative parameters that elude the design specs: ventilation/leakage and actual temperature set-point.Even if these values are specified during the design of the building, they are strongly userdependent and not completely predictable.The calibration method used is a parametric analysis of the uncertain or non-homogeneous values of the case study obtained through jEPlus, a powerful tool that runs several "batch of jobs" in parallel [14].The entire and generic procedure behind this work calculation is outlined in the list below: 1.In situ survey and data collection; 9. Identification of the simulation characterized by the lowest differences with the real building.This solution calibrates the model.

Case study
The case study is a retail store built in the 1970s, located in Bologna (Italy).The 3D building render is depicted in Figure 3.In accordance with Italian regulations as summarized in UNI/TS 11300-1, which divides Italy into six climate zones based on the degree-day value (e.g. Bologna is in the E climate zone [25]) and attribute to each of them a conventional heating period (from October 15 th to April 15 th in the case study).The total area occupied by the building is 3544 m 2 and most of it consists of the sales floor area.The building is composed by several areas, listed below: • Sales area = 3227 m 2 ; • Bar area = 163 m 2 ; • Bakery area = 39 m 2 ; A c c e p t e d M a n u s c r i p t • Fish shop area = 19 m 2 ;

Case study 11
• Entrance area = 34 m 2 ; • Low and normal temperature refrigerators and remaining areas = 62 The subdivision into areas concern the electrical consumption only, while all the activities take place within the same open space.For this reason the building can be modeled as a single thermal zone.
The building envelope is made of a structure in reinforced concrete, weakly isolated with 0.02 m of extruded polystyrene foam (XPS).The covering is a hollow-core concrete roof isolated with 0.1 m of expanded polystyrene (EPS).In the histograms in Fig. 4    vey and the real ones.Therefore also this parameter has been chosen for the calibration.
The calibration focused on the heating energy consumption and not on the electrical one; in fact the real and simulated electrical energy signatures showed a good matching since the first simulation, as reported in Figure 6.
In First of all the nameplate characteristics of the AHU were considered: A c c e p t e d M a n u s c r i p t Then the recircualtion rate was lowered to its 85% as consequence of the preliminary investigation done on this case-study [31]; therefore the return flow is reduced to 23800 m 3 /h.It follows that the total flow for each AHU is:

DesignBuilder modeling and jEPlus simulations 14
The mechanical ventilation system consists in 3 AHU, Dividing this flow by the volume within the building envelope, it is possible to obtain the first iterate value of leakage/ventilation (lv) in This value is in line with other studies in the literature, i.e.Zaatari et al. investigated the influence of air exchange rates from 0.5 to 2 on the concentration of some specific pollutants in the indoor air [32].
A c c e p t e d M a n u s c r i p t

DesignBuilder modeling and jEPlus simulations 15
The specific parameters are varied until the model is calibrated, and a significant range and a proper step of variation had to be defined for each of them.176 simulations were run with the support of jEPlus.
The following ranges of variation were considered: • Heating set-point temperature: from 18 o C to 23 o C, with a step of 0.5 o C; • Ventilation/leakage: from 1.4 to 2.9 V ol h , with a step of 0.1 V ol h .
After those 176 simulations, a thickening is done around the values of temperature and ventilation/leakage presenting square errors (defined below) in the neighborhood of 0.1.The thickening consists in 40 further simulations and it is aimed to verify that the temperature set-point and ventilation values that provide the absolute minima of root mean squared and safety aimed error indexes, which estimate the convergence between the real and the simulated case, were exactly the ones found during the first 176 simulation analysis: • Heating set-point temperature: from 20 o C to 20.9 o C, with a step of 0.1 o C; • Ventilation/leakage: from 1.5 to 1.8 V ol h , with a step of 0.1 V ol h .
The thickening is carried out for only the heating set-point temperature, while for the ventilation/leakage there is no need to fit the values because the step chosen before was already small enough.
A c c e p t e d M a n u s c r i p t

DesignBuilder modeling and jEPlus simulations 16
Once the simulations are carried out, the results are analyzed and discussed, comparing the real energy signatures with those obtained with the simulations.The results are analyzed with the help of two error indexes which estimate the correspondence between the real and simulated case.
• Root mean squared error (RMSE); • Safety aimed error (SAE): between m and q relative errors.
The slope m is that of the energy signature that corresponds to the dispersion through the building envelope for transmission and ventilation [12].A good slope matching indicates a similar monthly consumption trend between the simulated and the real building.The intercept q indicates the expected consumption for an external temperature of 0 o C and usually is a good indicator of the consumption behavior in winter months.
In order to estimate the actual matching between real and simulated m and q, the relative error of each one is considered. where: • E rel,m = slope (m) relative error; • m real = real energy signature slope; A c c e p t e d M a n u s c r i p t
The simulations with the closest resemblance to the real consumption produces a relative error lower than 10%.
The SAE error index is defined as follows: This error index is used to favor estimated energy signatures characterized by both slope and intercept values close to the real ones.An estimated energy signature with the same slope of the real signature but with a completely different intercept is not representative.In the same way, an estimated energy signature with an intercept matching the real one, but with a completely different slope is not representative.
The RMSE is defined as follows: where A c c e p t e d M a n u s c r i p t 18 • C real,i = real consumption of the i-th month; • C simulated,i = simulated consumption of the i-th month.
The results obtained show that, minimizing the presented error indexes, it is possible to achieve convergence with the best solution which minimizes the safety aimed error.

Preliminary considerations
The documentation provided by the staff does not contain all the required information related to the stratigraphy of the main walls of the building.Based on the final use of the case study building, its year of construction, information collected during the survey and using the abacus wall structures developed by CTI (Comitato Termotecnico Italiano [33]), the wall and roof stratigraphy is identified and reported in Table 1.Prior to carrying out the parametric analysis, some preliminary operations were conducted:

Main walls Roof
• Exporting the .idffile, that is the EnergyPlus input script file, from DesignBuilder (EnergyPlus 8.1 version); • Using IDFVersionUpdater, an EnergyPlus tool, it was possible to convert the .idffile into an EnergyPlus 8.2 version file (last release when the operations were done).
Then jEPlus requires three input files: • Weather data file (.epw format): the same used in DesignBuilder; • The .idf file; • The .rvi file: this file defines the required output variables.It can be generated by the EnergyPlus tool ReadvarsESO.

Results of the model without calibration (base case)
In this paragraph a comparison between the real thermal consumption and the base case simulated one is carried out as reported in Figure 7.
After the construction of real and simulated energy signatures, the quality of the model is evaluated through the proposed error indexes reported in Table 2.
From a graphical point of view, Figure 8 shows the comparison between the real (solid line) and simulated energy (dashed line) signatures  It is possible to observe that, on average, the real consumption is higher than the simulated one, although both are characterized by similar trends.

First batch of simulations results
After the first parametric analysis, 176 simulations were performed, in which the set-point temperature and the ventilation/leakage were varied.
In order to facilitate the comprehension of the influence of the chosen parameters on the calibration errors, Figure 9 was drafted plotting in a three dimensional graph all the results of the first batch of simulations.
The graph shows the influence of both set-point and leakage on the SAE.In Figure 9 it is possible to observe that the mutual variation of the two parameters correspond to an homogeneous trend of the error value.
A c c e p t e d M a n u s c r i p t

First batch of simulations results 21
The semi-transparent horizontal plan sections the surface dividing it into two areas.The area under the plane contains all the most representative simulations with a SAE less than 10%.In Figure 13 a comparison between real (solid line) and simulated (dashed line) energy signatures is shown.In Table 4 the two minimized error indexes calculated for this simulation are reported.In particular the SAE index is reduced the most during the calibration process (from 0.064 to 0.011).The RMSE was also reduced during the calibration but the reduction was less pronounced.In Figure 17, the final comparison between real (solid line) and simulated (dashed line) energy signatures is shown.

Figure 1 :
Figure 1: Generic parameter definition and calibration process flow chart

Figure 2 :
Figure 2: Real energy signature of the case study

A c c e p t e d M a n u s c r i p t 8 input
parameters only according to available evidence under defined priority.Other studies focus on the importance of the building audit in order to determine appropriate values for the observable parameters of a building simulation model.Surveys, field measurements and interviews with building managers are the first part of an accurate calibration since the real building operation is often different from the specifications assumed and documented during the design and construction phases.Heo et al.[24] improved a calibration method based on a statistical formula which takes into account three levels of uncertainties: parameter uncertainty in the energy simulation model, discrepancy between the model and the true behavior of the building, and observation errors.Their case study demonstrates that this methodology can correctly evaluate energy retrofit options.

2 .
Design of the base-case model with DesignBuilder; 3. Draft of the real-case energy signature of the case study; 4. Production of the energy signature of the simulated-case from model outputs and real weather data; 5. Comparison between energy signatures and evaluation of the error indexes defined in Section 2.3; A c c e p t e d M a n u s c r i p t 2.2 Case study 10 6. Choice of uncertain parameters X i (i=1,...,n), if the base-case model is not representative of the real consumption; 7. Definition of the variation ranges and the incremental step width for each of the X i parameters; 8. Parametric analysis, using jEPlus, of the simulations which characterize each possible combination of the above parameters;

Figure 3 :Figure 4 :
Figure 3: Case study and Fig. 5 the thermal and electric consumption of the superstore measured in 2013 are represented.In order to create both the real and the simulated energy signatures, weather data is required.It is obtained considering the hourly variations of the outside temperature over the entire period of simulation; the hourly external temperature was obtained from the database of the IdroMeteoClima Arpa Service, Dexter System [26].

A c c e p t e d M a n u s c r i p t 2 . 3
DesignBuilder modeling and jEPlus simulations 12

Figure 5 :A c c e p t e d M a n u s c r i p t 2 . 3
Figure 5: Superstore electric consumption in 2013

Figure 6 : 3 h•
Figure 6: Pumps, thermoventilation and lighting real and simulated energy signature

A c c e p t e d M a n u s c r i p t 3 . 2
Results of the model without calibration (base case) 19

A c c e p t e d M a n u s c r i p t 3 . 3 Table 2 :
First batch of simulations results 20 Error indexes of base case model for the uncalibrated model.

Figure 8 :
Figure 8: Real (solid line) and base case (dashed line) energy signature

Figure 10 was
Figure 10 was obtained as a rotation of Figure 9 and it shows that there is only one absolute minimum for the SAE function.The presence of a singular point of absolute minimum is welcome because it could represent the parameters set that most accurately calibrate the model.For the first step of calibration this set was found to be: temperature set-point of 20.5 o C and a ventilation/leakage of 1.6 Vol/h.From an opposite point of view, each other combination of the two parameters present in the plot represents a "possible superstore" where different temperature and leakage were set.Similarly to what was discussed about Figure 9 and 10, Figure 11 shows the 3D graph in which the two varied parameters are related to the RMSE.

Figure 11 :
Figure 11: First batch of simulations results -RMSE

A c c e p t e d M a n u s c r i p t 3 . 3
First batch of simulations results 22

Figure 13 :
Figure 13: Best solution of 176 simulations energy signature

A c c e p t e d M a n u s c r i p t 3 . 4
Second batch of jobs simulations 233.4.Second batch of jobs simulationsAfter the previous 176 simulations, a thickening has been performed in the neighborhood of the values of temperature that most accurately calibrated the model in the first batch of jobs.Forty simulations were held, as depicted in section 2.2 varying the set-point temperature in the range of 20-21 o C. Figure14shows the graph which relates the two varied parameters to the SAE.

Figure 14 :
Figure 14: Second batch of simulations results -SAE

Figure 15 reportsA c c e p t e d M a n u s c r i p t 3 . 5
Figure 15 reports the graph in which the two varied parameters are related to the other error index, the RMSE.There is an absolute mini-

Figure 17 :
Figure 17: Best solutions of 40 simulations energy signature

A c c e p t e d M a n u s c r i p t 3 . 5
Best solution and possible uses of the model 25rors simultaneously, which correspond to the combination of a set-point temperature of 20.6 o C and a Ventilation/leakage of 1.6 vol/h.This combination is the best solution that enables significant reduction of the error indexes and it can be considered the combination of parameters that leads to the calibration of the model.The calibrated model has an energy signature almost coincident with that of the real case, with an error less than 1%.This value is significantly lower than the value calculated for the first batch of jobs simulations.Once the model is calibrated it can be used for two main purposes:• The standard use of a calibrated model consists in testing possible improvements on it before moving forward with their real implementation • A different consequence of the proposed methodology is the indirect evaluation of some values that were impossible to define during the survey.In the case study investigated in this work, the calibration allowed to define the real air change and temperature set-point values.A comparison between these values and the design or optimal ones helps to immediate define primary actions to control energy consumption reductions based on ventilation, infiltration and temperature control.

•
Parametric analysis is used together with the energy signature for calibration of the model • Statistic indexes are used to determine the best calibrating solution • The calibrated solution allows to define better use of the HVAC system

Table 1 :
Main walls and roof stratigraphy Table 3 shows the comparison between the

Table 3 :
Absolute minimum solution errors

Table 4 :
Absolute minimum solution errors (Second batch of jobs)