The empirical analysis of our public performance characterization framework proceeds in several stages. We first present the methodology of the study, then the results of our analysis of the elements of the characterization framework.
2.1 - Methodology
The approach adopted in our empirical study is that of hypothetico-deductive research. Indeed, on the basis of the public of the six propositions on the multiple characteristics of PP, derived from proposition P1 of this study. In order to test the empirical relevance of our working model, we will now use a quantitative methodology based on a survey conducted via a questionnaire.
Performance characteristics singled out, we put forward six hypotheses on the multiple character of each of the six components identified. To test these hypotheses, we sent a questionnaire to 350 French local government authorities, members of AFIGESE-CT (Association FInances-Gestion-Evaluation des Collectivités Territoriales), and received 147 useable replies of the highest-ranking officials.
Table 2 - Presentation of the respondents
Number of response 147 Repartition of positions financial managers, 30% management controllers, 29% department heads, 29% general managers, 12% Repartition of population of the local government authorities (mean: 299,000) *<150,000 : 75 local authorities 150<*<300,000 : 26 300<*<500,000 :15 *>500,000 : 32
The questionnaire was made up of an informational part and parts concerning the characteristics of PP (fig.1).Each characteristic includes variables from literature that were evaluated using a seven-point Likert scale on the degree of agreement (1: no agreement at all; 7: complete agreement).
We processed the data in two stages. First, using the SPSS software we checked the internal consistency of a scale. There are several ways to assess the relevance of the items used in a scale (Correlation matrix of inter-item, Split-half method…), we used the most widespread method, the calculation of the Cronbach’s Alpha  Cronbach's alpha is used to determine if multiple items... index. We calculate the index for the six characteristics of public performance.
We then identified the most significant explanatory variables using regressions to check the presence of different dimensions derived from the literature review. The dependent variables (eg, multi-objects) are constructed variables from a score function which makes them binary and allows some logistic regressions. The Logistic regression is a multivariate model that is used when the dependent variable (Y) is qualitative, usually binary. The explanatory variables (independent variables, X) can be against by either qualitative or quantitative. The dependent variable is usually the occurrence or not of an event and the independent variables that may influence the occurrence of this event. The advantage of this technique is to quantify the strength of the association between each independent variable and the dependent variable, taking into account the effect of other variables in the model. The interpretation is easy because the coefficients estimated by the model are related mathematically with the odds ratio, which represents the strength of the association between two factors.
The explanatory variables (eg, performance of public policies) are variables from multiple items of the questionnaire. The quality of these regressions was evaluated using the Nagelkerke coefficient and the percentage of correct rankings of the individuals studied. The regression coefficients were evaluated according to their degree of significance  For example: Table 5 shows the 16 variables to ascertain....
In sum, each of the six characteristics of public performance identified (objects, objectives, measures, rationales, targets and learning) was presented and tested through their modalities derived from the analysis of the literature (cf. Part 1).
2.2 - Analysis of the objects of public performance
To analyze the objects of PP, four variables were put forward to respondents: performance related to individuals, to the higher administration/departments, to the local authority in general, and to the initiatives undertaken (the public policies implemented).
The variable "multi-objective" is a qualitative variable which results of a binary score calculated on the four explanatory variables we identified (score exceeding or not the average of the expected answers). The explanatory variables (eg, Performance of individuals, Performance of higher administration …) are score functions stemed from the items of questionnaire (for example, "Performance of individuals" is the result of four answers).
Regarding the component “object of evaluation”, we observe that Cronbach’s Alpha is 0.57 and that the four elements from the literature review are significant. The respective weight of the four explanatory variables is seen in the results of the logistic regression model.
Table 4 - Study of the model and the regression coefficients of the binary variable “PP objects”
Variables Beta coefficients Standard deviation Wald Signif. Exp(B) (constant) -1.277 3.086 9.668 .002 .001 Performance of individuals 0.156 .010 2.282 .140 2,344E-02 Performance of higher admin. 1.286 .451 8.149 .004 3.634 Performance of local authority 1.521 .482 9.961 .002 4.939 Performance of public policies and of initiatives undertaken 2.364 1.758 1.987 .766 .278
Model significant at 1%; R2Nagelkerke : 0.73; correct ranking: 95.1%
The regression model shows 73% of the variance explained; it is significant at the threshold of 1%. This significance shows the existence of at least one explanatory variable exercising a significant influence on the dependant variable.
This regression model shows that two variables have a statistically significant influence on the characteristic “object” of PP, namely local authority performance and higher administration performance. With these two variables, the regression model can correctly predict 95% of the rankings. Note that the variable “individual performance” is statistically significant at the threshold of 10%, but its high correlation with variable 3 reduces the effectiveness of the model when it is taken into account. The correlation and the regression analysis performed on the objects of PP (tables 2 and 3) both validate the multidimensional character of the objects (H1 validated).
2.3 - Analysis of the objectives of public performance
To assess the objectives pursued in PP approaches, sixteen variables, derived from the literature, are put forward.
Table 5 - Scale “objectives pursued”
Multi-objectives function (? : 0.71) appropriately allocate spending and resources to organizational objectives economically allocate resources to policies undertaken ensure compliance of initiatives undertaken with regulations optimize revenue recruit, motivate and deploy the personnel respond to the personnel’s aspirations to well-being control the volume of local public services on offer control the quality of local public services meet users’ needs at the right levels of satisfaction meet citizens’ needs at the right levels of satisfaction meet local actors’ needs at the right levels of satisfaction Communicate with the stakeholders learn from one’s practices in order to improve the allocation of resources to organizational objectives learn from one’s practices in order to reorient the objectives of the organization adapt the structure of the organization to the strategy of the local authority develop common standards, rules, and values for projects and results
Studying the relative importance of the explanatory variables, we see that the regression coefficients of the model reveal three significant factors (table 6). The three significant variables of the regression model explain 68.5% of the variation of the dependant variable, ranking almost 96% of individuals correctly. These variables, which explain the objectives of PP (meet citizens’ needs, learn from one’s current and past practices in order to improve the allocation of resources to organizational objectives, and recruit and motivate the personnel), concern the internal and external environment of the local authority. These analyses thus enable one to capture a multidimensional conception of PP in its territorial, public service, organizational, human and financial components (Hood, 1995; Bouckaert and Pollitt, 2004; Moullin, 2002).
Table 6 - Study of the model and the regression coefficients of the binary variables objectives pursued
Variables Beta coefficients Standard deviation Wald Signif. Exp(B) (constant) -18.277 9.086 5.668 .021 .001 recruit, motivate and deploy the personnel 1.638 1.158 2.001 0.098 5.144 meet citizens’ needs at the right levels of satisfaction 2.767 1.239 4.289 0.026 15.912 adapt the structure of the organization to the strategy of the local authority 3.958 2.145 1.584 0.109 3.425 learn from one’s practices in order to improve the allocation of resources to organizational objectives 2.152 0.929 5,448 0.019 8.602
R2 Nagelkerke: 0.685; ranking correct: 95.8%
The regression model uses four main variables to explain the objectives of public performance (table 6). We are concerned with the measurement of the characteristics revealed by our theoretical study of the objectives pursued in PPM. These characteristics (meeting citizens’ needs; learning from one’s current and past practices in order to better align the allocation of resources with the objectives of the organization; adapting structure to strategy; motivating the personnel) concern the internal and external environments of the local authority. These analyses enable one to achieve a multidimensional conception of public performance, integrating its territorial, public service, organizational, human and financial components (Hood, 1995; Bouckaert & Pollitt, 2004; Moullin, 2002).
Logistic regression for the dependant variable "multi-objective": Here is a presentation of the step by step method to expose only the significant variables.
2.5 - Analysis of the learning characteristics of public performance
To assess public performance learning, we put forward nine variables that validate the two modes of learning distinguished theoretically.
Table 9 - Scale the learning factors
Multi learning function (? : 0.63) learn from one’s current and passed practices in order to improve the allocation of resources to organizational objectives learn from one’s current and passed practices in order to reorient the objectives of the organization develop common standards, rules, and values for projects and results adapt the human, financial and technical resources to the objectives of the local authority adapt the administrative and political structure to the strategy of the local authority motivate the personnel (reward according to individual and collective performance) take a more proactive role or reorient public policies to be implemented influence the allocation of resources between national and local government authorities Improve the quality of information communicated to local actors (users, citizens, taxpayers, banks, companies, etc.)
For the nine variables of the ‘learning’ dimension of PP, Cronbach’s alpha (0.63) is relatively weak. For the variables at work in the ‘learning’ characteristic, the logistic regression model reveals three significant variables out of the nine.
Table 10 - Results of the regression of the binary variables PP learning variables
Variables Beta coeff. Standard deviation Wald Signif. Exp(B) (constant) -7.297E01 2.686 7.968 .005 .001 adapt the human, financial and technical resources to the objectives of the local authority 2.156 0.621 2.85 0.09 1.869 motivate the personnel (reward according to individual and collective performance) 1.433 .426 10.248 0.010 3.073 improve the quality of information communicated to local actors (users, citizens, taxpayers, banks, companies, etc.) 1.193 .432 .371 0.001 3.798
Model significant at 1%; R2 Nagelkerke: 0.728; correct ranking: 96.4%
The regression model, with three significant variables, explains 73% of the variation and ranks 96% of the individuals. The main types of learning pursued bear on the internal and external finalities of public performance, namely motivating the personnel (the human dimension) and improving the quality of information communicated to partners (the territorial dimension). In sum, thanks to the regression conducted on the PP learning variables, we can confirm the multidimensional character of the learning variables and validate H4.
We now turn to examine the targets concerned by the information provided by the system of public performance.
2.6 - Analysis of the targets of public performance
To assess the public performance targets, nine variables were put forward to respondents.
Table 11 - Scale the “target” variables
Multi target function (? : 0.79) the citizens the taxpayers the users the bankers the national government the local government authority the agents the established companies the prospective companies
For the nine variables of the ‘target’ dimension of PP, Cronbach’s alpha (0.79) and the significance of the items are good, in line with the conclusions of the literature review.
To complement this exploratory approach, we sought to discover the significant variables in the regression model. Five factors emerged:
Table 12 - Study of the model and the regression coefficients of the binary variable “PP targets”
Variables Beta coefficients Standard deviation Wald Signif. Exp(B) (constant) -21.877 8.986 10.168 .001 .000 the taxpayers 0.763 .435 3.088 0.079 2.147 the users 1.858 .757 6.200 0.013 6.578 the local government authority 2.430 .810 9.005 0.003 11.310 the agents 1.472 .616 5.906 0.015 4.351 the prospective companies 3.529 1.170 9.058 0.003 34.094
Model significant at 1%; R2 Nagelkerke: 0.891; correct ranking: 97%
The significant variables in this regression model concern actors as much internal as external to the local authority. Taxpayers, prospective companies and users represent targets external to the local authority, while the higher administration and the agents illustrate the communication of information to internal actors. In finding the multiple character of the dimension “target” in the explanation of PP, we validate H5.
Finally, we conclude our study of the characteristics of public performance with an analysis of its underlying rationales.
2.7 - Analysis of the rationales of public performance
To assess the rationales of PP, eight variables were put forward to respondents.
Table 13 - Scale the “rationales pursued”
Multi-rationale function (? : 0.80) Regulation of economic activity General interest Social justice Compliance/legality Return on investment Productivity Effectiveness Efficiency
The Cronbach’s alpha (0.8) and the significance of the items are statistically acceptable, in line with the conclusions of the literature review.
Complementary to this analysis, the regression model identifies the significant explanatory variables of the dependant variable “multi-rationale” (table 14). With three explanatory factors used in the model, the equation enables the correct ranking of 97% of the individuals for the explained variable “underlying rationales of PP”. The presence of variables derived from both the private sector (efficiency, productivity) and the public sector (legality and compliance) is explained by the multiple constraints (financial, organizational and territorial) to which local authorities are subject.
Table 14 - Study of the model and regression coefficients of the rationales of PP
Variables Beta coefficients Standard deviation Wald Signif. Exp(B) (constant) -2.877 1.986 1.468 .035 .000 Compliance/legality 2.819 1.320 4.565 .033 16.705 Productivity 1.720 .911 3.610 .058 5.593 Efficiency 3.811 1.905 4.061 .041 45.070
Model significant at 1%; R2 Nagelkerke: 0.871; correct ranking: 97%
These statistical analyses enable the validation, in the local context, of our hypothesis concerning the multiple and contingent character of public performance and they reveal the dimensions of public performance – territorial, public service, organizational, human, financial.
We have confirmed, via an empirical study conducted in the local public sector, the multidimensional character of objects, objectives, measures, learning, rationales and targets of public performance. Indeed, our contribution to the literature emphasizes the necessity for public organizations to pursue multiple objectives in terms of resources and public service realizations and their environmental effects. Thus, we can make some recommendations.
About the dimension in relation with the means mobilized, the public organization must master its deployment of:
human resources: optimize the effectiveness of its services (CAF and PSS models);
financial resources: ensure compliance and economy in its spending, respecting its sources of finance, namely the State, taxpayers and banks (Hood’s model);
organizational resources (foster cognitive, cultural and structural learning (Bouckaert and Pollitt’s model; CAF).
At the level of “public service dimension”, the organization must produce a public service offer of sufficient quantity and quality, satisfying the user (Hood’s model; the PSS model; the CAF). Finally, at the level of “territorial dimension”, the public organization must, by its public service offer, make a durable impact on its territory by meeting the socio-economic needs of citizens and companies and communicating openly and reliably about initiatives undertaken and resources deployed (Bouckaert and Pollitt’s model).
2.8 - Summary of the results and implications
Our study empirically validates the multidimensionality of the concept of public performance, usually affirmed in the literature without being demonstrated, and this in itself constitutes one of the contributions of our research.
Moreover, via a synoptic vision, we can define public performance as the capacity of an organization to control its human, financial and organizational resources, in order to produce an appropriate public service offer (in terms of quality and quantity) that meets the needs of its stakeholders and generates positive effects on its territory.
Although this is confirmed, it would seem that the modalities constituent of each of the dimensions vary by country, and even by public organization. Thus the French local authorities show a largely internal and administrative conception of public performance, favoring indicators of activity and input and focusing mainly on the dynamics of efficiency. The rationalization of budgetary decisions, the monitoring of activities, and a communication oriented towards internal public actors are all elements characteristic of the French model (Carassus and Gardey, 2009). Conversely, a number of studies (Folzand al., 2009; Fryer and al., 2009; Talbot, 1999) have shown that the UK and the USA have a much broader and externally-oriented conception of public performance, as much in its measurement and use as in its communication. Here, the accent is placed on the evaluation of the social impact of public policies and programmes (Ammons and Rivenbark, 2008), on a communication oriented towards users and the local population, and on a sustained involvement of external stakeholders. These national differences in performance management, already observed by Pollit (2005), could be explained by the countries’ or public organizations’ stage of development. The initial stages (the French case) are characterized by a limited approach to public performance, restricted to problems of efficiency and monitoring of activities and favoring internal learning. This geographic or cultural variability of the notion of public performance, already observed by Siegel and Summermatter (2009), promises to be a fruitful area for future research.
Our empirical findings concur on another point with those of Siegel and Summermatter (2009), namely the multidimensional nature of the concept of public performance, characterized by complexity and ambiguity. This reality complicates the definition, measurement and use of the concept of public performance, as much by public practitioners and managers as by academic researchers.
Beyond these scientific interests, and with respect to the previous literature, our findings also have managerial implications. Indeed, our definition and characterization of public performance, validated in the local context, allows, according to a normative vision, to propose a conceptual implementation of appropriate practices framework. For example, scorecards and more generally public policies assessment processes, need to more frequently adopt a multidimensional approach, with regards to measures, targets or goals. More specifically, the "strategic map", proposed by Gibert (2002) and adapted from the balanced scorecard (Kaplan and Norton), provides a balanced approach to public performance that can be completed through the integration of the five dimensions highlighted in our study. In addition, a second instrumental implication concerns the content of the assessments, which should not only aim to measure the control of public resources and spending in an endogenous way, but also to understand the quality of public service and its responsiveness to the needs of the territory. The decision learning process could be enriched, on the one hand, through the production of information that goes beyond the simple logic of controlling annual budget expenditure and the second hand, by focusing on the management of organizational value creation through the control of local action (Lorino, 1997).