Úvodní stránka » Archiv čísel » Články 02_2014 » Assessment Wielkopolska

Assessment of the System of Project Selection under the Cohesion Policy. The Case of the Wielkopolska Region.

Piotr Idczak, Ida Musiałkowska

Abstract

The current year 2014 allows/has allowed for the first attempts to evaluate the operations, organization and implementation of the Cohesion Policy 2007-2013. The authors have analysed one of the elements that might have an impact on the results obtained under the regional operational programme in the Wielkopolska region in Poland, namely the modes of the selection of projects. This element of the whole cycle of projects is an under-researched subject of scientific analyses even though the project selection itself has been deeply examined by (various) control institutions (the European Court of Auditors, Supreme Audit Offices etc.). The authors use evaluation concepts and a strategic management lens, and formulate five research questions on: the conditions necessary for the transparency of project selection, the main factors determining the choice of the adopted selection mode, the impact of the selection mode on the results, the quality of the work of experts, and the efficiency of the appeal procedures for beneficiaries – trying to answer all these questions in the subsequent sections of the article. The methods used comprise analyses of legislation, reports, primary data and in-depth-interviews with the stakeholders. The research has also practical implications and can be a valuable case-study for policy-makers.

JEL codes: R5: Regional Government Analysis: R58 Regional Development Planning and Policy, R59 Other; O220 Project Analysis; O5 Economy wide Country Studies: O50 General

Keywords

Cohesion policy, EU funds, evaluation, selection of projects, Poland, Wielkopolska region, Wielkopolska Regional Operational Programme 1.

1. Introduction

Evaluation is one of the important elements of the cycle of public policy (Milio 2011, European Commission 2004), and lesson drawing (Molle 2007) that should aim at improving the quality of public interventions (OECD 1986, OECD 2000, OECD, 1991). It is embedded in the strategic approach towards managing and governing the public sphere (among others: Bachtler and Yuill 2001, Bienias et al. 2012, Dąbrowski 2012, Idczak et al. 2012). It refers to different stages of the implementing of a particular policy and therefore it may be conducted before the final versions of strategic documents are adopted by the institutions (ex-ante), in the turn of the policy cycle (mid-term) or at the very end – while results are assessed (ex-post). It provides an analysis of whether public policies are efficient, effective, relevant and sustainable and of what kind of impact they brought (OECD 1986, OECD 2000, OECD 1991, Bienias and Sudak MRD 2008).

In Poland, the evolution of evaluation in public intervention has been an on-going process. It started in the first period of 2004-2006 within the cohesion policy on the bases of the National Development Plan and the EU. (Bienias et al. 2012, p.-9-10) and continued over the ensuing years, bringing numerous evaluation reports on different aspects of the implementation of public policies, including the cohesion policy. The current authors have identified hardly any reports that dealt with a very important element of the project cycle management, namely, the selection of projects under the cohesion policy in Poland (with exemptions being: CRSG 2013, the reports on the administrative potential and the assessment of the systems of monitoring and control of the Ministry of Infrastructure and Development, the former Ministry of Regional Development, including fragmentary analyses of the issue and Poździk 2013 who concentrates mainly on procedural aspects of selection of projects). A good selection process hampers the worsening of the quality of projects (and acts against Gresham and Copernicus’ law).

Selection of the best project (projects), from a relatively large number of projects, is always difficult and requires the use of objective selection modes. Picking a selection mode, on the other hand, is highly dependent on certain attributes – such as the objectives of the operational programme, the intervention’s nature and scope, problems diagnosed, amount of funding etc. The project selection modes in Poland are defined in Article 28 of the Act on the principles of development policy, and they include: the individual mode, systemic mode, and calls for proposals (including open/permanent and closed/temporary calls). In fact, the Act defines it in a very general and vague manner. Thus, so loose a definition allows all stakeholders to interpret it differently.

Managing authorities at both national and regional levels are responsible for choosing the best projects to be destined for support from European funds that can contribute as much as possible to achieving the objectives of operational programmes (OP). In Poland in 2007–2013 all operational programmes were divided into priority axes, measures and even sub-measures in certain cases. Selection of the projects was conducted at the level of measures/sub-measures with specific assessment criteria and conditions set up for each measure. The practice shows that the institutions participating in the implementation of operational programmes have used existing statutory solutions in a very flexible and creative way. The same projects are in some cases classified as systemic projects, while in other cases they are understood differently. Moreover, the selection criteria related to each of the modes are often imprecisely defined as well (our own analysis of the documentation of boards of Polish regions 2014, CRSG 2013). In consequence, most projects are able to meet the established criteria and this, in turn, makes the selection of projects even more difficult. In addition, the preliminary research indicates that the selection system based on calls for proposals generates large costs for the whole system of implementation because of excessive bureaucracy (MRD 2013, MID 2014). On the other hand, the same mode casts doubts on the quality of the experts’ work in the assessment[1] of projects. This is due to the large number of applications to be evaluated by one expert or the limited possibilities to compare other applications. It is interesting to note that the system for project selection provides for an ability to protest against the assessment of those projects which are assessed positively but which have not received funding due to a large number of applications.

At the beginning of the current multiannual financial framework 2014–2020, it is crucial to analyse how the system for project selection has been functioning in 2007–2013 and whether it was able to contribute effectively to achieving the objectives of the EU cohesion policy at the operational level. In order to provide an answer to these questions, the authors carried out an exploratory assessment of project selection. The research based on evaluation techniques examines the modes that are applied to make the selection of projects preferable for the European Union’s support. Drawing on the empirical evidence from the Wielkopolska region and taking into consideration other studies (OECD 1991, OECD 1986, OECD 2000), the paper investigates to what extent were the objectives likely to be achieved due to the fact of the mode of the selection of projects, and what were the major factors influencing the achievement/non-achievement of the objectives due to the fact of the particular mode. In other words, the paper is set to investigate in-depth the nature of the system for project selection in Poland and its effectiveness.

The paper is organised as follows. The next section presents the literature review on the significance of the projects selection and the role of the selection criteria and modes used in the implementation of the operational programmes. This is followed by an outline of the research procedure and methods applied in the study. Sections 4 and 5 illustrate the selection modes applied in the Wielkopolska region and discuss the main factors determining the selection of the particular selection mode. They also explain whether the system for project selection enables the selection of the best possible projects and ensures effective implementation of the objectives of the operational programme. Section 6 presents the manner of experts’ working on the projects assessment and clarifies whether their work allows for the proper and effective assessment of the projects. Section 7 explains how effective are the appeal (protest) procedures from the applicant and the institutional points of view. The paper concludes with the findings that are relevant for the literature and policy practice.

2. Selection of projects

Project approach is at present commonly seen as a comprehensive way to achieving objectives of the EU cohesion policy, i.e. in general to supporting the overall harmonious development of its regions. It is necessary to point out that cohesion policy in its basic principles does not fund single individual projects defined as a specific unit of activities aimed at bringing about intended purposes. Instead, it financially supports multi-annual national or regional operational programmes setting out all the measures aligned in EU objectives and priorities. But, in turn, the implementation of those programmes on the ground is undertaken through individual projects, which are applied for and implemented by a wide range of key stakeholders or stakeholder groups – beneficiaries[2]. This means that the organization of the implementation of the cohesion policy in one of its last steps consists of implementing hundreds of thousands of projects. The managing authorities take responsibility in full for selecting, monitoring and evaluating these projects, so as to ensure the achievement of objectives provided for in the operational programmes, and consequently, to contribute effectively to the overall objectives of the cohesion policy. As a result, the individual projects remain, according to the European Commission, “an appropriate aid delivery modality” aimed at directly supporting initiatives being implemented in eligible regions (European Commission 2004). The project is the most practical and effective option for delivering financial assistance and for promoting specific ways of working based on cooperation, programming, management, empowerment, evaluation etc.

Bearing in mind the above-mentioned information, it follows that the key to success of the cohesion policy lies in the selection of those projects which will provide the biggest positive impact on the territory and contribute highly to the objectives of the EU cohesion policy. The selection of projects for funding is one of the most relevant processes to ensure the effective and right implementation of the operational programme. Put simply, the selection of the project is the process of evaluating individual projects or groups of projects and then choosing to implement some of them so that the established objectives are achieved (Meredith and Mantel 2009). In order to take a proper decision regarding potential projects, it is important to have a clear definition of the criteria and modes to be used to provide support for the decision makers. Assessment criteria are not peculiar to any particular modes of project selection. Many modes have the flexibility to use different sets of criteria. It is also useful to remember here that there is a need to consider the same set of criteria to avoid unfair competition between different projects. Therefore, a common measurement system needs to be established to perform a direct comparison (Dutra, Ribeiro and de Carvalho 2014).

The first step for selecting and prioritizing the projects to be covered in the process of competition is to decide on the criteria that should be applied for project assessment. This is a critical component of the project’s success because it enables to deliver the desired outcomes. In this step it is important to ensure that the set of criteria is complete and consistent with and directly linked to the intervention logic of operational programmes and their monitoring and evaluation indicators. The way in which this can be arranged includes both technical and quality criteria. The former stresses the so called formal criteria that are in general common for all measures and types of projects to be implemented under operational programmes. The latter, known as substantive criteria, refer to the merits of projects and make it possible to assess, first of all, their quality but also the beneficiary’s capacity needed to implement projects, and consequently to ensure the achievement of their outcomes. The process of selection comprises a two-phase assessment procedure, carried out by the managing authorities (MA) or intermediate bodies (IB), and it is based on the above-mentioned selection criteria. Furthermore, the process of project selection is assisted by external experts that bring their professional expertise and offer an independent and objective viewpoint on the content of the submitted proposals within a specific area of the EU funds implementation. It is also noteworthy that the selection procedure, as well as the criteria for the selection of projects and experts, should be approved by the monitoring committee and made available to the public in a very understandable way.

Coming back to the matter of selection criteria, it is important to emphasize that the two-phase assessment procedure is indeed a way of picking the most corresponding projects to the requirements of a given operational programme. To put it briefly, in the first phase (formal appraisal) the project’s compliance with administrative rules and eligibility criteria is checked (date of submission, correctly filled application form, requested documents attached, proper target group for support etc.). The formal appraisal is "zero-one" verification. It means that projects are rejected because they do not meet the administrative and technical requirements. If so, they are excluded from the application for funding in the second phase of the procedure, unless the rules of the competition provide the opportunity to complete or improve the documentation. This assessment is usually taken by employees of the institution that is responsible for organizing the competition. In turn, the substantive appraisal, covering only those projects which fulfilled the formal criteria, is based mostly on a scoring system, and results in establishing a ranked list of all applications that have passed. In this phase the assessment includes the nature of the projects, their relevance with and contribution to the objectives of an operational programme, their time schedule, sustainability and results in a given territory, financial and economic efficiency as well as the monitoring, management and evaluation procedures planned (see more in table 1). The substantive appraisal is carried out by external experts. Some clarifications or completions concerning specific issues of the projects are allowed at any phase of the selection, as long as it is foreseen in the competition arrangements.

The second step in the process of selecting projects is a proper selection mode. This determines the selection of projects to achieve the objectives of the operational programme. Studies show that the objectives of the programme as well as characteristics of the eligible area are the main factors which affect the decision on what type of a particular mode of project selection should be used (MRD 2013). Therefore, an appropriate selection system of projects has a fundamental significance for the efficiency and effectiveness of interventions co-financed by EU funds. This also implies that the individual types of mode can be decisive in efficient allocation and consequently in achieving the overall objectives of programmes. In Poland, the project selection modes have been defined in Article 28 of the Act on the principles of development policy, and they include:

• individual mode – refers to projects whose implementation is relevant and justified from the point of view of a particular strategy or area and which will significantly contribute to achieving the objectives of the priority axis of a programme,
• systemic mode – relies on financial support of public tasks specified in some regulations (e.g. information society, labour market, business environment etc.) and implemented by specific institutions,
• calls for proposals – selected within the open/permanent or closed/temporary calls announced and conducted by a specific implementing body; it is particularly carried out in compliance with the principles of transparency and access to information.

In fact, the law defines the principles in a very general and vague manner. Thus, there is much that individual institutions can do in this field and, in addition, they can interpret them to some extent by themselves.

As a matter of course, the system for project selection provides value for all stakeholders engaged and gives an opportunity to determine if a project will have a strategic impact to improve the local or regional situation. Hence, the success of the project’s approach depends on the proper selection of projects, and moreover selection of projects is a fundamental in the project’s implementation (Jigeesh 2012). So, there is a need to provide further evidence for the debate on project selection
within the framework of European funds.

3. Research methods

The methodology used in the paper bases substantially on qualitative approach but elements of quantitative approach are also included. This approach differs from those applied in the reports by CRSG from 2013 for selected Polish operational programmes (with the main focus on sectoral programmes) or the European Commission report from 2012 where each step of the project selection was compared country-by-country (European Commission 2012). The assessment carried out in the study is based on methods and techniques that are appropriate for evaluation. However, it must be stressed that the adopted research path does not fulfill fully the principles of evaluation[4]. The system of project selection has a broad nature and requires the use of many different methods.

As a matter of course, the methods used in the paper comprise desk research, including analyses of documents related to the implementation of various operational programmes, the documentation of projects selected under each mode, the results of other evaluation studies or reports, legislation analyses (including legislation coming from the European Union, Polish acts and regional law coming from the Wielkopolska region that set up an interesting three-layers of generally applicable and non-generally applicable law), statistical analyses of primary data obtained from the Marshall’s Office of the Wielkopolska region (Managing Authority – the MA of the Wielkopolska Regional Operational Programme for the years 2007–2013 (WROP). The analyses of the statistical data were complemented by In-Depth-Interviews (IDI) with the stakeholders taking part in the process of project selection and application (9) and our own experience in the process of law creation and consultation with the Polish Ombudsman by one of the authors.

4. Selection modes in the Wielkopolska region

Regarding all of the above considerations, it is noteworthy that the Managing Authority of the Wielkopolska region while building the system for project selection has adopted exactly the manner considered in this paper. In the process of the implementation of the Wielkopolska Regional Operational Programme for 2007–2013 (WROP), procedures for project selection have been adjusted to the particular measures and priorities with regard to the specific component of the Programme. Basically, only two selection modes are applied, namely, the individual mode and the call for proposals (open or closed calls) (BoWR 2014) as mentioned in the section 2 (see table 1). In moving on to discuss the call for proposals, it is crucial to highlight that this mode can be carried out in the form of two proceedings. The first one is based on the one-phase approach and results in selecting projects from the total number of submitted applications. The second proceeding relies on a two-phase approach, which encompasses the so called pre-selection as a first review of projects in term of their suitability for the OP priorities, and the right selection in which potential beneficiaries who have passed the pre-selection are invited to complete the documentation[5]. As a result of the pre-selection procedure, applicants do not need to bear the additional preparation costs of their project documentation when it is rejected due to formal or technical reasons. Moreover, the pre-selection reduces risk, because it enables to complete the application as a consequence of the preliminary assessment on the project’s merits, in particular when obtaining funding is more likely (see more Scheme 1 and Scheme 2).

 

The analysis shows that the one-phase selection of projects has been used most frequently by the Managing Authority in the Wielkopolska region. In total, 79 proceedings of the call for proposals have been conducted to date and only 8 of them were preceded by pre-selection as a preliminary assessment. About 10% of all submitted applications (6942) within the last seven years were assessed through the two-phase procedures. At the same time this corresponds to 23.4% of WROP funding allocated among beneficiaries in the Wielkopolska region. Furthermore, it must also be said that the pre-selection procedure is especially adequate for those measures where financial aid is offered to a relatively wide group of beneficiaries, and where the preparation of projects requires a high attention and engagement from applicants as well as it entails comparatively high costs. This refers in particular to technical and infrastructural projects. Within the framework of WROP, the pre-selection was applied for projects concerning the following interventions: investment area development, sewage treatment, waste management, investments in education infrastructure and projects for the use of alternative energy sources.

Table 2 presents the data concerning progress in the implementation of WROP as well as the results of the selections of projects. According to these data, since the start of programmes until 5th May 2014, 6942 applications were submitted to the regional Managing Authority. As a result of the formal appraisal, 4010 of them (for the global amount of co-financing of PLN 8.6 billion) fulfilled the formal criteria and were approved for the further assessment. This means that over 40% of all applications submitted were rejected at the first phase of selection, which ranks the Wielkopolska region second among all 16 Polish regions (MID 2014, p. 63). During the same period, after carrying out the substantive appraisal, 2071 contracts for co-financing were signed with beneficiaries for the amount of PLN 5.5 billion. This highlights that only less than 30% of all the applications submitted were granted financial support within the framework of the WROP.

By referring to these results, it is possible to argue that the process of project selection, under the conditions of a limited amount of funding, enables to reject a relatively large number of projects that do not meet the OP requirements even at the formal appraisal phase. It is worth noting that under the procedure including pre-selection, the number of projects to be rejected is much smaller and amounts to 408, which constitutes almost 57% of all applications submitted to the preliminary appraisal. It is also reasonable to think that the use of pre-selection seems to be a proper mechanism to carry out the selection of projects more effectively and significantly increases the efficiency of the entire system of selection.

Apart from the call for proposals, a certain number of projects can count on funding without being assessed in terms of fair competition. These projects are covered by the individual mode as a type of project selection. The individual projects selected under this mode are considered as very important from the programme’s strategic point of view. Their implementation is essential to achieve the indicators of the socio-economic development of the region and they should contribute significantly to the objectives of the Strategy for Development of the Wielkopolska region. According to the assumptions of the Board of the Wielkopolska region, such investments are characterized by their primary importance for the region, are very specific in particular areas, innovative and often unique (BoWR 2010). These projects are selected on the basis of decisions taken by the Board of the region following widely conducted public consultations on fulfilling the strategic criteria of the Programme by those projects. However, research conducted in this field shows that in practice the public consultations influence the final shape of the list of individual projects in quite a limited way (IDI 2014[6]).

 

By comparing the data in Table 3 we obtain a picture providing the information that the largest number of individual projects (27) falls to the priority V “Infrastructure for human capital”. However, the biggest amount of WROP funding, PLN 1.1 billion, was allocated to projects included in the priority axis II “Communication infrastructure”. It is remarkable that 60 projects (less than 3% of the total number of contracts signed) were selected by using the procedure reserved for individual projects, which received almost 30% of the total allocation spent to date. This means that a relatively high amount of WROP funding is distributed through the procedure where the so called strategic criteria, often set at a high level of generality, are taken into consideration and which in fact do not always contribute properly to the strategic objectives of the region (IDI 2014[7]). As a result, most of the projects are able to meet the established criteria, which make this type of selection very questionable.

5. Procedure of project selection

Selection of the best projects to be destined for support from European funds, assure contributing as much as possible to achieving the objectives of the OP. The key question addressed in the paper is whether the system for project selection in the Wielkopolska region enables to select such projects that bring about the implementation of the OP objectives effectively and successfully. In order to provide an answer to this question, a couple of interviews with applicants and beneficiaries were carried out (IDI 2014[8]).

First of all, it must be stressed that among applicants and beneficiaries there is a full understanding of the need to conduct such proceedings which result in selecting those projects that to the highest possible extent fulfill the approved criteria. However, some criteria are defined sometimes imprecisely, in particular, when taking into consideration the individual projects. Although this may be a cause of doubts on the final results of project selection, the interviewees are aware of the fact that the individual mode provides the opportunity to finance projects of particular importance for the region. They suggest that the individual projects should be limited to large projects (though what “large” really means remains to be defined), and they should be also assessed in term of their suitability on the basis of ex-ante evaluation. This should be done because public consultations alone seem to be insufficient.

According to the interview results, the call for proposals is the most appropriate mode of project selection. This mode offers the highest quality of criteria to be applied for a project’s selection. In other words it makes it possible to select the best undertakings because it provides usually a fair competition. Results also show that the selection, without doubts, refers to some types of projects, e.g. environmental, in communication infrastructure, thermo-modernization, health care etc. These projects can be characterized by a relatively wide set of data based on objective sources. Hence, the technological, financial and economic viability of a project expressed by many indicators allows for an objective assessing and comparing of that project. As a consequence, it is possible to: find out whether the project should be directed to implementation; make a fair comparison of submitted projects in a given sector; select the most effective projects in conditions of a limited amount of budget; and finally, select the most effective projects from mutually exclusive ones. Such an opportunity is very limited in the area of cultural, social and tourism projects, where it is difficult to collect objective data concerning especially their future functioning and sustainability. These analyses are often based on assumptions which are difficult to be verified in practice. Therefore, according to interviewees, the assessment of this kind of project is less competitive and the process of projects selection – less transparent (IDI 2014[9]).

In addition, the interviewees responded positively to the keep both individual mode of selection of projects as well as the call for proposals selection of those (activities) projects that are envisaged for implementing in some strategic documents, e.g. the National programme of domestic sewage treatment, National plan for waste management 2010 etc. These projects play a key-role in providing access to public goods. In principle, it is supposed that these projects are better prepared and the process of selection seems to be more objective. The interviewees pointed out also the role of pre-selection, which to date is used to a limited extent but which would certainly contribute to reducing the costs of selection, in particular on the applicant’s side. There is a necessity, according to the applicants, to widen the extent of the call for proposals encompassed by the preliminary appraisal (pre-selection), which needs to be addressed.

6. Assessment of expert work in WROP 2007–2013

As mentioned in the previous section, in WROP 2007-2013 there were two modes of project selection among 4 present in all the operational programmes in Poland (Article 28 of the Act on the principles of development policy, (Dz. U. 2009, Nr 84, p. 712 with changes, u.z.p.p.r):

• Individual projects
• Systemic projects
• Closed/temporary Call for proposals (CfP, with a deadline for the submission of applications), or so called “competitions”
• Open/permanent Call for proposals (without a deadline for the submission of applications)/competitions.

The majority of projects of WROP were selected as a result of a closed CfP that according to the report on evaluations of the operational programmes in Poland should generate better quality projects (CRSG 2013) due to the participation of experts in the selection process.

The experts involved in the whole process of the assessment of proposals of WROP were working under the closed CfP mode. They were selected in a competitive way under another and earlier call for experts. The experts had to fulfill the criteria for experts set down in article 31 of the Act on the principles of development policy (Dz. U. 2009, Nr 84, p. 712 with changes, u.z.p.p.r), namely, the expert: enjoys full public rights, has full legal capacity, has not been convicted of an offence or for an intentional crime and fulfills other requirements laid down in the system of implementation of the operational programmes. In the case of WROP, the expert had also to have graduated from university and have had at least 3 years of professional experience in the area of expertise (WROP, 2014)[10]. One of the most important criteria, however, is the declaration of objectivity and the lack of any conflict of interests during the assessment procedure. In the case of WROP this criterion was very much checked and, as a result, it excluded many of the potential evaluators from each particular CfP. It has also another negative aspect when, in some years, the experts remaining for the evaluation process were overloaded with work, because the others were involved in the process of the application or consultancy to companies that had prepared the applications for beneficiaries (IDI 2014[11]).

The whole list of potential experts was published and updated on the website of the MA of WROP and integrated with the central list of experts managed by the Ministry of Infrastructure and Development (the former Ministry of Regional Development – MA for all sectoral OP). The experts were grouped into specialization areas of WROP and were invited to evaluate the proposals by the Marshall’s Office (MA) each time when the CfP was launched[12].

The procedure of selection was based, as mentioned in the previous sections, on two kinds of appraisal/assessment:

• Formal/Internal appraisal (the formal one with elements of strategic criteria) – done by the officials of the MA and IB
• Substantive/External appraisal – on merit, where 2 or 3 specialists in the area were assessing the proposal with the use of assessment sheets[13].

The total number of closed calls for proposals from the beginning of the financial perspective till the end of 2013 was 73 in the case of WROP, while the average number of proposals sent per CfP was 103 (MID, 2014 p. 54, 59)[14]. In the most numerous calls, which were calls for enterprises during the whole period 2007–2013, the number of applications per session per expert varied, depending on the total number of applications (e.g. during the two-week evaluation session experts could have assessed between 20 to 60 applications – IDI 2014[15]). However, due to the quite efficient system of work, there were no delays comparing to formal documentations providing information on the time of the overall appraisal procedure.

The experts, as already mentioned, used evaluation sheets with a point scale. Each session was opened by the MA/IB that organized a short seminar on how to interpret criteria and what changes had been introduced in the subsequent call for proposals that was evaluated very positively by the experts. In the opinion of the experts the criteria were clear, especially when a “0–1” scale was used. The most subjective and difficult were the evaluation of the level of innovativeness in the applications of enterprises where a 0–9 points-scale was introduced. In this case, each decision needed to be clearly justified. The experts positively evaluated the fact of a possible confrontation of their opinions and the fact of having two specialists from different areas of specialization dealing with the same application form. When the experts’ assessment differed to a large extent, a third expert was involved in the process and this opinion prevailed in the whole appraisal. In the case of smaller differences, very often the MA/IB played a role of mediator in the whole process and the experts tried to come to a common final decision on the status of the application.

Among other positive elements, the experts underlined the efficiency of the experts, their mutual complementary knowledge and experience.

The negative aspects related to the whole process in the WROP were: an obligation to assess applications in the headquarter of the MA, that was problematic for some of the experts (due to the low elasticity of working time), but this had also its positive aspect as far as the possibility of consultation among experts is concerned. Another negative feature according to the experts was a lack of transparency in inviting experts for the assessment sessions. (IDI 2014[16])

The general assessment of the expert system in the eyes of the experts was positive as it brought more objectivity and a selection of better quality projects, which was one of the assumptions of the whole system. In the course of time, the quality of the submitted proposals has also risen.

 

7. Appeal procedures

The right to appeal is one of the most important rights of each citizen, entity or institution that comes from the right “to a fair and public hearing of his case, without undue delay, before a competent, impartial and independent court” (art. 45 of the Constitution of the Republic of Poland of April 2 1997). It is a means for the defense of freedoms and rights. Art. 78 of the Constitution states “each party shall have the right to appeal against judgments and decisions made at the first stage. Exceptions to this principle and the procedure for such appeals shall be specified by statute.” (Dz. U. 1997, No. 78, item 483). It is guaranteed in the basic legal act regardless of the area in which the appeal can be reported. However, the problem of the legal protection of potential beneficiaries applying for public resources has been debated in Poland for many years because of difficulties in defining the character of decisions, or contracts and therefore the cognition of courts (Gajda 2005, p. 652; Szuster 2005, III SA/Wa 2380/04, Wosik 2006, p. 56-59, Talaga 2012, p.320-322).

With reference to appeal procedures in the Cohesion Policy in Poland 2007–2013, till the end of 2008 these procedures were not present in the Polish legislation dealing explicitly with EU funding. This was an infringement of the rights of potential beneficiaries and against mainly articles 45 and 78 of the Constitution. The amendments of 20 December 2008 of the Act on the principles of development policy introduced the rights to appeal. However, what is still noteworthy, Poland prepared specific legal solutions that are different to those referring to the use of other public resources or procedures before the courts. There have been: the special administrative procedure on granting EU funds (an exemption from the Administrative Procedure Code of 14 June 1960) and the special administrative courts’ procedure on control of the distribution of EU funds (an exemption from the Law on Proceedings before Administrative Courts of 30 June 2002, Nyikos, Talaga 2014 p. 12.). This differentiation altogether with new areas of analyses coming from the proposals of projects were challenging for the judges (IDI 2014[17]). Moreover, different appeal modes in the operational programmes could have been applied by the MA that was a sign of a differentiation of beneficiaries’ rights. This fact was also examined by the Constitutional Tribunal of Poland. As a consequence, in the second part of 2013 the subsequent amendment to the Act on the principles of development policy, namely, a unification of the appeal procedure for beneficiaries, formed a further step towards procedural coordination (Nyikos, Talaga 2014 p. 10; Musiałkowska, Talaga 2013 p. 12).

The appeal procedure introduced on 20 December 2008 on the basis of the horizontal guidelines by the Minister of Regional Development opened to the MA a possibility of choosing the appeal means and their number (in 1 or 2 instances) at the stage of pre-judiciary control. Then, after exhaustion of the pre-judiciary appeal means, the judiciary control could have been started, namely appeals to the provincial/voivodship administrative courts (VAC) and as the highest instance – appeal to the Supreme Administrative Court (SAC)[18].

In the case of the Wielkopolska region, the detailed description of the priorities of WROP 2007-2013 adopted by the Act by Board of WROP 2007–2013 included information on the system of project selection together with the appeal procedures under the WROP. It consisted of one pre-judiciary appeal means – protest and two of judiciary control – an appeal to the VAC and appeal to the SAC.

Due to the strong competition and “fight for money” many appeals from negative decisions have appeared that also confirm the learning process and rising of awareness of beneficiaries of their rights.

While analyzing the appeal procedures of WROP 2007–2013 in numbers, one may observe the following tendencies. At the stage of pre-judiciary control:

• Between 20.12.2008–31.13.2013 there were 1306 protests (while in all Polish ROP – 17 076 first instance appeal means +598 second instance appeal means; in all OP – 63 539 appeal means, including 5119 auto-controls by the institutions)
• Between 20.12.2008–31.06.2013 the number of the WROP appeal procedures with a positive decision for the beneficiaries was 190 (which were 12% of the total number of 1585). The number of positive decisions for beneficiaries is one of the lowest in the country, after the Podlaskie region (7% of total protests) and in contrast to the Świętokrzyskie and Masovia regions – with the highest percentage of positive decisions – respectively 47 and 43% (MID 2014, p.67-70).

 

In general, the percentage of appeal procedures that ended with a positive decision for the beneficiaries between 20.12.2008 and 31.12.2013 in all the ROP was 32% in the first instance and 25% in the second instance (MID 2014, p69).

 At the stages of judiciary control:

• Between 20.12.2008 and 31.12.2013 there were 66 appeals[19] to the Voivodship Administrative Court in Poznań in WROP (while in all ROP the number was 1559 – the majority of all 2157 OP appeals to the VAC, MID 2014 p. 67-71). 73% of all appeals were considered by the VAC, 26% of all – not considered by the VAC (according to the u.z.p.p.r.) and the rest – 1% – is waiting consideration. In the Wielkopolska region 85% of appeals were considered, 12% – not considered – and 3% – rejected (SAC 2014[20]).
• The percentage of appeal procedures to the Voivodship Administrative Courts that ended with a positive decision for the beneficiaries[21] in the whole of the country between 20.12.2008 and 31.12.2013 was 26% (MID 2014 p. 67-72). In the case of WROP only 20% (13 of all analyzed cases) ended with a positive judgement for the beneficiary) (SAC 2014[22]).
• Between 20.12.2008 and 31.12.2013 there were 5 appeals to the Supreme Administrative Court in the WROP (SAC 2014). In the whole of Poland the number was 519 for all OP – 391 came from potential beneficiaries/beneficiaries, while 128 – from institutions (MA, IB etc.). Respectively, for all ROP it was 349: 257 – from potential beneficiaries/beneficiaries, 92 – from institutions (MID 2014, p. 67-72).
• In total 29% of all appeals ended with a positive decision for beneficiaries and 23% of all – ended with a positive decision for institutions (MID 2014, p. 67-72). In the case of all appeals reported on the WROP all ended with negative decisions (3 judgements, 1 – rejected, 1 – waiting consideration).

With regard to appeals to the administrative courts in Wielkopolska there was no domination of appeals under any particular axis of the WROP. However, what was striking, very often beneficiaries expected judgements on the merit of their application and not on the legality of the whole proceeding with the application and distribution of funds. In conclusion, there is a confirmation of a non-deep enough learning process of the beneficiaries (IDI 2014[23]). Over the years there have been a growing number of appeals, which confirms the growing awareness of the rights to appeal and also due to the obligation of there being detailed information on the rights of beneficiaries. On the other hand, the potential beneficiaries were not aware enough of the role of the administrative court (IDI 2014)[24].

The results confirm also a low illegality of the institutions performance in the case of WROP:

• There was a relatively low number of positive decisions for potential beneficiaries – 26% in the whole of Poland and even lower in the Wielkopolska region – 20% (but the majority of them
  after obtaining the decision obtained a grant from the reserve of the MA).
• The relatively few reasonable arguments confirmed irregularities and illegality of operations of institutions.

 

Summing up, during the years 2007–2013 there was quite a striking lack of coherence of law creation and execution with regards to appeal procedures (due to the necessity of and focus on timely spending of EU money and the fears of time-consuming appeal procedures stressed by the administrative bodies: MRD and other MA[25]).

During these years the judgements by the VAC and SAC have played a signalising role in the process of the improving of legislation, while potential beneficiaries have started to use their potential during appeals and contributed to the better implementation of the Cohesion Policy in Poland, including WROP (Musiałkowska and Talaga 2012, p. 309, IDI 2014). Both the judgements of the administrative courts and the activity of beneficiaries led to the final judgement by the Constitutional Tribunal (P 1/11) that unified the procedures in the whole country.

8. Conclusions

The literature comes to the conclusion that a success of the project approach aimed at achieving specific objectives depends on the proper selection of projects, and additionally project selection is a fundamental component in the process of the project implementation. Therefore, the selection of projects for funding within the framework of the EU cohesion policy is one of the most relevant processes to ensure the effective and efficient implementation of the operational programme.

An initial finding of the paper confirms that the system for project selection under WROP provides in general clear rules of selection and the selection itself is carried out transparently. The system based on the formal appraisal and substantive appraisal, as a second stage of the assessment, should contribute successfully to achieving the desired outcomes. To be more precise, the set of criteria in particular is directly linked to the intervention logic of operational programmes and their monitoring and evaluation indicators. Thus, this might be enough to bring about the objective of WROP and, consequently, the overall harmonious development of the Wielkopolska region. In turn, as regards the selection modes of a project, this paper indicates that basically two selection modes have been applied, namely, the individual mode and the closed call for proposals. As far as the second mode is concerned, it relies on a one-phase or two-phase approach. The analysis shows that the one-phase approach has been used most frequently by the Managing Authority in the Wielkopolska region, making up 71 out of 79 carried out proceedings of the call for proposals. The two-phase approach encompasses the so called pre-selection as a preliminary assessment in terms of the projects suitability for the OP priorities. In the Wielkopolska region, about 10% of all submitted applications within the last seven years were assessed through the two-phase procedures. It was also noted that the number of rejected projects within the procedure, including pre-selection, was much smaller, which means that this way of selection increases the efficiency of the entire system of selection.

The evidence regarding the individual mode of project selection indicates that less than 3% of the total number of contracts signed within the last seven years was selected by using the procedure reserved for individual projects. It is worthwhile noting that those projects received almost 30% of the total allocation spent to date under WROP. This means that a relatively high amount of the WROP funding is distributed through a very general set of (strategic) criteria. So, there is a need to make the selection of these projects more precise, and to introduce for instance the ex-ante evaluation conducted in terms of their suitability. In addition, there is also a necessity to widen the extent of the call for proposals encompassed by pre-selection because it has contributed certainly to reducing the costs of selection, especially for applicants.

While referring to the quality of the work of experts involved in the process of the assessment of the proposals, one may conclude that they played an important role in contributing to picking up the best possible projects under the closed call for proposals. The element of competitiveness introduced to the system was positively assessed by the experts, institutions and beneficiaries themselves, because of their greater objectivity and better quality of projects that obtained financing from EU funds. Also the MA contributed to having the system working properly through the selection of experienced experts and by providing training sessions for them at the beginning of each assessment session. The element that requires improvement is greater transparency in the invitation of the experts to the evaluation sessions. Moreover, quite the good quality of the WROP system is confirmed by the statistics and the opinions of judges while referring to the appeal procedures. The growing number of appeals confirms the rising awareness of the potential beneficiaries of their rights on one hand, but the low percentage of positive decisions towards them at both the pre-judiciary and judiciary stages of appealing, was to some extent a confirmation of quality and most of all the legality of procedural aspects of project selection on the other.

More flow of information and common platforms of learning are also claimed by both, the MA/IB and experts or judges involved in the whole process in order to know better the assumptions of the system and improving its quality (IDI 2014[26]).

 

 

 

Underline notes:
[1] In the paper two terms are used: “evaluation” with regard to concepts and techniques of the evaluation process and “assessment” – with regard to assessment/appraisal of the projects. Only once were these terms used interchangeably while writing about experts’ assessment or evaluation of the project with a meaning of taking part in the procedure of assessment.
[2] The term beneficiary is used while referring to an entity that implements the project, while the term “potential beneficiary” – to an entity applying for funding before signing the agreement on co-financing the project from EU money and often being a part of appeal procedures (see more: next sections of the article).
[3] Systemic projects are not analysed because in the case study – WROP 2007–2013 – only two modes were applied: individual mode and call for proposal (see more section 4).
[4] Evaluation is a systematic acquisition and assessment of information with the purpose of providing useful feedback about the quality and value of public programs to some audience. It is a process determining the extent to which some actions were successful in achieving intended outcomes. In contrast, assessment is only one element of this process [Forunier 2005, Rossi, Lipsey, Freeman 2004, Olejniczak 2008].
[5] The second approach fits in well with approaches identified in six Member States with regard to the selection of projects co-financed from the European Regional Development Fund: Austria, Czech Republic, Germany, the Netherlands, Slovakia and Slovenia.[European Commission, 2012, p. 25]
[6] Interview with beneficiaries of the WROP, 29 May 2014, Poznań
[7] Interview with the member of the Monitoring Committee of the WROP, 24 June 2014, Poznań
[8] Interview with the beneficiaries of the WROP, 5-6 June 2014, Poznań
[9] Interview with the beneficiaries of WROP, 9 April, 29 May, 5-6 June 2014, Poznań
[10] http://www.wrpo.wielkopolskie.pl/index.php/wrpo-informacje/instytucje-realizujce-wrpo118/instytucje-zaangaowane-w-realizacj-wrpo accessed July 10 2014
[11] Interviews with the expert 1 and expert 2 on 29 May, 4 June 2014
[12] According to the list for August 2013 there were 350 experts grouped in 43 specialization areas (some experts belonged to more than one category). The most numerous areas were those related to operation of enterprises and economic analyses while the least numerous – those that required specific legal and technical knowledge (own calculations www.wrpo.wielkopolskie.pl/index.php/wrpo-informacje/instytucje-realizujce-wrpo118
/instytucje-zaangaowane-w-realizacj-wrpo accessed July 10 2014)
[13] Both sheets serving for formal and substantive appraisal are available for potential beneficiaries and are part of CfP documentation.
[14] The number of applications sent under different axes of the programme differed very much. Since 2010, in general the number of application in all Polish ROP has dropped comparing to the sectoral progammes [MID 2014, p. 59]
[15] Interviews with the expert 1 and expert 2 – 29 May, 4 June 2014
[16] Interviews with the expert 1 and expert 2, 29 May, 4 June 2014
[17] Interview with the judge of the Voivodship Administrative Court in Poznan, 24 July 2014
[18] Article 184 of the Polish Constitution states that: “The Supreme Administrative Court and other administrative courts shall exercise, to the extent specified by statute, control over the performance of public administration. Such control shall also extend to judgments on the conformity to statute of resolutions of organs of local government and normative acts of territorial organs of government administration.” Dz. U. 1997, No. 78, item 483
[19] SAC 2014 www.orzeczenia.nsa.gov.pl accessed 6 June 2014
[20] Own work on the SAC 2014 www.orzeczenia.nsa.gov.pl accessed 6 June 2014
[21] Positive decision did not have to lead to obtaining of the grant however. If the allocation per call for proposal and the reserve were used the beneficiary was entitled to claim only.
[22] Own work on the SAC 2014 www.orzeczenia.nsa.gov.pl accessed 6 June 2014
[23] Interviews with the judge and assistant to judge and referendary of the Voivodship Administrative Court in Poznan, 28 May 2014 and 24 July 2014
[24] Interviews with the judge and assistant to judge and referendary of the Voivodship Administrative Court in Poznan, 28 May 2014 and 24 July 2014
[25] Participation, observation and consultation of one of the authors in legislative process in the advisor group on EU Funds of the Office of Polish Ombudsman (the years 2007–2010).
[26] Interviews with the judge of the Voivodship Administrative Court in Poznan 24 July 2014 and expert 1 and expert 2, 29 May, 4 June 2014

 

References

[1] Badanie ewaluacyjne „Ocena stosowanych trybów wyboru projektów w ramach programów operacyjnych na lata 2007-2013”. https://www.ewaluacja.gov.pl/Wyniki/Documents/Ocena_stosowanych_trybow_wyboru_projektow_13052013.pdf

[2] Bachtler, J., Yuill, D., (2001) Policies and Strategies for Regional Development: A shift in Paradigm?. Regional and Industrial Policy Research Paper, No. 46: p. 7-12. http://www.eprc.strath.ac.uk/eprc/documents/PDF_files/R46
PoliciesandStrategiesforRegionalDevelopment.pdf

[3] Bienias, S. et al. (eds) (2012) Ewaluacja. Poradnik dla pracowników administracji publicznej, MRD, Warsaw. https://www.mir.gov.pl/rozwoj_regionalny/Polityka_rozwoju/Projekt_Zarzadzanie_Strategiczne_Rozwojem/
Przydatne_dokumenty/Documents/EWALUACJA_z%20okladkami.pdf

[4] Bienias, S., Sudak, S. (eds.) (2008) Proces ewaluacji polityki spójności. Podsumowanie dotychczasowych doświadczeń. Plany i wyzwania na przyszłość. Krajowa Jednostka Oceny, Departament Koordynacji Polityki Strukturalnej, MRD, Warsaw.

[5] BoWR (2010) Zasady wprowadzania projektów do Indykatywnego Wykazu Indywidualnych Projektów Kluczowych Wielkopolskiego Regionalnego Programu Operacyjnego na lata 2007-2013. Poznan.

[6] BoWR (2014a) Wielkopolska Regional Operational Programme for 2007–2013 Bord of the Wielkoposka Region, Poznan.

[7] BoWR (2014b) Detailed Description of the Priorities of the Wielkopolska Regional Operational Programme for 2007–2013. Bord of the Wielkoposka Region, Poznan.

[8] CRSG (2013) Badanie ewaluacyjne „Ocena stosowanych trybów wyboru projektów w ramach programów operacyjnych na lata 2007-2013”, Szczecin.

[9] Dąbrowski, M. (2012) Towards Strategic Regional Development Planning in Central and Eastern Europe?. Regional Insights, Vol. 3(2): p. 6-8.

[10] Dutra, C., Ribeiro, J., de Carvalho, M. (2014) An Economic–Probabilistic Model for Project Selection and Prioritization. International Journal of Project Management, No. 32: p. 1042–1055.

[11] Dz. U. of 1997, No. 78, item 483, The Constitution of the Republic of Poland of 2nd April 1997.

[12] Dz. U. of 2000, No. 98, item 1071, The Act of 14th June 1960 – the Code of Administrative Proceedings.

[13] Dz. U. of 2002, No. 153, item 1270, The Act of 30th August 2002 – the Law on Proceedings before Administrative Courts.

[14] Dz. U. of 2009, No. 84, item 712, The Act of 6th December 2006 on the Principles of Development Policy.

[15] European Commission (2004) Aid Delivery Methods. Project Cycle Management Guidelines. Brussels.

[16] European Commission (2012) Comparative Study of the Project Selection Applied in Cohesion Policy Programmes 2007-2013 in a Number of Member States. Final Report. Brussels http://ec.europa.eu/regional_policy/sources/docgener/studies/pdf/selection/selection_process.pdf

[17] Forunier, D. M. (2005) Evaluation, in: Mathison S. (ed.), Encyclopedia of Evaluation. Thousand Oaks: Sage Publications.

[18] Gajda, A. (2005) Spójność społeczna i gospodarcza [in:] Prawo Unii Europejskiej – prawo materialne i polityki (2005), ed. J. Barcz. Warszawa: Wydawnictwo Prawo i Praktyka Gospodarcza.

[19] IBC Group/MRD (2010) Raport końcowy z badania „Ocena systemu zarządzania i monitorowania projektów indywidualnych”. Warszawa

[20] Idczak, P., Musiałkowska, I., Sapała, M. (2012) Performance Turn in the Cohesion Policy in Poland in the Years 2014–2020? Analysis of the Strategy of Development of the Łódź Region; RSA workshop “EU Cohesion policy at the crossroads: budget reform, geographical allocations and the performance turn in the 2014–2020 period”. Glasgow.

[21] Jigeesh, N. (2012) Selection of Project as Important Beginning for Information Technology Project Management. The IUP Journal of Operations Management, Vol. XI(1).

[22] Meredith, J., Mantel, S. (2009) Project Management. A Managerial Approach. Hoboken: Seventh edition, John Wiley & Sons, Inc.

[23] MID (2014) Potencjał administracyjny systemu instytucjonalnego Narodowych Strategicznych Ram Odniesienia na lata 2007-2013 (stan na 31 grudnia 2013 r.). Warszawa. http://www.funduszeeuropejskie.gov.pl/Wiadomosci/Str
ony/Raport_o_potencjale_administracyjnym_instytucji_realizujacych_NSRO_2007_2013_150714.aspx

[24] Millio, S. (2007) Structural Funds Implementation and Administrative Capacity. The case of Italian Objective 1 Regions. Regional studies, Vol. 41(4): p. 429-442. http://www.ssoar.info/ssoar/files/peer/Peer_91-2010-09-03.pdf.

[25] Molle, W. (2007) European Cohesion/Regional Policy. Routledge.

[26] MRD (2013) Badanie ewaluacyjne „Ocena stosowanych trybów wyboru projektów w ramach programów operacyjnych na lata 2007-2013”. Warszawa.

[27] Musiałkowska, I., Talaga, R. (2012) Kontrola wydatkowania środków z funduszy strukturalnych jako element zarządzania procesem rozwoju regionalnego w Polsce, Studia Komitetu Przestrzennego Zagospodarowania Kraju (KPZK) PAN, 01/2012, Volume 140/2011, Issue: Perspektywy rozwoju regionalnego Polski w okresie programowania po 2013 r. Cz. II: p. 297-311.

[28] Musiałkowska, I., Talaga, R. (2013) Legal Aspects of the Implementation of EU Funds 2007-2013 in Poland – Practice and Challenges, Regionalisation and Inter-regional Co-operation. 21th NISPAcee Annual Conference, 09/2013. Belgrade.

[29] Nyikos, G., Talaga, R. (2014) Cohesion Policy in Transition, conference paper from the 22nd NISPAcee Conference. Budapest.

[30] OECD (1986) Glossary of Terms Used in Evaluation, in Methods and Procedures in Aid Evaluation. Paris.

[31] OECD (1991) The DAC Principles for the Evaluation of Development Assistance. Paris. http://www.oecd.org/dac/evaluation/2755284.pdf

[32] OECD (2000) Glossary of Evaluation and Results Based Management (RBM) Terms. Paris.

[33] Olejniczak, K. (2008) Mechanizmy wykorzystania ewaluacji. Studium ewaluacji średniookresowych INTERREG III. Warszawa: Wydawnictwo Naukowe Scholar.

[34] P 1/11, The judgement of the Constitutional Tribunal of Poland of 12 December 2011.

[35] Poździk, R., (2013) Ocena i wybór projektów do dofinansowania z Europejskiego Funduszu Rozwoju Regionalnego, Europejskiego Funduszu Społecznego i Funduszu Spójności. Warszawa: Wydawnictwo Sejmowe.

[36] Rossi, P. H., Lipsey, M. W., Freeman, H. E. (2004) Evaluation: A Systematic Approach. Thousand Oaks–London: Sage Publications.

[37] SAC (2014) www.orzeczenia.nsa.gov.pl accessed 6 June 2014.

[38] Szuster, S. (2005) Komentarz do ustawy z dnia 20 marca 2002 r. o finansowym wspieraniu inwestycji (Dz.U.02.41.363). Stan prawny: 2002.07.30; LEX 2002 – p. 5-6 adnotacji dotyczącej art. 13 ustawy o finansowym wspieraniu inwestycji; postanowienie WSA w Warszawie z 06.04.2005 r., sygn. akt III SA/Wa 2380/04.

[39] Talaga, R. (2012) Wytyczne Ministra Rozwoju Regionalnego w zakresie prowadzenia polityki rozwoju, [Guidelines of the Minister of Regional Development in the scope of development policy] in: Legislacja administracyjna. Teoria, orzecznictwo, praktyka (2012), eds. M. Stahl, Z. Duniewska, LEX a Wolters Kluwers business. Warszawa. pp. 320-333.

[40] Wosik, Ł. (2006) Charakter prawny aktu o przyznaniu dofinansowania z funduszy strukturalnych UE - glosa do wyroku NSA z 8 czerwca 2006 r. (II GSK 63/06). EPS, No 12: p. 56-59.

Interviews

Interview with the expert 1 in the WROP – 29 May 2014, Brussels

Interview with the expert 2 in the WROP – 4 June 2014, Poznań,

Interview with the assistant of judge and court referendary of the Voivodship Administrative Court in Poznań, 28 May 2014, Poznań

Interview with the judge of the Voivodship Administrative Court in Poznań, 24 July 2014, Poznań

Interview with the member of the Monitoring Committee of the WROP, 24 June 2014, Poznań

Interview with four beneficiaries of the WROP on: 9 April, 29 May, 5 and 6 June 2014.

 

Participation and consultation of Ida Musiałkowska in legislative process in the advisor group on EU Funds of the Office of Polish Ombudsman (years 2007–2010).