Effective Evaluation and Funding of Research Projects in TUBITAK

Nuket Yetis , Ömer Cebeci, and A. Yavuz Oruc

TUBITAK is the central funding agency that oversees the allocation of government research and development funds to various research projects in both academia and small business firms. TUBITAK, which was founded in 1963, funds research in both basic and applied sciences through the following 9 research divisions

1. Basic Sciences Division
2. Health Sciences Division
3. Electrical, Electronic and Informatics Division
4. Engineering Division
5. Environmental, Atmospheric and Marine Sciences Division
6. Agriculture, Veterinary and Forestry Sciences Division
7. Defense and Security Technologies Division
8. Social and Behavioral Sciences Division
9. Public Research Projects Division

Beginning in 2004, TUBITAK adopted a new panel-based proposal review system to evaluate the proposals it receives from academia, private and public sectors. The general structure of the TUBITAK proposal review process is depicted in Figure 1. All proposals are received and classified by their fields of research into appropriate research divisions. Program directors within each research division then further divide the proposals they receive into groups by topics and identify panelists whose expertise can cover the research topics of the proposals in each such group. Once a panel is determined for a group of proposals, panelists are sent between 8 to 12 proposals to provide their evaluations of these proposals with respect to intellectual merit, broader impact, and feasibility criteria. This review process is generally concluded within two weeks of a scheduled panel meeting. Then the panel convenes to evaluate all the proposals collectively. During the panel meeting, each proposal is discussed at length and given a score between 0 and 10 with respect to each of the three criteria of intellectual merit, broader impact, and feasibility. Proposals are funded according to their weighted scores and availability of funds.


Figure 1. TUBITAK proposal evaluation and funding process.

Project reporting is a critical part of the funding decision process. Research projects that produce articles that are published in high quality journals are rewarded with additional funding. All proposals are required to include a section that summarizes the accomplishments of the principal investigator in previous TUBITAK projects that were awarded to the principal investigator. Table I shows the various indicators for the TUBITAK proposal evaluation process for years 2004, 2005 and 2006. As seen in the table, about a third of all proposals have been recommended for funding. During the last three years, more than 7000 proposals have been received and evaluated. One of the challenges TUBITAK faces is the limited pool of active researchers to form reliable panels. This is both a short and long term problem that must be resolved in order to make the proposal evaluation process more robust and reliable. One option that is being considered is to tap reviewers from other countries. 

Table 1. TUBITAK proposal evaluation and processing indicators.

The composition of a panel is largely determined by the subjects of the proposals that are to be reviewed by the panel. One of the most difficult problems in proposal evaluation is how to compose a panel for a given set of proposals. There are several constraints that must simultaneously be satisfied. These include (i) the cost of panels as determined mostly by the number of panelists, (ii) covering the subject of each proposal, (iii) managing conflicts of interests, and (iv) ensuring a reliable evaluation of proposals. Typically, a reliable evaluation of a proposal requires at least two expert panelists so that each proposal is assigned to 2 panelists, and each panelist is assigned 4 proposals at maximum.