What is the Peer Review Process?
The Peer Review Process was implemented on foot of a Government Decision dated 11th Oct 2005. The process is applicable to major ICT projects being run by Government Departments and State Agencies and organisations under the aegis of Departments and Agencies.
A peer review is an independent, structured review of a programme or a project carried out at key decision points by a team of experienced people, external to the Sponsoring Organisation, i.e. the Department/Agency in charge of the project.
Reviews are carried out at key decision points of a project, including, inter alia:
• the business case;
• the preparation of the RFT and subsequent tender evaluation exercise;
• the proposed contract;
• project initiation;
• key milestones in the project cycle; and
• project completion.
The Peer Review Process does not override the ultimate decision making and the concomitant accountability arrangements of the sanctioning authority and/or sponsoring organisation.
The Peer Review Process is facilitated/managed by the Peer Review Unit (PRU) of IT Vote Control in the Department of Public Expenditure & Reform in respect of projects sponsored by central Departments and Offices. The PRU, if requested, also advises other public sector bodies on the operation of the process in respect of projects that they sponsor.
How are projects selected for review?
In deciding whether an ICT project should be submitted to the peer review process, the Sponsoring Organisation should take the following factors into consideration:
• estimated cost – projects with an estimated cost of €5 million or more must be submitted to peer review;
• risk – projects with a significant element of risk due to their scope, time-scale, technology approach; priority within an organisation; and the capacity of an organisation to take it on board;
• impact – projects which propose large-scale changes to work practices or staffing issues or have significant programme or policy implications;
• regulatory framework – projects with significant legal compliance outcomes; and
• dependency – projects with cross-organisational or multi-organisational interfacing or dependency elements.
The sponsoring organisation must inform IT Vote Control in the Department of Public Expenditure of any project within the remit of Circular 2/09 – Arrangements for ICT Expenditure in the Civil and Public Service – that should be submitted to Peer Review.
IT Vote Control may also of its own accord request a Peer Review in respect of any project for which it is required to sanction expenditure. IT Vote Control formally requests the initiation of a review using the Peer Review Request Form.
Good practice would suggest that projects not selected for Peer Review should be subject to an equivalent internal review within an organisation.
The PRU nominates individuals with relevant experience and proven track records in a variety of disciplines to act as Peer Reviewers. Typically, they will be experienced in the following disciplines:
• programme/project management;
• contracts and contract management;
• change management;
• project lifecycles; and
• technical architectures.
Reviewers are generally officials (serving and retired) from the public service, the third-level education sector, and relevant private sector organisations etc.
The PRU normally assigns a Peer Review Group (PRG) of four members to a project. One member of the group is nominated to act as chairperson. The chairperson facilitates all interactions between the PRG and the Sponsoring Organisation.
The PRU briefs each PRG on the process to be followed and assists in the administration of the review.
Role of the Peer Review Group
The PRG examines project documentation and consults with the project management team, stakeholders and any other relevant parties. Based on this review the PRG is tasked with reaching a conclusion as to whether, in its opinion, a project is fit to proceed to the next stage of the lifecycle.
In arriving at this viewpoint the PRG has regard to certain criteria, relevant to the stage in question. These include, inter alia:
• business requirements and justification;
• governance arrangements;
• risk management;
• project management;
• stakeholder involvement and commitment;
• outcomes and operational benefits;
• procurement approach and compliance; and
• best practices including compliance with relevant legislation and guidelines.
Having considered the issues appropriate to the stage review, the PRG completes a standard report template and makes any observations which it considers appropriate, including a recommendation as to whether the project should continue, with or without conditions, to the next stage. This report is provided to the PRU for dissemination to the Sponsoring Organisation and the relevant sanctioning authority for consideration. The PRG is reconvened by the PRU when the Sponsoring Organisation has been given sanction to continue by the sanctioning authority.
The Sponsoring Organisation is required to arrange a briefing session with the PRG to make introductions and to provide a high-level overview of the project’s aims and background. The PRG will identify at this stage the documentation required to begin the formal review process.
The PRG will begin a formal review based on the documentation received and consultations with the project team and relevant parties. Meetings between the project team and the PRG, site visits etc., take place at the discretion of the PRG.
The sanctioning authority may define a reporting schedule for the Sponsoring Organisation to adhere to during the lifetime of the project. Where this exists the PRG are issued with a copy of each status report and can request a meeting with the project team if there are any concerns raised by a status report.
Reports are published on the Peer Review website (www.peerreview.gov.ie) as soon as it is feasible to due so. Publication does not normally occur until after a contract has been signed with the preferred supplier, to prevent any undue influence in the procurement process.
Each Sponsoring Organisation determines if any or all of a report is to be restricted under Freedom of Information or other regulations. Where this occurs a notice will be placed on the report publication page of the relevant project.
Business Case Review
The Business Case Review is an upfront check on the validity and feasibility of a formal business case and the adequacy of underpinning arrangements associated with the project, including the impact on stakeholders, project scope, resourcing, change management, and risk management.
The business case must be revisited throughout the life of the project (and particularly in the formal reviews) to determine that the business need is still justified and the business benefits can still be achieved. The business case documentation at the end of this review stage will, therefore, be the basis and the reference point for all subsequent reviews of a project.
The RFT Review assesses the proposed RFT documents and the proposed procurement approach having regard to national and EU procurement rules and the clarity with which the RFT sets out the business objectives, selection criteria and the evaluation approach. It also seeks to establish that the RFT constitutes a reasonable request to the marketplace and that it is conducive to delivering fixed price tenders.
Tender Evaluation Review
The Tender Evaluation Review focuses on the tender evaluation procedure, on the functionality of the preferred solution, including the appropriateness of costs and timescales. It also considers the adequacy of the arrangements to manage key risks, including contingency and fallback plans.
The Contract Review considers the proposed contract with the successful tenderer. It considers the proposed solution and its capacity to deliver business objectives, any legal advice available to the project team, the fixing of payments to stated deliverables, penalties, schedules of costs, fixed price requirements, and project plans and timescales.
Project Initiation Review
The Project Initiation Review examines the adequacy of the project initiation documentation (PID), project management controls, benefits measurement and realisation approach, risk management and the necessary resources required to begin the project development. It ensures that all necessary elements are in place to allow the project to begin.
The Review Group may request a Checkpoint Review at any point after the contract is signed. The Checkpoint Review assesses developments in the project with particular emphasis on projected timescales, costs, scope, project management and risk management. A Checkpoint Review is likely to be requested if a status report from the project team highlights any significant deviation from stated objectives, costings or schedule.
Project Completion Review
The Project Completion Report is documented by the Sponsoring Organisation and circulated to the PRG and the relevant sanctioning authority. It ensures that all the necessary steps have been completed to allow the project close gracefully. It examines any significant variation in stated objectives versus delivered objectives, forecasted versus actual expenditure and forecasted versus actual timescales.
Members of the PRG are paid according to their status. Serving civil/public servants receive vouched travel and subsistence expenses. Private sector individuals receive a flat per-diem. Retired civil/public servants receive a grade-based per-diem (based on the grade at which they retired) and any vouched travel and subsistence expenses. Any payments to retired civil/public servants are subject to pension abatement rules.
All costs are borne by the Sponsoring Organisation.
Details of the current per-diem rates can be provided by the PRU on request.