# MP5301.9001(b) Clearance – Multi-Functional Independent Review

#### Mandatory Procedure

MP5301.9001(b)
Clearance – Multi-Functional Independent Review

March 2009

1. Policy. The Clearance Approval Authority (CAA) will use Multi-functional Independent Review Teams (MIRT) as an integral component of the clearance process in AFFARS 5301.9001(a) by validating each critical decision point (CDP) as described in paragraph 3.2 of this mandatory procedure.

2. Applicability. This procedure is required for all competitive acquisitions meeting the requirements of AFFARS 5301.90 when contract values are $50M or more and where the CAA is the Senior Contracting Official (SCO), the Senior Center Contracting Official (SCCO), the Associate Deputy Assistant Secretary (Contracting) (ADAS(C)), or the Deputy Assistant Secretary (Contracting) (DAS(C)). This includes task orders for services issued against multiple award indefinite delivery/indefinite quantity contracts (to include GSA schedules). At the discretion of the CAA, this mandatory procedure may be applicable to competitive acquisitions below $50M or to non-competitive acquisitions at any dollar threshold.

3. Independent Review. The CAA will establish an independent and objective process, employing cross-functional subject matter experts (SME) to constitute a MIRT. The purpose of the MIRT is to review and assess CDPs within the business and contract clearance process. Ideally, the MIRT will operate in an advisory manner such that its inputs and recommendations will facilitate frank and candid discussions regarding the soundness of the business and contracting approaches employed in the particular acquisition, and that the results of these discussions (solutions, lessons-learned, etc.) may be shared with acquisition organizations across the Air Force.

3.1. Multi-functional Independent Review Team (MIRT). The MIRT will be formed at the beginning of each competitive acquisition with membership approved by the CAA. The CAA may use existing Independent Review Teams, peer reviews, Acquisition Center of Excellence (ACE) established review teams, or other established processes to satisfy this requirement. Note 1 To promote consistency, it is desirable that the same MIRT members participate in each critical decision point review for the duration of the acquisition unless otherwise approved by the CAA. The team will be comprised of government personnel to the maximum extent practicable.

The MIRT will have representation from:

-- Technical or Requirements Activity (e.g., engineering, program management, A1, A5, etc.)
-- Legal (e.g., other program counsel, SAF/GCQ, AFLOA/JAQ, etc.)
-- Contracting (.e.g., Contracting Officer, Chief of Policy, Technical Director/Advisor, etc.)
-- Others as applicable (e.g., finance/cost, small business, contract management (DCMA) etc.)

The members of MIRTs shall not be members of the source selection/competitive acquisition team. Note 2 Involvement of peers from other bases, MAJCOMs, or other DoD Agencies provides a broader approach to the independent review process and facilitates the sharing of lessons-learned and best practices.IG 3.1.2.1

It is desirable for members of MIRTs to have previous acquisition/procurement or source selection experience in order to provide sound, impartial perspective which in turn produces an improved procurement validation. The MIRT does not perform, but may support, the duties of a clearance reviewer which is typically performed by a procurement analyst in direct support of the CAA.

3.1.1. Independent Review Structure. The CAA will structure the independent review in a way that best achieves the objectives of the business and contract clearance process. Notably, the review procedures employed should be commensurate with the acquisition size, complexity, personnel experience, and other applicable considerations.IG 3.1.1 The process must focus on reviewing CDPs in a thorough and substantive manner where SMEs are granted the level of access to acquisition/source selection documents necessary to provide the CAA with a clear assessment of the soundness of the approach and methodology used for the acquisition. The CAA should ensure that the established review process is not compartmentalized such that individual members of the MIRT conduct reviews independently of one another (i.e. without the benefits of cross-feed).

3.1.2. MIRT Work Product. The MIRT review process will be more useful to the particular acquisition team and to the broader Air Force acquisition community if it is characterized by transparency and openness. The MIRT should generate its work product, to include any memoranda, reports, recommendations or other documentation provided to the CAA or a source selection/procurement team, understanding this objective. To this end, as a general rule, although attorneys likely will participate in preparing any written work product, this work product should not be characterized or couched as legal advice and, therefore, should not be marked or construed as being protected from release to outside sources by the attorney-client privilege.

At a minimum, the MIRT will convene an out brief with the source selection/procurement team at the conclusion of each CDP conducted. If requested by the source selection/procurement team, written documentation will be provided to the team and handled IAW FAR 2.101 and 3.104. Documentation of MIRT review(s) is required for the official contract file IAW paragraph 4. It is desirable for the MIRT to provide on-the-spot feedback to the source selection/procurement team to generate discussions in order to understand the rationale for comments or actions to be taken. The MIRT will provide an assessment to the CAA on the state of the source selection/procurement. All comments, will be dispositioned or adjudicated by the CAA, including any unresolved comments made by an ACE. The CAA may provide feedback to the SSA or the CAA may request that the source selection team provide the feedback.

3.2. Critical Decision Points (CDP). CDPs are precursor actions required in order to obtain approval to support formal clearance events (i.e. to issue the solicitation, to award without discussions, to request final proposal revisions, or for the SSA to make a source selection decision). These CDPs are conducted by the MIRT prior to obtaining the appropriate business or contract clearance from the CAA. For purposes of this MP, pre-business clearance CDPs include activities associated with acquisition planning activities as required by FAR Part 7 and applicable FAR supplements. Also, CDPs should not be considered as “prerequisite”, i.e., review of Sections L and M of the RFP could be done without having had a MIRT review of the draft Acquisition Strategy Panel (ASP) or Acquisition Plan (AP), etc.

3.2.1 Mandatory CDPs.

3.2.1.1 Pre-Business Clearance CDP.

-- Review of draft ASP Brief Note 3 or review of draft AP Note 4

-- Review of Sections L and M of the Request for Proposal (RFP)

3.2.1.2. Pre-Contract Clearance CDP.

-- Review of draft Competitive Range Brief or review of draft Award without Discussions Brief (Review of these draft briefs may include review of supporting documentation and evaluation notices or interim ratings, etc.)

-- Review of draft Final Proposal Revision (FPR) Request (Review of this draft brief may include review of pre-FPR brief including interim ratings after discussions, etc.)

-- Review of draft Source Selection Decision briefing (Review of this draft briefing may include review of pre-source selection decision brief, PAR, etc.)

3.2.1.3. MIRT Focus Areas. CDPs are representative of milestone events, which may involve the review of supporting documentation and actions leading to CDP events. IG 3.2.1.3

3.2.1.4. Areas of Special Interest (ASI). The CAA may require the MIRT to assess ASI in addition to the CDPs discussed in the previous paragraphs. The MIRT also has the discretion to review ASIs that may impact their assessment of the CDPs. ASIs may include review of source selection plans or debriefing charts/script, the conduct of mock debriefs, review of model contracts and all attachments, etc.

4. Documentation. Documentation that independent reviews were conducted IAW this procedure will be included in the contract file under the business and contract clearance tabs. Included with the documentation are the actions taken to disposition and/or adjudicate the MIRT comments by the CAA. The documentation will be considered “Source Selection Information” and/or “For Official Use only” and will be handled IAW FAR 2.101 and 3.104, respectively.

5. Waiver. The CAA may consider waiving the use of MIRTs in support of a specific clearance action for individual acquisitions or specific CDPs based on acquisition/source selection history and procurement/source selection experience of the acquisition team (i.e. recurring nature of the requirement, no history of sustained protest in a competitive acquisition, and successful experience with evaluation criteria in a competitive acquisition).

6. Annual Assessment. The CAA will provide an assessment of its multi-functional independent review process. Each MAJCOM will submit a report to SAF/AQCP (safaqcp.workflow@pentagon.af.mil) by 1 December of each year beginning in 2009. This report will include as a minimum, the following information for each acquisition of $50M or more, or when a MIRT is otherwise used:

-- Description of Acquisition, including dollar amount, period of performance/delivery dates

-- Type of source selection/competitive acquisition, (e.g., PPT, LPTA, Full-Trade-off, etc.)

-- ASIs requested by CAA (and by the MIRT, as applicable)

-- Major issues identified by the MIRT and resolutions taken

-- Assessment of value-added by MIRT

-- Other comments

********************************************

INFORMATIONAL GUIDANCE
for
MP5301.9001(b)
CLEARANCE – INDEPENDENT REVIEW

3.1.1 - MIRT Other Considerations

1. Operational Considerations

a) Identification of content of required documentation such as briefing charts, record of reviews, adjudication/disposition of review comments;
b) Notification procedures for MIRT reviews;
c) Method of feedback to source selection team, to the CAA, and if applicable to the SSA;
d) Procedures for acquisition team and the MIRT to share and exchange information;
e) Appropriate forum to conduct the MIRT reviews i.e. roundtable, telecom, video conferencing, etc; or
f) Procedures for the source selection team to provide the CAA with its response to the findings of the MIRT.

2. Teaming/Membership Considerations

a) Whether to establish standing or ad hoc MIRTs or to assign members based on unique circumstances of each acquisition. Beware of the possibility that continuous membership may lead to members losing their independence or objectivity over time and getting pulled into the “group think” scenario;
b) Whether the team should be led by one individual or co-chaired; or
c) Whether to partner less experienced personnel with senior SME to foster OJT and mentoring.

3.1.2.1 Participation Level. It is recommended that the CAA consider the participation level of the MIRTs to be based on the program and dollar thresholds as follows:

Program and/or Threshold | Participation Level
ACAT I Programs | DoD (see note below) or SAF/AQC level or Air Force equivalent 3 letter or other Air Force MAJCOM/Center
ACAT II/III Programs | MAJCOM or Center
Services and other Contracting - $1B or more | DoD (see note below) or SAF/AQC level or Air Force equivalent 3 letter or other Air Force MAJCOM/Center
Services and other Contracting – greater than $500M and less than $1B | Other Air Force MAJCOM or Other Air Force Center
Services and other Contracting not classified in programs above - $50M to $500M | Air Force MAJCOM or Air Force Center or Air Force Operational Level

Note: DoD Level is the Army, Navy, DLA, DCMA, or other Defense Agencies.

3.2.1.3 – MIRT Focus Areas. The review process may include the following focus areas:

1. Pre-Business Clearance CDP

a) Pre-ASP

Source Selection Team Membership
-- Are there appropriate, knowledgeable, and qualified source selection team members, to include a sufficient number, assigned?
-- Did the briefers effectively articulate their decisions/justifications?
ASP Brief/Draft Acquisition Plan
-- Is the requirement captured by performance-based objectives?
-- Is the requirement defined to capture the minimum program objectives and can it be realistically proposed to?
-- Are the requirements documents (including operational capability requirements) translatable into appropriate, evaluative criteria?
-- Does the market research support the requirements, acquisition strategy and recommendations?
-- Does the risk analysis include contractors’ risk as well as government’s?
-- Does the risk analysis address and support the recommendations of contract type, pricing structure, type of source selection?
-- Were other contractual vehicles considered, i.e. GSA, multiple award contracts, other DoD contracts, etc.? Was a cost/benefit analysis performed?
-- Are there special contract requirements?
-- Are potential Organizational Conflict of Interest (OCI) issues addressed?
-- Are the milestones established by focusing on the key events and with realistic dates to accommodate accomplishing the actions required of the key events? (i.e. is the source selection pacing event driven and not schedule driven?)
-- Do the milestones include review/participation by a MIRT?
-- Does the briefing support the recommendations, i.e. type of contract, pricing structure, type of source selection, etc.
-- Is the acquisition strategy executable?
-- If using ASP as an acquisition plan – are there sufficient details, either in the brief or in the notes – that the ASP can be approved as the acquisition plan?
-- If there has been any consolidation of contracts, were the bundling and consolidation analysis performed prior to deciding on an acquisition strategy? Is the basis for bundling or consolidation justified?
-- If this is a single-award IDIQ, do any of the exceptions at 10 USC 2304a(d) apply?

b) RFP Sections L and M

-- Are evaluation criteria appropriate (i.e. limited key discriminators based on risk analysis) in order to select the best value contractor?
-- Are cost considerations appropriate for strategy? If one of the criteria is to evaluate Most Probable Life Cycle Costs (MPLCC) this is a “red flag” – were lessons-learned from previous source selections where MPLCC was used discussed before recommending it as a criteria? Was required approval obtained? Was rationale and methodology thoroughly explained and executable?
-- If most probable cost (MPC) is being evaluated – is it only for the instant contract?
-- Are the criteria defined so that they are finite, measurable, and prioritized?
-- Was a side by side comparison of Sections L & M made to ensure how what is being asked for will be used in the evaluation process?
-- If a full-trade off source selection, does the evaluation identify and prioritize specific trade space or performance thresholds for which the government is willing to pay more? Is it clear from the evaluation criteria what the offeror would provide that the government would constitute more advantageous?
-- Are minimum thresholds and maximum performance objectives clearly defined?
-- Are requirements stated in certain terms such that the evaluators will be able to assess whether the offeror meets or exceeds a particular outcome?

2. Pre-Contract Clearance CDP

a) Pre-Initial Evaluation/Pre-Competitive Range (Award w/o Discussions)/Pre-release of Interim Ratings

Evaluation
-- Did the evaluation team follow/comply with:

--- Sections L&M of the RFP?
--- Source Selection Evaluation Guide?
--- Source Selection Plan?

-- Was there consistency and fairness in evaluating each offer against Section M (evaluation criteria)
-- Was there consistency in applying the ratings across the offerors?
-- What clarifications and/or communications, if any, were conducted?

--- Were they appropriate clarifications and/or communications?
--- Were meaningful and open communications conducted in which the offeror and government clearly understood each other’s position and assumptions? (It is not sufficient to handle discussions in a way that is it only for the benefit of the government to understand the offeror’s proposal.)

-- Did the evaluation team apply any standards or undisclosed evaluation criteria to their evaluation of proposals or past performance?
Documentation
-- At a minimum, were deficiencies, weaknesses, significant weaknesses, and adverse past performance described in the evaluation notices? Were assumptions addressed as appropriate or acceptable?
-- Is supporting documentation describing details of the evaluations readily available during the briefing?
Briefing Charts
-- Does the briefing tell the story of proper evaluations from receipt of proposals, through the technical, past performance and cost evaluations, through the type of clarifications and communications conducted, through formations of evaluation notices, to the initial ratings/standings?
-- Does the briefing show the evolution of discussions with each offeror?
-- Were the interim ratings reflective of achieving meaningful discussions?
-- Does the content and quality of the briefing reflect Contracting officer and SSET Chairperson(s) or equivalent review?
-- Does the briefing support the recommended competitive range (or to award without discussions)?
-- Did the briefers effectively articulate their decisions/justifications?

b) Pre-Final Proposal Revision Request Brief

Discussions
-- Were discussions conducted IAW sections L & M?
-- Did the discussion process utilized achieve meaningful and transparent communications?
Documentation
-- Does the supporting documentation adequately describe the basis and justification for the ratings?
-- Do the Interim Ratings prior to final evaluation support the degree of discussions held with each offeror? (e.g. there is no misunderstanding by the offeror and GAO that the Contracting Officer clearly discussed the issues that resulted in the red or yellow ratings.)
Briefing Charts
-- Does the content of the briefing bring the SSA up to date from approval to establish competitive range to completion of meaningful discussions (with or without releasing of Interim Ratings)?
-- Does the briefing content confirm that meaningful discussions were conducted?
-- If interim ratings were not released prior to Final Proposal Revision request, has the Contracting Officer clearly articulated his/her rationale in the briefing charts?
-- If any offeror’s proposal should no longer be in the competitive range, does the briefing support the recommendation to eliminate it from consideration for award?
-- Did the briefers effectively articulate their decisions/justifications?

c) Pre-Source Selection Decision Brief

Documentation
-- Did the SSET complete proposal evaluation, incorporate information provided through discussions and Final Proposal Revisions IAW section M?
-- Is the Proposal Analysis Report (PAR) reflective of the evaluations and justification documents?
-- Is the Source Selection Decision Document reflective of the SSA’s integrated assessment and own personal decision leading to the selection of a contractor(s)?
Briefing Charts
-- Do the briefing charts clearly summarize and justify the evaluation results?
-- If Proposal Analysis Report is not used, does the briefing contain adequate details of the evaluation results and a comparative analysis (cost/price, past performance, mission capability, proposal risk, and a source selection recommendation) of the competitive offers?
-- Can the SSA make an integrated assessment best value decision based on the information presented in the briefing?
-- Can the information in the briefing be used, as applicable, for the source selection decision document?
-- Did the briefers effectively articulate their decisions/justifications?

3. Areas of Special Interest

a) Pre-Debriefs

-- If Sections L&M were complied with; if meaningful discussions were conducted with the offeror; and if offeror aware of its standings (via initial & interim ratings) throughout the evaluation process, the majority of the debriefing will focus on the final evaluation.
-- The content of the debriefing slides should be taken from those used for the offeror during the competitive range, interim ratings, and final proposal revision requests briefings. (The offeror should not be surprised with the content of this information during a debriefing.)
-- Purpose is to put the “debriefing” team in a mock situation to be able to handle offeror’s questions and comments.
-- Can the members of the debriefing team effectively articulate their findings and their justifications?
-- Does the debriefing focus on the government’s evaluation of the offeror’s proposals instead of the selected winner?
-- Will the debriefing assist/instruct the offeror on what it did well and what it could have done better?

********************************************

Note 1: Personnel participating as members of or advisors to source selection teams or procurement teams may not be an acceptable substitute for the independent review due to their lack of objectivity and independence. However, the CAA has the discretion to use “advisors” as members of a MIRT.

Note 2: See Note 1.

Note 3: In the event the contents of ASP do not align with FAR Part 7 and its supplements or an ASP is not held, review of written AP or equivalent documentation will be conducted.

Note 4: Applicable to Life Cycle Management Plan (LCMP), Commodity Acquisition Management Plan (CAMP), Integrated Program Summary (IPS) and other similar documentation.