1870.301 Purpose.
The acquisition of goods and services is among the most important activities that NASA performs and demands the agency's best management efforts. Therefore, the source evaluation and selection procedures must emphasize the application of sound judgment to the problems of source evaluation and assure that Source Evaluation Boards (SEB) conduct their activities impartially and efficiently in ways which will effectively accomplish the source evaluation task.
1870.302 Regulations.
The basic regulations governing source selection using the SEB process appear at 1815.613-71. Detailed operating instructions and procedures for the evaluation and negotiation of procurements by Source Evaluation Boards appear at 1870.303. These instructions provide general and specific policies and procedures for SEB's evaluating offerors' proposals and related capabilities. They are designed for use by procurement personnel and by the wide range of individuals who participate in the SEB process. The instructions will also be of help to the public in understanding NASA's source selection policies and procedures.
1870.303 Source Evaluation Board Procedures.
(a) SEB procedures are prescribed by Appendix I to this section 1870.303.
(b) NASA may reprint Appendix I as a separate document, which may be referred to as the NASA Source Evaluation Board Handbook, for sale and/or distribution provided the following conditions are met:
(1) The issuance date ("cover date") of the Handbook shall be the date of the NASA FAR Supplement version from which the text is extracted.
(2) With the exception of availability, distribution and other special prefatory notices, any subsequent modification in the text shall be preceded by a change to the NASA FAR Supplement 1870.303, Appendix I.
(3) The following notice shall be included in the prefatory material of the Handbook:
This Handbook is a separately bound, verbatim version of NASA FAR Supplement (NFS) (48 CFR 1870.303) Section 1870.303, Appendix I. Reference to other parts of the Federal Acquisition Regulation (FAR) and the NFS will be required for complete coverage of all procurement aspects. NASA reserves the right to make changes to NFS 1870.303, Appendix I, without issuing a new edition of this Handbook. Any such changes will be published in the Federal Register; however, it is anticipated that such changes will be rare, unless mandated by statute or unusual circumstances. In the event of apparent conflict between this Handbook and the NFS, the NFS shall govern.
The acquisition of goods and services is among the most important activities that NASA performs and demands our best management efforts. The acquisition system we use must be such that individuals performing within it are challenged to high standards of performance because they know that their efforts contribute to and form part of the Government's decision-making process.
The source evaluation and selection process covered by this handbook exemplifies our efforts to emphasize the application of sound judgment to the problems of source evaluation. In addition, the handbook emphasizes the responsibility which line and staff management retains to assure that Source Evaluation Boards (SEB's) conduct their activities impartially and efficiently in ways which will effectively accomplish the source evaluation task.
The process provides for an equitable and comprehensive evaluation of offerors' proposals to assist the Source Selection Official (SSO) in selecting the source(s) whose proposal(s) presents the highest probability of quality performance to best meet NASA's requirements at a realistic cost/price. This handbook provides guidance and general and specific policies and procedures for SEB's evaluating offerors' proposals and related capabilities in negotiated procurements. Its intention is to encourage the exercise of sound judgment in the many important aspects of the process. SEB's are expected to apply common sense in determining appropriate variations and adaptations necessary in individual situations, provided that these do not constitute a departure from basic concepts and intent. Substantive deviations may be authorized only by the Associate Administrator for Procurement.
While this handbook is intended primarily for NASA use, it is also available to the public so that NASA's source selection policies and procedures can be understood by all participants in the process.
The provisions of this handbook are applicable to all elements of NASA and are to be implemented on any Request for Proposal (RFP) subject to evaluation by an SEB and issued on or after October 1, 1988. The source selection policies and procedures set forth in this handbook are to be implemented in accordance with the Federal Acquisition Regulation (FAR) 15.6 and NASA/FAR Supplement (NFS) 1815.613. However, the procedures in this handbook may be used in any other competitively negotiated procurement where the SSO determines it desirable to do so.
CHAPTER 1 KEY PARTICIPANTS IN THE SOURCE EVALUATION
BOARD PROCESS
100 Introduction
101 Cognizant Management Responsibilities
102 Source Selection Official
103 Source Evaluation Board
CHAPTER 2 MEMBERSHIP, ORGANIZATION, AND RESPONSIBILITIES
200 General
201 Membership
202 Organization
203 Responsibilities
CHAPTER 3 EVALUATION FACTORS, SUBFACTORS, AND ELEMENTS
300 General
301 Mission Suitability
302 Cost
303 Relevant Experience and Past Performance
304 Other Considerations
305 Continuing Evaluative Responsibility of SEB
CHAPTER 4 SEB OPERATING PROCEDURES FOR SOLICITATION AND
EVALUATION
400 General
401 Initial SEB Activities
402 Evaluation Plan
403 Qualification Standards
404 Request for Proposals (RFP's)--Review and Approval
405 Preproposal Conference
406 Initial Evaluation
407 Final Evaluation
CHAPTER 5 SOURCE EVALUATION BOARD REPORT AND PRESENTATION
500 General
501 Responsibilities and Procedures
CHAPTER 6 SOURCE SELECTION
600 General
601 Source Selection Decision
602 Notice and Debriefing for Unsuccessful Offerors
603 Source Selection Statement
604 Multiple Selection Decisions
APPENDIX A SAMPLE LETTER OF DESIGNATION
APPENDIX B INDIVIDUAL CERTIFICATE FOR SOURCE EVALUATION
BOARD PARTICIPANTS
APPENDIX C GUIDELINES FOR THE SEB REPORT AND PRESENTATION
TO THE SOURCE SELECTION OFFICIAL
100 INTRODUCTION
This Chapter describes the role of key participants in the Source Evaluation Board, (SEB) process, including cognizant line and staff management, the Source Evaluation Board and the Source Selection Official.
101 COGNIZANT MANAGEMENT
RESPONSIBILITIES
1. The SEB process described in this handbook shall be used for major procurements as defined by NFS 1815.613-71(a). The process is used to accomplish and document the source evaluation and selection function.
2. Upon request of the cognizant Installation Procurement Officer, as a part of the Master Buy Plan procedure, the Associate Administrator for Procurement or an appropriate designee may consider streamlining these source selection procedures for a particular procurement. This decision, granting a deviation to the formal SEB process, will be communicated to the cognizant Procurement Officer as a part of the written response to the installation regarding the Master Buy Plan submission. Requests for use of streamlined procedures may also be submitted independent of the Master Buy Plan procedure.
3. When the SEB process is determined to be applicable in accordance with the NFS, the specific SEB procedures detailed in this handbook are employed to--
a. Prepare a solicitation that accurately conveys to offerors the technical, schedule, cost, and other contractual requirements of the procurement;
b. Ensure equitable and comprehensive evaluation of the competitors' proposals;
c. Ensure fairness and freedom from outside influence throughout the process;
d. Ensure selection of the source(s) whose performance can be expected to be most advantageous to the Government with respect to the Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations factors considered; and
e. Protect the business confidential and proprietary information contained in proposals submitted to the Government for evaluation.
4. Appropriate cognizant line and staff management shall--
a. Establish each SEB with fully qualified Government personnel possessing broad experience and the professional skills and knowledge required for proper evaluation and assessment of offerors' proposals;
b. Ensure all personnel assigned to the SEB are unencumbered by other assignments which compete with SEB activities and, further, ensure that personnel assigned to the SEB are officially appointed to the activity for the duration of the SEB;
c. Ensure the SEB is provided all current NASA policies and procedures relevant to SEB operation;
d. Ensure acquisition strategy and planning objectives are achieved as reflected in the acquisition's requirements;
e. Ensure the SEB works in harmony with the needs and objectives of the requiring activity;
f. Concur in the substance and weight of evaluation factors, subfactors, and elements;
g. Ensure the RFP is complete, clear, and consistent with agency objectives and with the needs of the activity requiring the procurement, that the procedures for evaluation and selection are clearly set forth, and that the offeror is not burdened with unnecessary requests for data not pertinent to source selection;
h. Ensure appropriate actions are taken, consistent with the FAR and NFS, to obtain full and open competition in the selection process, or to obtain appropriate approvals for exceptions;
i. Establish an SEB advisory group or individual at the field installation to ensure proper source selection procedures are employed;
j. Ensure an environment exists in which evaluation and selection activities can be effectively conducted; and
k. Ensure that the activities of the SEB are carefully planned, all key events identified and scheduled, and progress monitored. Normally, the goal of all SEBs should be to complete activities from receipt of proposal to contract award in not more than 120 calendar days. Realization of this goal can be furthered through the synergism of acquisition streamlining techniques such as RFP and proposal page limitations, reduced SEB memberships, and limitation of evaluation sub-factors and elements to key discriminators. However, there may be instances when this goal cannot be attained even when these streamlining techniques are used, for example, in procurements of unusual complexity or when a large number of proposals is anticipated. In these cases, a schedule longer than the 120-day goal should be planned, consistent with the exercise of good judgment.
5. For purposes of this handbook, in cases where the SSO is at the Headquarters level, "cognizant line and staff management, "as addressed in subparagraph 4, includes--
a. The NASA Acquisition Executive for systems designated as major acquisitions under NMI 7100.14, "Major System Acquisitions";
b. Officials-in-Charge of the cognizant Headquarters Program and Headquarters Offices or their Deputies.
(See NMI 1101.2, "NASA Organization and Definition of Terms.");
c. The cognizant Field Installation Director and Deputy/Associate Director;
d. The Field Installation Project Manager;
e. The Associate Administrator for Procurement;
f. The General Counsel and/or Associate General Counsel (Contracts);
g. The Associate Administrator for Safety, Reliability, Maintainability, and Quality Assurance (SRM&QA);
h. The Field Installation Principal Official-in-Charge of Administration, when such a position exists;
i. The Field Installation Chief Counsel;
j. The Field Installation Procurement Officer.
6. When the Administrator or an Official-in-Charge of a Headquarters Office is the SSO, selected cognizant line and staff management, including the General Counsel or designee and the Associate Administrator for Procurement, will advise the SSO in executive session, at the conclusion of the presentation of the SEB report, regarding their views concerning the SEB's findings. Management personnel are encouraged to seek the advice of the General Counsel, the Associate Administrator for Procurement, and any other responsible Headquarters official regarding any SEB-level procurement problem where their participation would be helpful in the conduct of the SEB process.
102 SOURCE SELECTION OFFICIAL
1. The SSO is the senior agency official responsible on a particular procurement for proper and efficient conduct of the source selection process and for making the final source selection decision. It is the SSO's responsibility to decide which of the proposals submitted in response to the solicitation would prove most advantageous to the Government, all RFP evaluation factors considered. The decision must reflect the SSO's determination of relative quality and suitability of what is proposed by each offeror in light of the Government's stated requirements and the confidence level associated with the offeror's ability to accomplish what is proposed. This includes an assessment of the probable cost of each proposal in the competitive range. Trade-off judgments may be required among the evaluation factors (Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations). For procurements designated as Headquarters selections, the SSO will be identified as a part of the Master Buy Plan process. For field installation selections, the Field Installation Director will serve as the SSO. When source selection official authority is at the installation level, either by delegation from Headquarters or by virtue of a procurement value under the installation Master Buy Plan limitation, the head of the installation may further delegate this authority to the lowest reasonable level.
2. The SSO will--
a. Approve the substance and weight of evaluation factors, subfactors, and elements prior to release of the RFP, or delegate this responsibility to appropriate management personnel;
b. Appoint the SEB Chairperson and members, both voting and nonvoting, except when the Administrator will serve as the SSO, in which case the Official-in-Charge of the cognizant Headquarters Program Office will appoint the SEB Chairperson and members;
c. Provide the SEB with appropriate guidance and special instructions to conduct the evaluation and selection procedures; and
d. Make the final selection decision. Selection for final negotiations will be based on an integrated assessment of each offeror's proposal, taking into consideration the SEB's report and advice from senior officials.
103 SOURCE EVALUATION BOARD
1. The solicitation, receipt, and evaluation of proposals will be implemented by the SEB in accordance with the procedures outlined in this handbook. It is paramount that the SEB's evaluation not be influenced by persons outside the SEB process, whether or not such persons are NASA employees. The SEB is established with special status and shall be staffed with competent people fully qualified to identify the strengths, weaknesses, and risks associated with proposals submitted in response to the Government solicitation. The SEB assists the SSO in decision making by providing expert analyses of the offerors' proposals in relation to the evaluation factors, subfactors, and elements contained in the solicitation.
2. In particular, the SEB shall--
a. Conduct an in-depth review and evaluation of each proposal against the solicitation requirements and the approved evaluation factors, subfactors, and elements;
b. Provide an integrated assessment of each offeror's probable performance relative to the evaluation factors, subfactors, and elements contained in the solicitation related to Mission Suitability;
c. Perform cost analysis by reviewing and analyzing each offeror's proposed costs by individual cost element to determine the validity and reasonableness of the proposed costs. Perform price analysis by comparing each offeror's proposed price (when it is feasible to do so) against the independent Government cost estimate and/or prior experience and knowledge regarding similar efforts to assist in determination of a fair and reasonable price. Develop a probable cost for each proposal within the competitive range;
d. Evaluate the relevant experience and past performance of each offeror, including the relevant experience and past performance of proposed subcontractors intended to perform a major role in accomplishment of procurement objectives or to participate in a substantive manner;
e. Evaluate proposals under the Other Considerations factor included in the RFP and present the analysis of that factor as reflected in each offeror's proposal;
f. Ensure proper SEB procedures are implemented, including those necessary to prevent disclosure of source selection data;
g. Prepare and present the SEB report which accurately and clearly reflects the findings of the SEB.
NOTE: The SEB is not to make recommendations for selection to the SSO. The SEB reports its findings, avoiding trade-off judgments among either the individual offerors or among the Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations evaluation factors.
200 GENERAL
This Chapter sets forth NASA policy on membership, organization, and responsibilities of SEB's.
201 MEMBERSHIP
1. Composition
a. SEB's shall be comprised of qualified management, technical, scientific, contracting, and business experts including, where appropriate, SRM&QA representatives. Each SEB shall have a legal advisor.
b. While in general the SEB Chairperson, SEB members, and committee members are drawn from the cognizant installation, personnel from other NASA installations or other Government agencies may be used when their services are required in a particular area of expertise and would significantly contribute to the evaluation of proposals.
c. It is NASA policy to have proposals evaluated by the most competent technical and management sources available in NASA and the Jet Propulsion Laboratory (JPL). All personnel participating in evaluation proceedings shall be instructed to observe the restrictions in the FAR and NFS regarding personal conflicts of interest and the disclosure of information concerning the evaluation. Non-NASA personnel shall not serve as voting members of a NASA SEB.
d. In evaluating a proposal, the SEB may find that it is necessary to disclose the proposal (in whole or in part) outside the Government to meet its evaluation needs. When it is clearly necessary to disclose the proposal (in whole or in part) outside the Government, arrangements shall be made in accordance with NFS 1815.413-2.
e. It is desirable that voting members of the SEB include people who will have key assignments on the project to which the procurement is directed. However, it is important that this should be tempered to ensure objectivity and to avoid an improper balance. It may even be appropriate to designate a management official from outside the project as SEB Chairperson.
f. SEB membership shall be limited to a maximum of 7 voting individuals. Wherever feasible, an assignment to SEB membership as a voting member shall be on a full-time basis. Where this is not feasible, SEB membership shall take precedence over other duties.
g. If additional support is needed, committees and panels may be used to assist in the evaluation. The total of all such evaluators (committees, panels, etc., excluding SEB voting members and ex officio members) shall be limited to a maximum of 20 individuals, unless approved in writing by the Procurement Officer.
h. The number of nonvoting ex officio (advisory) members shall be kept as small as the nature of the procurement allows. Ex officio members should be selected for the experience and expertise they can provide to the SEB. Since their advisory role may require access to highly sensitive SEB material and findings, ex officio membership for persons other than those identified in paragraph 3.b. should be carefully considered.
2. Designation
a. Designation of the SEB Chairperson and members, both voting and nonvoting, shall be by the SSO except when the Administrator will serve as the SSO, in which case the designation shall be by the Official-in-Charge of the cognizant Headquarters Program Office. The designation letter will be addressed to the cognizant Field Installation Director when the SSO is at the Headquarters level or to the SEB Chairperson when the SSO is the Field Installation Director. The letter shall be prepared in accordance with the sample letter in Appendix A and forwarded for signature to the SSO or the cognizant Official-in-Charge of the Headquarters Program Office early in the procurement cycle, but not later than coordination and approval of the procurement plan. Prior to signature, concurrence by the Headquarters Office of Procurement (Code HS) is required when the SSO is at the Headquarters level or by the cognizant Field Installation Procurement Officer when the SSO is the Field Installation Director.
b. When the Administrator or the Official-in-Charge of the cognizant Headquarters Program Office is the SSO, resumes of voting members shall be provided along with the proposed designation letter. These resumes shall include functional title, grade level and related SEB experience.
3. Voting and nonvoting Members
a. The following people shall be on all SEB's and designated as voting members:
(1) Chairperson.
(2) A senior, key technical representative for the project.
(3) An experienced procurement representative.
(4) A senior SRM&QA representative, as appropriate.
(5) Committee chairpersons (except where this imposes an undue workload).
All voting members of the SEB shall have equal status as rating officials.
b. The following people shall be on all SEB's as nonvoting ex officio members:
(1) Associate Administrator for Procurement and designee from the Program Operations Division (Code HS) when either an Official-in-Charge of a Headquarters Program Office or the Administrator is the SSO.
(2) General Counsel and/or the Associate General Counsel (Contracts) when either an Official-in-Charge of a Headquarters Program Office or the Administrator is the SSO.
(3) Associate Administrator for Safety, Reliability, Maintainability, and Quality Assurance or designee when either an Official-in-Charge of a Headquarters Program Office or the Administrator is the SSO.
(4) Director of the cognizant Field Installation or designee when either an Official-in-Charge of a Headquarters Program Office or the Administrator is the SSO.
(5) Cognizant Program Director or designee when an Official-in-Charge of a Headquarters Program Office is the SSO.
(6) Official-in-Charge of the cognizant Headquarters Program Office or a designee when the Administrator is the SSO.
(7) Chairpersons of SEB committees when they are non-voting members.
(8) The Procurement Officer of the installation, unless designated a voting member.
(9) The contracting officer responsible for the procurement, unless designated a voting member.
(10) The Chief Counsel and/or designee of the installation.
(11) The SEB recorder.
c. The Associate Administrator for Procurement, the Code HS Director and cognizant procurement analyst, and the Associate General Counsel (Contracts) have access to all SEB activity by virtue of their positions in the agency.
d. The Associate General Counsel (Contracts) will assign legal advisors to Headquarters SEB's. The installation Chief Counsels will assign legal advisors to SEB's at their installations. The assigned advisor will provide legal advice and counsel on all matters pertaining to the procurement. Upon appointment of the SEB, the Associate General Counsel (Contracts) or designee, or Chief Counsel or designee, as appropriate, will brief SEB personnel on required standards of conduct.
e. Nonvoting ex officio members may state their views and contribute to the discussions in SEB deliberations, but they may not participate in the actual rating process. However, the SEB recorder should be present during rating sessions. The Chairperson is responsible for determining appropriate attendance at SEB meetings and is encouraged to seek the advice and counsel of nonvoting ex officio members whenever necessary without convening a special meeting.
202 ORGANIZATION
The organization of an SEB is tailored to the requirements of the particular procurement. This can range from the simplest situation, where the SEB conducts the evaluation and fact-finding without the use of committees or panels, to a highly complex situation involving a major acquisition where two or more committees are formed and these, in turn, are assisted by special panels in particular areas. Appropriate organization somewhere between these two extremes is generally expected but in all cases the number of committees or panels should be kept to a minimum consistent with the requirements of the procurement.
203 RESPONSIBILITIES
1. The SEB is the central group in the source evaluation process. Its function is to carry out the activities set forth in this handbook, culminating in final evaluation of all proposals and its report to the SSO. The SEB's evaluation is based on all available information, including proposals, committee and panel reports, written and/or oral discussions, personal knowledge of the members in the area of experience and past performance, and other appropriate reference checks. Subject to reviews by the SSO and cognizant management personnel as may be required, the SEB establishes evaluation factors, subfactors, elements and their relative importance; generates qualification standards (where applicable); and reviews and approves the RFP prior to issuance. The SEB reviews offeror satisfaction of applicable qualification standards, if any, and makes an identification of unacceptable proposals. It provides an initial evaluation of acceptable proposals sufficient for determination of the competitive range and participates in written and/or oral discussions held with all offerors in the competitive range. The SEB shall not delegate its evaluation responsibility in whole or in part. Findings of committees or panels must be reviewed by the SEB, with its own collective judgment being applied in arriving at the SEB evaluation findings reported to the SSO.
2. The SEB Chairperson is the principal operating executive of the SEB. This carries with it a responsibility broader in scope and including more requirements for coordination across more different specialized disciplines and through more diverse management channels than is expected in most management situations. The Chairperson is expected to manage the team efficiently without compromising the validity of the findings provided to the SSO as the basis for a sound selection decision.
3. The SEB Recorder functions as the principal administrative assistant to the SEB Chairperson. The duties and responsibilities of the position are as follows:
a. Attends all SEB meetings and serves as principal assistant to the SEB Chairperson.
b. Obtains secure work areas for conduct of SEB activity and develops and implements procedures for controlling access and safeguarding SEB proceedings and documentation.
c. Obtains materials, supplies, and equipment needed by the SEB.
d. Arranges for preparation, reproduction, control, and distribution of material relating to the activity of the SEB and its committees.
e. Prepares and distributes the agenda for SEB meetings.
f. Obtains and distributes current applicable procedures, policies, and instructions to the SEB and committee members and others involved.
g. Records the substantive issues discussed.
h. Follows up on action items assigned to SEB members to ensure no delays in the SEB schedule will occur.
i. Obtains the SEB Chairperson's approval of SEB minutes. Provides copies to all voting SEB members and/or nonvoting members, as directed by the SEB Chairperson. Retains the original copy of the minutes and incorporates them into an official record book.
j. Assists in preparation and assembly of the SEB's report of findings and presentation charts and arranges for reproduction and distribution.
k. Destroys all duplicate material in excess of the SEB's need or retains material, as defined by the contracting officer, the SEB, and/or the SSO.
l. After formal selection announcement, accumulates, packages, and forwards documentation pertinent to the SEB's work to the cognizant contracting officer for retention throughout the life of the contract.
m. At the conclusion of all SEB activity, surveys the area where SEB activity occurred and arranges for the return of equipment and materials, as appropriate.
4. An SEB Committee functions as a fact-finding arm of the SEB, usually in a broad grouping of related disciplines (e.g., technical or management). It is comprised of people well-versed and experienced in each of the major disciplines under its aegis. For example, a "Management Committee" could include experts in such areas as organization, pricing, personnel, labor, contracting, and facilities operation. The committee examines in detail each proposal, or portion thereof, assigned by the SEB. It evaluates such proposals or excerpts in accordance with the approved evaluation factors, subfactors, and elements and submits a written report to the SEB summarizing its evaluation. The committee will also respond to requirements assigned by the SEB, including further justification or reconsideration of its findings.
5. Committee chairpersons shall, with respect to their committees, exercise the same responsibility for administrative and procedural matters as does the SEB Chairperson for the SEB.
6. An SEB panel functions as a fact-finding arm of the committee in a specialized area of the committee's responsibilities. Panels are established when a particular area requires deeper analysis than can be provided by individual members of the committee.
7. a. All personnel involved in SEB activities are responsible for complying with the requirements of this handbook and other applicable regulations. Accordingly, they are advised to seek counsel and guidance from appropriate personnel, such as; the SEB Chairperson, the SEB procurement voting member, the contracting officer and/or the SEB legal advisor as well as any cognizant installation SEB advisory group with respect to any questions concerning compliance with these requirements. The Chairperson shall require each SEB, committee, and panel member to be familiar with the provisions of NHB 1900.1, "Standards of Conduct for NASA Employees," regarding conflict of interest and to inform the Chairperson in writing if, in the opinion of the member, their participation presents a real or apparent conflict of interest. In addition, the SEB Chairperson shall ensure that each SEB, committee, and panel member has filed a completed NASA Form 1270, "Confidential Statement of Employment and Financial Interests." Code HS procurement analysts will comply with the necessary disclosure requirements and will sign the necessary certificates on an annual basis to cover all SEB activities that the analyst may participate in throughout the year. Completed NASA Form 1270's from Code HS personnel will be generated and reviewed at the Headquarters level and will remain on file in the Office of General Counsel.
b. Prior to selection and announcement of an offeror for final negotiations, NASA personnel shall not reveal any information concerning the evaluation to anyone not participating in the same evaluation proceedings and then only to the extent such information is required by such proceedings and approved by the SSO or designee.
c. Subsequent to selection and announcement of an offeror for final negotiations, information concerning SEB proceedings and data will be made available to others within NASA only when the requester demonstrates a need to know for a NASA purpose. Information will be made available to persons outside NASA, including other Government agencies, only when such disclosure is concurred in by the Office of General Counsel. Title 18, U.S. Code, Section 1905, prohibits any officer or employee of the United States from disclosing or divulging certain kinds of business confidential and trade secret information unless authorized by law. Possible penalties upon conviction include imprisonment of up to 1 year, a fine of not more than $1,000, and removal from the Federal Service.
300 GENERAL
1. This Chapter describes the factors, subfactors, and elements used in evaluation of proposals and discusses the manner in which the SEB will develop, describe, and structure such factors, subfactors, and elements.
2. The SEB's responsibility is to provide analysis of proposals to aid the SSO in selecting the offeror(s) who best meets the Government's requirements at a reasonable cost.
3. In making a selection, the SSO normally considers four evaluation factors: a. Mission Suitability (which reflects how well the offerors can be expected to perform the work from a technical and management perspective); b. Cost (which reflects what it will probably cost the Government to do business with the offerors); c. Relevant Experience and Past Performance (which reflects the amount and quality of previous work accomplished by the offerors comparable to the work to be performed under the procurement being evaluated); and d. Other Considerations (which are those considerations, stated in the RFP, other than Mission Suitability, Cost, and Relevant Experience and Past Performance that can affect contract performance).
4. Performing the proposed work properly (Mission Suitability) is always important, and so is the probable cost of performing that work. Probable cost is not necessarily the offeror's estimate of costs; rather, it is the SEB's assessment of what the proposed work is likely to cost. Depending on circumstances, relevant experience and past performance or other considerations may or may not be of prime importance.
5. The SEB evaluates proposals with respect to the four evaluation factors, as follows:
a. Mission Suitability. This factor indicates, for each offeror, the merit or excellence of the work to be performed or product to be delivered. It includes, as appropriate, both technical and management subfactors. Because this factor can be highly technical and must be integrated in order to convey an overall evaluation of relative merit, Mission Suitability and its supporting subfactors shall be numerically weighted and scored. Elements may or may not be numerically weighted and scored.
b. Cost. This factor evaluates what each offeror's proposal will probably cost the Government should it be selected for negotiations leading to award. Proposed costs are analyzed to determine the probable "cost of doing business" based upon the offeror's proposed approach. Further, this analysis identifies and assesses the impact of features that cause a proposal to cost more or less than other proposals. (See paragraph 302 for detailed coverage.) Cost is not numerically weighted or scored.
c. Relevant Experience and Past Performance. This factor indicates the relevant quantitative and qualitative aspects of each offeror's record of performing services or delivering products similar in size, content, and complexity to the requirements of the instant procurement. The Relevant Experience and Past Performance factor provides an opportunity to evaluate the quality of goods and services provided by the offeror(s) to the agency and other Government organizations. The agency has acquired a substantial amount of firsthand experience and past performance data over a variety of program and contract efforts. Evaluation of this factor also utilizes relevant experience and past performance data from programs acquired by other Governmental organizations, covering both prime and subcontractor performance. The Relevant Experience and Past Performance factor is not numerically weighted or scored.
d. Other Considerations. This factor includes those considerations other than Mission Suitability, Cost, and Relevant Experience and Past Performance. They include, but are not limited to, such items as: financial condition, labor relations considerations, small and small disadvantaged business considerations, and geographic distribution of subcontracts. Other Considerations is not numerically weighted or scored.
6. Mission Suitability is the only factor which is numerically weighted (normally 1000 points) and scored. Further, each Mission Suitability subfactor is numerically weighted and scored. The sum of the subfactor weights will total the weight of the Mission Suitability factor. A subfactor may, at the discretion of the SEB, be divided into discrete elements which, in total, comprise the subfactor evaluation area. If deemed conducive to the evaluation process, the SEB may weigh and score individual elements of a subfactor. The sum of the weights of the individual elements will equal the weight of the subfactor.
7. The general format for inclusion of the factors, subfactors, and elements in the RFP are:
a. Factor (numerically scored)-- Mission Suitability
Grouping/Category (not numerically scored)--i.e., Technical or Management (optional)
(1) Subfactor (numerically scored)
(2) Subfactor (numerically scored)
(3) Subfactor (numerically scored)
Element (optional). May or may not be numerically scored.
b. Factor (not numerically scored)--Cost
c. Factor (not numerically scored)--Relevant Experience and Past Performance
d. Factor (not numerically scored)--Other Considerations
301 MISSION SUITABILITY
1. Evaluation subfactors
a. Evaluation subfactors are the weighted areas within the Mission Suitability factor that further identify, for proposal preparation and evaluation purposes, the content of the factor. Examples of Mission Suitability subfactors found by experience to be relevant to many procurements are: Understanding of the Requirement; Management Plan; Key Personnel; Corporate or Company Resources; and Excellence of Proposed Design for hardware procurements. However, citation of these specific subfactors is not intended to be restrictive or all inclusive. The nature and thrust of the requirements and objectives of the procurement may logically call for the use of some subfactors titled and described in a somewhat different manner than those described below:
(1) Understanding of the Requirement. An offeror's proposal reflects how well the offeror comprehends the work and the data requirements. The offeror's proposal should be examined and analyzed to evaluate the offeror's understanding of the requirements set forth in the RFP. Understanding of the requirement can be evaluated as a separate subfactor or can be evaluated as an element to be considered as a part of the evaluation of each subfactor. Although costs are analyzed separately from Mission Suitability, they may be significant in indicating an offeror's understanding of the resources, human and material, required for performance of the contract. Accordingly, technical personnel assigned to committees or panels, in evaluation of the Mission Suitability factor pursuant to the weighted subfactors and elements, may be given access to the cost proposals or portions of the cost proposals to help determine the offeror's understanding of the requirements of the RFP. Such cost information may also help them to assess the validity of the offeror's approach to performing the work in accordance with the requirements. Cost realism, or the lack thereof, should enter into the SEB's assessment of the measure of understanding possessed by each offeror. Normally this would entail a consideration of cost realism in the evaluation of all subfactors and elements wherein understanding is an essential concept. Similarly, an offeror's justification or rationale for proposed costs can give insight into how well the work to be performed is understood.
(2) Management Plan
(a) The offeror's management plan sets forth the offeror's approach for efficiently managing the work as demonstrated by the proposed organization, the recognition of essential management functions, and the effective overall integration of these functions.
(b) The management plan describes the project organization proposed for the work, including internal operations and lines of authority, together with external interfaces and relationships with the Government, major subcontractors, and associate contractors. When properly prepared, the authority of the project manager, the project manager's relationship to the next echelon of management, and the project manager's command of company resources can be ascertained from the management plan. Likewise, the management plan provides schedules necessary for the logical and timely pursuit of the work, accompanied by a description of the offeror's work plan.
(3) Excellence of Proposed Design. In hardware acquisition, design of the product is generally a major aspect of competition. In order to arrive at an informed judgment, the SSO may require the SEB's evaluation of the merits of competing designs in relationship to the stated requirement. In evaluating the proposed designs, the SEB should consider the resources required to perform the work inherent in the differing designs. Evaluation of design may extend to whatever subsystem level is deemed appropriate by the SEB and may include producibility, reliability, maintainability, and, as applicable, warranties.
(4) Key Personnel
(a) Thorough evaluation of proposed key personnel is usually one of the most vital aspects of SEB activity.
(b) Experience demonstrates that the qualifications and performance of a few people--the top half-dozen or so directly involved managers--are extremely important to successful accomplishment of a contract. For evaluation purposes, the SEB may designate a separate subfactor entitled "Key Personnel." The SEB may define the number and identity of the key personnel for each offeror in the solicitation or may, subject to such limitations as the SEB deems appropriate to assure a reasonable basis for comparison, permit each offeror to define its own key personnel consistent with its proposed organization.
(c) Written resumes should be the baseline from which the evaluation of key personnel begins. Personal reference checks with people knowledgeable of an individual's training, experience, and performance constitute part of this baseline; these should be made at levels commensurate with the role of the individual in the program or project involved. However, the written and/or oral discussions, if conducted, will include all offerors in the competitive range and may be used to establish the relative merits of personnel proposed by each competing firm.
(d) The presentation to the SSO must clearly and concisely set forth the results of the evaluation and discussion, including the strengths and weaknesses of each offeror's key personnel.
(5) Corporate or Company Resources. The SEB should assess the resources proposed by each offeror in the general areas of human resources and facilities. For example, are the proper skill mixes and numbers of people to do the work being offered? Does the offeror propose facilities and, where required, special test equipment suitable and adequate to assure timely performance of the work? If the offeror does not possess adequate resources internally, is there a demonstration of the ability to acquire them through subcontracts or otherwise?
b. Elements that further define the content of each subfactor may be used. If individually numerically scored, these elements must be revealed in the RFP and must be assigned a specific weighting in the SEB's evaluation plan as a portion of the total points allotted. If they are not numerically scored, identification in the RFP is not required.
c. Establishment of evaluation subfactors, elements, and their weights requires judgment on a case-by-case basis. The subfactors and elements established and included in the RFP will then be utilized to determine each offeror's rating in Mission Suitability, including its understanding of the requirements, approach to the work, and the competence of personnel to be directly involved.
d. By carefully considering the requirement(s) to be satisfied through the products or services being procured, the SEB should be able to identify, analyze, and score discrete subfactors and, where appropriate, elements that determine how well the proposed product or service can be expected to meet the demands of the specific requirement(s). If individual subfactors, elements, and their weights are prudently established, the integrated scores of the subfactors and elements will give a representative picture of the merit of each offeror's Mission Suitability.
e. (1) In structuring evaluation subfactors and elements, emphasis should be placed on identification of significant discriminators, or "key swingers" - the essential information required to support a source selection decision. Too many subfactors and elements are detrimental to effective evaluation of proposals and may result in a leveling or averaging out of scores over all proposals. To avoid this negative effect, the number of subfactors under Mission Suitability shall be no more than 4 and the number of elements no more than 8. Other evaluation factors shall also be limited to only essential subfactors and elements. Further, care should be taken to avoid overlap and redundancy by clearly defining each evaluation subfactor and element. Avoiding such overlap assures an offeror is not scored in two or more areas for the same work.
(2) The following example conveys one approach to describing a subfactor, utilizing elements, in an RFP. Within the "management plan" subfactor the following elements may be determined by the SEB as most suitable to assess how well each offeror's overall management proposal would contribute to the probability of performing the contract in an excellent manner: (a) management approach and organization; (b) staffing plan; and (c) management systems.
f. (1) Proposal risk will be carefully considered in evaluating proposals. The proposal risks to be assessed are those associated with cost, schedule, and performance or technical aspects of the program. These risks will be considered in the Mission Suitability subfactors and the Cost factor evaluation. Risks may be inherent in a program by virtue of the program objectives relative to the state of the art. Risks may occur as a result of a particular technical approach, manufacturing plan, the selection of certain materials, processes, equipment, etc., or as a result of the cost, schedule and economic impacts associated with these approaches. Risk may also occur from the impact that these will have on the offeror's ability to perform.
(2) As part of their proposal, offerors should be required to submit a risk analysis which identifies risk areas and the recommended approaches to minimize the impact of those risks on the overall success of the program.
(3) In evaluating risks, the evaluators must consider the offeror's assessment and make an independent judgment of the probability of success, the impact of failure, and the alternatives available to meet the requirements.
(4) Risk assessments shall be discussed in the SEB Report and be considered in determining the overall numerical and adjectival ratings and the strengths and weaknesses.
(5) It is the responsibility of the evaluation teams to inform the cost team of identified risk areas and the potential for cost impact.
g. For each subfactor and element, there should be detailed instructions provided in the RFP's Section L, "Instructions, Conditions, and Notices to Offerors or Quoters," specifying what supporting information should be included in the proposals and the specific format to be used. This increases the probability that SEB evaluators will be provided necessary data in the format easiest to understand and evaluate relative to the subfactors and elements assigned for evaluation and assessment. Thought given to carefully structuring these instructions will generally result in proposals that address what the Government is most interested in relative to the work to be performed. For example, if the evaluation will utilize SEB committees or SEB panels, instructions for proposal preparation in the RFP's Section L that direct offerors to group together information required by such committees or panels will facilitate and accelerate the evaluation process.
h. Where the procurement involves acquisition of a major system under NMI 7100.14, "Major System Acquisitions," and the evaluation is to identify the most promising system design concept(s) to be selected for further exploration, the evaluation should also address the benefits to be derived by tradeoffs, where feasible, among technical performance, acquisition cost, ownership cost, and time to develop and procure.
2. Weighting of factors, subfactors, and elements
a. Numerical weights shall be used for evaluating the Mission Suitability factor for competing offerors.
b. (1) Once the Mission Suitability subfactors are established, the SEB will determine the weight assigned to each. Likewise, if elements are established, the SEB will determine the weight, if any, assigned to each. The proposed subfactors, elements, and weights will be presented to the SSO or designee for approval. The weight assigned to each subfactor and, if numerically scored, each element must reflect its relative importance within the overall Mission Suitability factor. In conjunction, an evaluation plan covering not only Mission Suitability evaluation, but all evaluation factors (Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations), will be established by the SEB (see Chapter 4). Mission Suitability evaluation subfactors, elements, and their weights shall be established and approved in advance of RFP issuance. The four factors and their supporting subfactors and, if numerically scored, elements shall be described in the RFP's Section M, "Evaluation Factors for Award," and the weights associated with the individual Mission Suitability subfactors and elements shall be revealed in Section M as well. However, care should be taken to avoid the impression of a mathematical evaluation devoid of judgment. The weights are intended to be used by the SSO as a guideline.
(2) If all evaluation factors are considered by the SEB to be of approximately equal importance, a statement to that effect shall be included in the RFP. However, if there is a difference in the level of importance among the factors, then a statement shall be included in the RFP to advise the offerors of the relative importance of the factors. In this regard, one example of a statement that might be appropriate, depending on the nature of the requirements, type of contract, and objectives of the acquisition, is as follows:
"Of the four evaluation factors identified above, Mission Suitability and Cost are most important, and, as related to each other, are approximately equal in importance. The Relevant Experience and Past Performance factor is of somewhat less importance than either Mission Suitability or Cost, and the Other Considerations factor is of considerably less importance than Relevant Experience and Past Performance.
The subfactors to be used in evaluating Mission Suitability and their corresponding weights are listed below in descending order of importance:
Understanding of the Requirement
(40 percent)
Excellence of Proposed Design
(30 percent)
Management Plan
(15 percent)
Key Personnel
(10 percent)
Corporate or Company Resources
(5 percent)
The numerical weights assigned to the five subfactors identified above are indicative of the relative importance of those evaluation areas. The weights will be utilized only as a guide."
302 COST
1. Before issuing a solicitation, the contracting officer and technical personnel shall (when it is feasible to do so) develop a Government cost estimate for the planned acquisition. Estimates can range from simple budgetary estimates to complex estimates based on inspection of the product itself and review of such items as drawings, specifications, and prior data. The SEB is tasked with the responsibility to ensure that the Government cost estimate properly reflects the effort to which the RFP applies.
2. In the Cost factor evaluation, the SEB shall analyze the proposed costs or prices of all offerors in accordance with the criteria in FAR 15.805. The SEB may use any or all tools available in performing these analyses, including information in the Armed Services Pricing Manual (ASPM). If field pricing/audit support is requested, the request should be tailored to reduce the time required for the support and still enable the SEB to properly review proposals. Whenever possible and appropriate, pricing/audit support should be limited to rates and factors checks, or otherwise limited to that information specifically required for the SEB deliberation process. All pricing/audit support requests should specifically identify the limited areas for which pricing assistance is required and should also specify the response date required for timely accomplishment of SEB proceedings. At the conclusion of its analyses, the SEB shall advise the SSO concerning--
a. The costs or prices as proposed by all offerors, including those not within the competitive range;
b. The comparison of costs proposed by all offerors, with the independent Government cost estimate, when feasible;
c. The realism of costs proposed by all offerors determined to be within the competitive range. Cost realism is a review of the proposal to determine if the overall costs proposed are realistic for the work to be performed, if the costs reflect an offeror's understanding of the requirements, and if the costs are consistent with the various elements of the technical proposal. This type of analysis will be used in the Cost area and Mission Suitability or other technical areas;
d. The probable cost to the Government of, at a minimum, each proposal within the competitive range. If it appears to the SEB that any offeror's approach(es) or plan(s) for accomplishing the proposed work will require modification in order to be acceptable to the Government, the SEB shall identify and assess the modification required, determine the probable cost of such modification, and include that probable cost assessment in its report to the SSO;
e. The differences in business methods, operating procedures, and practices as they impact cost; and
f. Its level of confidence in the probable cost assessments as they pertain to each fully evaluated proposal.
3. The probable cost should reflect the SEB's best estimate of the cost of any contract which might result from that offeror's proposal, including any recommended additions or reductions in personnel, equipment, or materials. To the extent that the recommended additions or reductions reflect a lack of understanding of the requirements of the RFP, that lack of understanding should be reflected in the scoring of the Mission Suitability factor, subfactors, and elements.
4. A well-defined statement of work reflecting clear, concise work breakdown structures is of great value in obtaining well-structured proposals and in allowing the SEB to understand and assess proposed costs.
5. All cost categories and amounts present in an offeror's cost proposal (including options) are to be analyzed by the SEB and reported to the SSO. In the event SEB members have different opinions as to the cost analyses and assessments of probable costs, these differing opinions should be reported to the SSO to aid in forming an opinion regarding the confidence to be attributed to the analyses and assessments.
303 RELEVANT EXPERIENCE AND
PAST PERFORMANCE
1. This factor addresses evaluation of overall relevant experience and past performance for the company, not the experience and past performance of individuals involved with contract performance; the latter are to be evaluated under key personnel within the Mission Suitability factor.
2. Relevant experience reflects the accomplishment of work by an offeror that is comparable to or related to the work or effort required under the instant procurement. Programs or projects of comparable magnitude that include technical, cost, schedule, and management constraints similar to those expected to be encountered in the instant procurement are clearly relevant.
3. Past performance is especially important; how well the offeror performed on similar work may be a significant indicator of performance on the job at hand. Many organizations exhibit characteristics that persist over time--for example, the ability to move projects out of the research environment and to translate research findings into practical, results-oriented hardware; difficulty in transitioning conceptual efforts into soundly engineered "hardcopy" plans which can be produced economically; resiliency in the face of trouble; resourcefulness; management determination that the organization live up to its commitments; and skill in development of key people. It is essential to develop such an indicator independent of the offeror's proposal. It is the responsibility of the SEB to collect and document information on the past performance of, at a minimum, all offerors within the competitive range. The SEB should personally contact program or project managers within NASA or other Government agencies in a position to have responsibly observed performance of the offerors as either a prime or major subcontractor and obtain their views concerning the quality of the work the offeror did (or is doing) on comparable jobs. In the event a substantially unfavorable response is received which, in the opinion of the SEB will be of special significance to the SSO, the offeror in question will be provided an opportunity during written and/or oral discussions (see Chapter 4) to clarify that response.
4. The SEB is responsible for collecting and developing relevant experience and past performance information, identifying the source, and presenting it to the SSO. While the clearly relevant experience or past performance is of prime importance, other experience or past performance determined by the SEB to be significant or to be indicative of company experience or past performance should be noted and brought to the attention of the SSO. All pertinent information, including project manager assessments and offeror responses, will be:
a. Made part of the SEB's records,
b. Contained in the SEB report, and
c. Presented to the SSO.
5. The Relevant Experience and Past Performance factor is not numerically scored but is assigned an adjectival rating by the SEB.
6. This factor may be a significant consideration in the selection process. Therefore, the basis of the evaluation must be thoroughly identified and well documented. While objectivity is desirable, reasonable judgments often have to be made in the evaluation process. Subjectivity is not improper; an SEB should not hesitate to make reasonable subjective evaluations of relevant experience and past performance based upon as much factual data and experiential information as can be reasonably accumulated.
304 OTHER CONSIDERATIONS
1. This factor includes all considerations other than Mission Suitability, Cost, and Relevant Experience and Past Performance that the SSO will consider in making a final selection. Only Other Considerations specifically identified in the RFP shall be considered by the SSO in making a decision.
2. Following is a listing of Other Considerations that may be appropriate to include in an RFP. Not all of the subfactors listed are necessarily applicable to all procurements; nor is the listing intended to be all inclusive or restrictive:
a. Financial condition and capability.
b. Corporate priority on the work being proposed, or importance of the business to corporate management.
c. Labor-management relations.
d. Extent of proposed small and small disadvantaged business, and women-owned small business enterprise participation in subcontracting arrangements.
e. Geographic distribution of the work to be performed.
f. Acceptance of contract terms and conditions set forth in the RFP.
g. Quality and productivity improvement plan.
h. Any other subfactors pertinent to the particular procurement and identified under the Other Considerations factor within the RFP.
3. Other Considerations shall be defined specifically in the RFP, evaluated by the SEB, and reported to the SSO. Certain subfactors under the Other Considerations factor, such as financial condition and capability, may undergo change up to the moment of source selection.
4. Information regarding some of the Other Considerations is generally available to an SEB in the form of preaward surveys, NASA inspection reports, facility capability reports, purchasing system surveys, audit reports, and equal employment opportunity surveys. The SEB should make every reasonable effort to identify and use timely existing reports before initiating original inquiries. While written reports such as those mentioned in this paragraph may be significant, they should not be exclusively relied upon by the SEB. In addition, personal inquiries should be directed to Government managers likely to be knowledgeable about the offeror's record in these areas.
305 CONTINUING EVALUATIVE
RESPONSIBILITY OF SEB
Even after the SEB has made its formal report to the SSO, the SEB shall have continuing responsibility to report to the SSO, until its discharge, any circumstances that would change the SEB's evaluation findings relative to any evaluation factor (Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations) for any offeror. It is not intended that, after its report, the SEB actively pursue continuing evaluation. What is expected is that matters related to any of the evaluation factors which come to the attention of the SEB, and which might be expected to be pertinent to the selection decision, will be communicated to the SSO.
400 GENERAL
This Chapter describes procedural steps in preparing the SEB for its work and outlines SEB activities for solicitation and evaluation of proposals.
401 INITIAL SEB ACTIVITIES
1. Official SEB activities commence upon receipt by the Chairperson of the letter establishing the SEB and designating its members which should be no later than approval of the procurement plan. However, prior to official establishment of the SEB, the proposed SEB Chairperson and members may hold meetings to accomplish such tasks as SEB member orientation to proposed operational procedures, security measures that will be utilized, and preliminary review of the proposed RFP.
2. Once the SEB is established, the SEB Chairperson shall ensure that--
a. A management and staffing plan is prepared, indicating necessary personnel and other resource requirements, including a time schedule for SEB actions and events leading to presentation of findings to the SSO;
b. Each SEB voting member is furnished appropriate information regarding the nature of the procurement in addition to a copy of their designation letter, approved procurement plan, and the projected time schedule;
c. Each SEB participant, voting and nonvoting, is cautioned concerning restrictions on disclosure of information during the SEB process, avoidance of conflicts of interest, and conformity with the NASA Standards of Conduct. Each SEB participant, voting and nonvoting, shall file a NASA Form 1270, "Confidential Statement of Employment and Financial Interests," in accordance with NHB 1900.1, "Standards of Conduct for NASA Employees." In addition, each SEB participant shall sign an appropriate nondisclosure statement (a sample nondisclosure statement is included in Appendix B). These documents shall be reviewed by the Associate General Counsel for General Law or designee for Headquarters employees or by the cognizant Field Installation Chief Counsel or designee for field installation employees.
d. Letters are issued to all personnel involved in the SEB's activities advising them of pertinent restrictions and prohibitions, including a caution not to discuss any aspect of the procurement with anyone not having a need-to-know. The right to information on a need-to-know basis does not extend to the normal chain of supervision of any member of the SEB or to any individual having technical responsibility for the effort being evaluated except as specifically approved by the SEB Chairperson on a case-by-case basis. Those individuals will also be notified by the SEB Chairperson, in writing, of the privileged character of proposal information;
e. The list of sources to be solicited is developed and approved by the SEB with the assistance of the cognizant procurement and program or project offices;
f. A draft RFP is an effective method of obtaining industry comment on our requirements and engendering industry goodwill. A draft RFP should be used whenever it is expected to be beneficial. The draft RFP should be complete and include all applicable sections, including Sections L and M. Where appropriate, the Statement of Work or specifications may be released in advance of the draft RFP;
g. Prior to RFP issuance, the substance and the weight of evaluation factors, subfactors, and elements, as well as the substance of any applicable qualification standards, are presented to the SSO or designee for approval;
h. The RFP is reviewed and approved by the SEB prior to issuance;
i. During the competitive phase of the procurement, every effort must be made to assure fair and equitable treatment of all offerors (both prospective and actual). Upon release of the formal RFP, the Chairperson shall impose a communication blackout, in writing, by directing all personnel associated with the procurement to refrain from communicating with prospective offerors, formally or informally, regarding any aspects of the procurement. All inquiries regarding the procurement shall be referred to the contracting officer; and
j. Prior to receipt of proposals, the evaluation plan is reviewed and approved by the SEB and cognizant management personnel. The evaluation plan, once approved, must be impartially applied by the SEB to each proposal.
402 EVALUATION PLAN
1. The SEB evaluation plan consists of general and specific evaluation guidelines (and qualification standards, where applicable) established to assess each offeror's proposal relative to the evaluation factors, subfactors, and elements set forth in the RFP. The evaluation guidelines are designed to focus the evaluators' assessment. They are not weighted and are not listed in the RFP. However, the substance of the guidelines may be included in a narrative description of the subfactors and elements. In addition, the plan includes the system used in conducting the evaluation and scoring each offeror's proposal.
2. The SEB determines what evaluation factors, subfactors, and elements to assign to the various committees and panels. While it is necessary to give committees and panels all information required to conduct an evaluation of their assigned area, it is not appropriate to disclose to them the specific scoring system used by the SEB in scoring proposals.
3. The detailed SEB evaluation plan to be used by the SEB, committees, and panels should be approved before the RFP is issued but, in any case, shall be approved before proposals are evaluated.
403 QUALIFICATION STANDARDS
1. It is NASA policy to offer the opportunity to compete for its procurements as broadly as is consistent with the nature of each particular procurement. However, in view of the distinctive characteristics of NASA programs, those potential offerors that do not possess the minimum qualifications and resources necessary to perform the proposed work of a given procurement should not be encouraged to incur proposal and other expenses involved in competitive submissions.
2. To accomplish this objective without limiting meaningful competition requires early and intense effort on the part of the SEB, working in conjunction with the program and procurement staff most familiar with the procurement requirements.
3. When the nature of the procurement requires, qualification standards may be established in accordance with FAR 9.104-2. These will consist of special experience, capability, or specialized facilities critical to program performance aspects of the procurement.
4. In establishing qualification standards, care must be exercised to restrict them to those essential to the successful completion of the contract work. Qualification standards may be employed only where it is possible for the SEB to establish standards which are justified by the nature of the particular procurement. These standards must be applied equally to all potential sources.
5. When developed by the SEB and approved by the SSO or designee, qualification standards shall be clearly set forth in the RFP and included in the Commerce Business Daily synopsis.
6. Approved qualification standards will be used by the SEB to screen proposed source list(s) so that only firms possessing those unique specialized facilities, capability, or experience deemed critical to program performance aspects of the procurement will be solicited. Notwithstanding the considerations that lead to the elimination of sources from solicitation, any firm may submit a proposal. These same qualification standards will be used to consider all sources who may submit a proposal. This assures that the presolicitation consideration of prospective contractor qualification standards does not act to restrict competition, but only to discourage costly proposal submissions from potential offerors to whom award would not appear likely and that all offerors shall have an equal opportunity to compete. If an offeror fails to meet the stated qualification standards, the proposal shall be rejected, not considered further in the evaluation, and the offeror informed of the basis of rejection.
404 REQUEST FOR PROPOSALS
(RFP's)--REVIEW AND
APPROVAL
1. Effectiveness of the evaluation is dependent, in large measure, on how well the work to be performed and the basic ground rules under which the competition will be conducted are described in the RFP. Accordingly, the RFP shall be reviewed by the SEB and approved by appropriate levels of management prior to issuance to ensure its acceptability. To streamline the solicitation review cycle, use of the Solicitation Review Board (SRB) technique, or its functional equivalent, is encouraged as an alternative to the traditional procedure of serial or sequential coordination of the solicitation with reviewing offices. The SRB is a meeting in which all offices having review and approval responsibilities meet to discuss the solicitation and their concerns. Individual reviewers should be given a reasonable amount of time to review the document prior to the meeting. At the conclusion of the meeting, recommendations for changes are made and the solicitation is formally approved or disapproved. Afterwards, formal written minutes reflecting the agreement of the SRB members shall be prepared and included in the contract file. Use of this technique not only expedites the review and approval process, but it also encourages the synergism of a number of acquisition professionals discussing their concerns in one forum.
2. In reviewing the RFP, the SEB and appropriate management shall ensure that the following matters pertinent to source evaluation and selection are fully covered in the solicitation:
a. Any qualification standards shall be identified and described.
b. Evaluation factors shall be described, including a general narrative explanation of their relative importance.
c. The Mission Suitability factor shall be described. Evaluation subfactors shall be identified and described, including their numerical weights. If deemed conducive to the evaluation process, elements of a subfactor shall be identified and described, including their numerical weights.
d. The Cost factor shall be described.
(1) When uncertainties involved in contract requirements necessitate the use of a cost-type contract, offerors may be motivated to perform within the proposed cost estimate through the inclusion of appropriate cost incentive arrangements.
(2) Negotiated indirect cost rate ceilings shall be used only in accordance with FAR 42.707.
e. The Relevant Experience and Past Performance factor shall be described. The identification data and contact or customer references required by NFS 1815.406-70(b)(6) for the conduct of this evaluation shall be stated. Offerors should be cautioned that omissions or an inaccurate or inadequate response to this evaluation category could have a negative effect on overall evaluation. Offerors should also be advised that, in addition to information they provide, the SEB will consider all information available to NASA regarding the offeror's relevant experience and past performance.
f. The Other Considerations factor shall be described.
g. The method of evaluation shall be explained clearly, but concisely, so that prospective offerors understand the SEB's use and treatment of the factors, subfactors, and elements.
h. When cost proposals are permitted to be submitted after technical proposals, a notice shall be included in the RFP stating the required date for each submission.
i. When applicable, a notice that a preproposal conference is to be held shall be included, stating its purpose, time, place, and scope.
j. The Statement of Work shall describe as clearly and concisely as possible the product or service to be procured. It shall be structured to identify the important areas of emphasis. There must be no inconsistencies between it and the evaluation factors, subfactors, and elements to be used by the SEB and the SSO. Nonessential or unduly restrictive requirements shall be eliminated.
k. Requirements for reports and data essential to contract evaluation and performance shall be clearly described, including a cross reference to the Statement of Work and a schedule for submission.
l. When the procurement involves a major system under NMI 7120.4, Management of Major System Programs and Projects, and NHB 7120.5, Management of Major System Programs and Projects Handbook, the SEB will ensure that the RFP is prepared in terms of mission need so each offeror can respond with an alternative system design concept proposal to satisfy the mission need and can propose a technical approach, design features, and alternatives to schedule, cost, and capability goals consistent with that concept. In order to remove inhibitions to innovative solutions and to improve the approach to achieving program objectives, consideration should be given to conducting orientation briefings for prospective offerors and, where appropriate, solicit comments on a draft of the solicitation.
m. Limitations on pages and the number of copies of offerors' initial proposals shall be included in accordance with NFS 1815.406(d). In no case shall the total proposal, excluding cost/price information, exceed 500 pages using the page definition of 1815.406(c), unless a different limitation is approved in writing by the Procurement Officer. Firm page limitations shall also be established for BAFOs, if requested. The appropriate BAFO page limitations should be determined by considering the complexity of the procurement and the extent of any written or oral discussions. The same BAFO page limitations shall apply to all offerors. Proposal pages exceeding the specified maximums shall be removed from the proposal or BAFO and not evaluated. One copy of these pages shall be retained for the official contract file and the remaining copies returned to the offeror. When excess pages are returned to the offeror, cite the Proposal Page Limitations provision of the solicitation.
n. When applicable, a notice is included that offerors are invited to give oral presentations after receipt of proposals but before initial evaluation.
3. The SSO may request a detailed review of the RFP beyond that of the SEB and cognizant management personnel or may, more specifically, request a detailed review of the Statement of Work, qualification standards, evaluation factors, subfactors, elements, or other areas pertinent to proposal preparation, evaluation, and source selection. Specific reviewing officials or offices may be designated for this purpose.
4. The RFP shall comply with all current regulations and directives applicable to NASA solicitations; particular emphasis is placed on the requirements of FAR 15.406 and NFS 1815.406.
5. When detailed program or project support plans will be required as part of the offeror's proposal but are not considered important discriminators in the evaluation process, the requirements for these plans shall be described in separate appendices to the Statement of Work in accordance with NFS 1815.406-70(a)(7).
6. When, either before or after receipt of proposals, the Government modifies its requirements, the contracting officer is required to issue a written amendment to the RFP in accordance with FAR 15.606.
405 PREPROPOSAL CONFERENCE
1. A preproposal conference to brief prospective offerors may be conducted after a solicitation has been issued but before proposals are received, in accordance with FAR 15.409. The contracting officer, in conjunction with the SEB, shall make a determination prior to issuance of the solicitation whether a preproposal con-ference will be held. Conferences should be held only when their value to the Govern-ment and prospective offerors is apparent, and the specifics of any given procurement should be carefully considered in assessing this value. Generally, these conferences are of most value in complex acquisitions where it is necessary to ensure that complicated specifications or requirements are fully and clearly understood. These conferences can also be of value in major on-site service contracts to familiarize offerors with the center organization and its physical layout, as well as the specific contract tasks. (In these latter cases, site visits and observation of on-going operations should also be considered). In more routine, less complex procurements, however, the preproposal conference offers far less value and should normally not be used.
2. When a preproposal conference is held, the solicitation shall include the Preproposal/Pre-Bid Conference provision at 1852.215-77, indicating the time and place of the conference. The conference shall be scheduled to permit prospective offerors sufficient time after the issuance of the solicitation to become familiar with its requirements but not too late to allow meaningful use of the information obtained at the conference.
3. The conference should include a presentation of the significant aspects of the procurement followed by a question and answer session. A record of all information provided at the conference, together with a copy of all questions and answers, shall be provided to all prospective offerors by formal written amendment to the solicitation. If it becomes apparent at the conference that the RFP needs revision, the revised requirements must be included in a written amendment.
406 OFFEROR ORAL PRESENTATIONS
1. In some cases, offeror oral presentations may expedite the evaluation and provide insight into the proposals. These presentations are of particular value in procurements when the proposal content is expected to be complex or when the proposal will include an unusually large number of volumes. The SEB should consider the specifics of a given procurement in assessing the value of offeror oral presentations, and should utilize them only when their value is apparent.
2. If offeror oral presentations are held, they should be conducted before commencement of initial evaluation of proposals, normally not more than three days after receipt. The offerors should be instructed that the presentations are to be a "roadmap" to understanding their proposals, i.e., a discussion of the proposal organization and layout, and where required information and elements are located. Although the offeror's basic approach to satisfying solicitation requirements may be explained, it is to be done so only in general terms and only to expedite the Government's formal evaluation. The presentation is not intended for the offerors to provide additional information or supporting rationale that was not included in the proposal, nor is it to be a forum to "market" the proposal. Government attendees shall not engage in any discussions during the oral presentation, and no proposal revisions may be accepted as part of the presentation. The Government's evaluation of offeror proposals is to be based on the contents of the initial proposal, and any information not included in the initial proposal that is provided at the oral presentation shall not be evaluated.
3. A typical presentation consists of a strictly enforced one to two hour briefing period followed by a question and answer period. Government evaluators may ask questions about the proposal "roadmap" only and should not engage in detailed discussions over the value or validity of the offeror's proposed approach to satisfying solicitation requirements. All offerors must be afforded the same opportunity to brief and the same briefing groundrules. If considered appropriate, limitations may be placed on the number of offeror and Government attendees to ensure the presentation meets its intended purpose. The order of presentation should be determined at random.
4. The SEB may determine which of the Government participants in the evaluation may attend the offeror oral presentations. However, to eliminate bias and to ensure the objectivity during the evaluation process, any individual who attends one presentation must attend all presentations, unless prevented from doing so by emergency or other unforeseen event. The SEB shall retain one copy of all briefing materials (if any) used in the presentation, as well as SEB-prepared minutes of the meeting, as historical records for the official contract file. However, the briefing materials and minutes shall not be used in the formal evaluation process.
407 INITIAL EVALUATION
1. Upon receipt of proposals, the contracting officer shall mark each proposal with the date and time of receipt and forward all proposals to the SEB Chairperson or the SEB Recorder for control and safeguarding throughout the evaluation process.
2. Late proposals or modifications shall be handled in accordance with FAR 15.412.
3. Procedure for Evaluation of a Single Proposal
a. If only one proposal is received in response to the solicitation, the contracting officer shall examine the situation to ascertain the reasons for the single response. The contracting officer shall include a written notation in the contract file describing the circumstances surrounding the receipt of a single proposal as well as a determination whether or not the solicitation was or was not flawed or unduly restrictive, prior to releasing the proposal to the SEB for evaluation.
b. The SEB shall complete a limited preliminary evaluation to determine that the single proposal submitted satisfies stated qualification standards, if any, and is otherwise an acceptable proposal (see subparagraph 406.5). Upon completion of this limited preliminary evaluation, a letter shall be forwarded to the SSO presenting a summary of the evaluation results.
c. The SSO shall as a result of reviewing the SEB summary, and the contracting officer's examination of the facts and determination, notify the SEB Chairperson that either--
(1) The SEB is instructed to proceed with pre-established SEB procedures, including complete initial evaluation, oral and written discussions, request and receipt of a Best and Final Offer (BAFO), and final evaluation, culminating in a formal presentation to the SSO for approval to enter into negotiations;
(2) The contracting officer is instructed to immediately enter into negotiations resulting in a complete contract document, signed by the offeror, to be signed by the contracting officer upon approval by the SSO. Upon completion of the negotiations, the SEB shall present the results to the SSO for approval; or
(3) The SEB Chairperson is granted full delegation of authority to disband the SEB and to instruct the contracting officer to conduct negotiations for contract award without further SSO involvement.
(4) The contracting officer is instructed to reject the single proposal received and to cancel the solicitation.
d. These procedures are likewise applicable when the number of proposals equals the number of awards contemplated or when only one acceptable proposal is received as defined by subparagraph 406.5 of this handbook.
4. Committee Evaluations
a. The initial phase of evaluation generally will involve established committees. As promptly as possible, committees are to be convened. The SEB Chairperson shall transmit to the committees proposals or portions of proposals to be evaluated, instructions regarding the expected function of each committee, and all data considered necessary or helpful. The committee chairperson is responsible for instructing the members as to committee functions, responsibilities, and procedures.
b. While oral reports may be given to the SEB, the committee function requires the submission of a written report which should include--
(1) Copies of individual worksheets and supporting comments to the lowest level evaluated;
(2) An evaluation sheet summarized for the committee as a whole; and
(3) A statement for each proposal describing any strengths or weaknesses which significantly affected the evaluation and stating any reservations or concerns, together with supporting rationale, which the committee or any of its members want to bring to the attention of the SEB.
c. It is imperative that the SEB provide clear traceability throughout the evaluation process. This traceability must exist at all levels of the SEB process. All reports submitted by committees or panels will be retained as part of the SEB records but need not necessarily be included as part of the SEB report to the SSO. A committee report should be included with the SEB report if it is so significant that its inclusion is necessary to the SSO's understanding of the SEB's action.
d. Each voting SEB member shall thoroughly review each proposal. Committee reports and findings shall be reviewed by the SEB. The SEB is to consider the committee evaluation, take into account any reservations or concerns stated by the SEB members and the committee, and rate or score the proposals for each evaluation factor, subfactor, and element according to its own collective judgment. SEB minutes shall reflect this evaluation process which shall be consistent with the approved evaluation plan.
5. Identification of Unacceptable Proposals
a. The SEB may discontinue the evaluation of any proposal which is unacceptable because--
(1) It does not represent a reasonable initial effort to address itself to the essential requirements of the RFP or clearly demonstrates that the offeror does not understand the requirements of the RFP;
(2) In research and development procurement, a substantial design drawback is evident in the proposal, and sufficient correction or improvement to consider the proposal acceptable would require virtually an entirely new technical proposal; or
(3) It contains major technical or business deficiencies or omissions or out-of-line costs which discussions with the offeror could not reasonably be expected to cure.
b. Simple technical nonresponsiveness in the sense in which the term is used in sealed bidding is not alone sufficient to constitute unacceptability if the proposal is otherwise competitive and written and/or oral discussions, or negotiations after selection, reasonably offer the likelihood of resolution. When there is doubt as to whether a proposal should be rejected initially as unacceptable, that doubt shall be resolved by including it for further consideration.
c. The SEB must document its judgment that the deficiencies of any proposal are sufficiently significant to warrant discontinuing evaluation of the proposal at this point in the process.
6. SEB Findings
a. Preparing the results of the initial evaluation in a narrative SEB report is an important aspect of the evaluation process. The SEB should be aware that the SEB report and presentation provide the principal tools available to the SSO to perform a comparative analysis in making a source selection decision. Guidelines for the report and presentation are set forth in Chapter 5 and Appendix C. For each proposal, the SEB should indicate in the narrative whether the proposal meets or fails to meet any of the requirements of the RFP; all strengths and weaknesses of the proposal and whether they are correctable; the proposed and probable cost (normally required only for those offeror(s) in the competitive range or those offeror(s) eliminated from competitive range on the basis of cost); the technical, schedule, and cost risk associated with the proposal; and the adjectival rating of the Relevant Experience and Past Performance and Other Considerations evaluation factors. Strengths and weaknesses must be further defined in the SEB report as major or minor to provide a valuable summary of discriminators among proposals in addition to supporting the adjectival ratings assigned to each Mission Suitability subfactor and element by the SEB, and for the Mission Suitability factor as a whole. In addition to the narrative report of evaluation findings, the SEB will apply the Mission Suitability scoring system detailed in the evaluation plan to rate each proposal in relation to the Mission Suitability evaluation subfactors and elements.
b. In structuring an applicable rating system, the following adjective ratings, definitions, and percentile ranges shall be used for the evaluation of Mission Suitability subfactors, and, if individually weighted and scored, elements. (The total proposal should also be classified with an appropriate adjectival rating for Mission Suitability.)
PERCENTILE RANGE Excellent A comprehensive 91-100 and thorough pro- posal of exceptional merit with one or more major strengths. No weaknesses or only minor correctable weaknesses exist. Very Good A proposal which 71-90 demonstrates over- all competence. One or more major strengths have been found, and strengths outbalance any weaknesses that exist. Any major weak- nesses are correctable. Good A proposal which 51-70 shows a reasonably sound response. There may be strengths or weaknesses, or both. As a whole, weaknesses, not off-set by strengths, do not significantly detract from the offeror's response. Major weak- nesses are probably correctable. Fair A proposal that has 31-50 one or more weaknesses. Weaknesses have been found that outbalance any strengths that exist. Major weaknesses can probably be improved, minimized, or corrected. Poor A proposal that has one 0-30 or more major weaknesses which are expected to be difficult to correct, or are not correctable.
c. Normally, proposals are to be rated using adjectival ratings on two occasions as follows:
(1) Upon completion of the initial evaluation of proposals, and
(2) Upon completion of the final evaluation of the BAFO's.
Both ratings will be maintained by the SEB, included in the SEB report, and presented to the SSO. There shall be clear traceability of any scoring changes between initial and best and final proposals.
d. Proposals may also be evaluated and scored under an alternate method that does not require scoring after initial evaluation of proposals. It normally may only be used when two conditions exist: (1) relatively few (e.g., five or fewer) proposals are received; and (2) a full scoring or ranking of proposals is not required to make the determination that all offerors are in the competitive range. If during the initial evaluation of proposals, it becomes apparent that all offerors will be in the competitive range, the SEB may then elect to use the alternate method of initial evaluation by making this competitive range determination without formally scoring or ranking the proposals or generating the extensive documentation required to support this scoring or ranking. Once the alternate method is chosen, the initial evaluation is completed when the SEB determines strong and weak points for all proposals and develops questions for written and oral discussions. The SEB will then proceed directly to the discussion phase of the evaluation process. Only the BAFO is fully scored and ranked, and the results of this scoring or ranking are presented to the SSO to assist the selection decision. The principal benefit in using the alternate scoring method is the reduced time and resources associated with not performing two separate proposal scorings along with the attendant documentation.
7. Determination of Competitive Range
a. Subsequent to the initial identification of proposals considered unacceptable, the SEB will compile initial evaluation findings, of all remaining acceptable proposals, sufficient for determination of the competitive range in accordance with FAR 15.609 and NFS 1815.613-71(b)(4). The competitive range shall be determined by the SEB together with the contracting officer on the basis of Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations stated in the solicitation and shall include all proposals that have a reasonable chance of being selected for award. Competitive range determinations are final unless the SSO determines otherwise.
b. The objective of a competitive range determination is not to eliminate proposals but to facilitate competition by conducting written and/or oral discussions with only those offerors who have a reasonable chance of being selected for award. Where there is doubt as to whether a proposal is or is not within the competitive range, the proposal should be included within the competitive range. The determination of competitive range is a complex determination based on informed judgment. All competitive range decisions shall be completely and adequately documented in the contract file in accordance with NFS 1815.613-71(b)(4).
8. Notification of Unsuccessful Offerors. The contracting officer shall notify each unsuccessful offeror in accordance with FAR 15.1002.
408 FINAL EVALUATION
1. Evaluation of Plant and Facilities
a. Inspections at the plants of competing offerors could provide valuable insight into the SEB's evaluation of proposals. For instance, in procurements where significant experimental, research, developmental, testing, fabrication, or other work is to be performed (the quality of which may be affected by a contractor's plant or facilities), a complete evaluation may require an on-site visit by the SEB. For other procurements, e.g., support services contracts, plant inspections may serve no useful purpose.
b. When plant inspections are conducted, the visiting team shall include SEB members and any qualified experts required by the SEB. At least one SEB member shall be the same on all team visits to provide continuity and a basis for comparison. Visits are to be conducted, generally, after the SEB's initial evaluation has been completed and for a specific, clearly understood purpose. Visits shall be conducted only with the approval of the SEB Chairperson, who will ensure that all visits are made on an impartial basis. All personnel must remember that only the contracting officer can commit NASA and that they must avoid any contact with an offeror that is not essential or which could raise questions of impropriety. A visit to all offerors within the competitive range is advisable if plant inspections are being conducted.
c. Some potential benefits of plant visits are--
(1) Reviewing with resident Government personnel relevant experience and past performance;
(2) Reviewing the degree of capability and interest of the offeror to undertake the project in light of other work planned or in process; and
(3) Examining such matters as--
(a) Plant capacities;
(b) Management and technical capability of personnel;
(c) Availability of existing facilities, both Government-owned and contractor-owned;
(d) Adequacy of accounting practices and cost controls;
(e) Adequacy of estimating systems; and
(f) Ability to forecast and meet program schedules.
2. Written and/or Oral Discussions
a. Except as provided in FAR 15.610(a), the contracting officer shall conduct written and/or oral discussions with all responsible offerors in the competitive range. If, however, a decision is made in accordance with FAR 15.610(a)(3) to award without conducting written and/or oral discussions upon the completion of the initial evaluation, the SEB Chairperson shall forward a letter to the SSO for approval presenting a summary of the evaluation results. Upon SSO approval, the contracting officer shall undertake to complete all actions necessary to award the contract, without discussions or negotiations, to the offeror submitting the lowest overall cost proposal.
b. Written and/or oral discussions shall be conducted in accordance with FAR 15.610 (with the exception of FAR 15.610(c)(2)) and NFS 1815.613-71(b)(5). Preparation for written and/or oral discussions should include, but not be limited to, the establishment of the following:
(1) The time(s) and place(s) for conducting discussions. This requires establishment of the order of discussions with offerors. When feasible, the order should be established alphabetically or by lot. When discussions are to be at offeror's plants or offices it may be that geographic dispersion will be the deciding factor. The objective is impartiality.
(2) Topics for discussion. This will include preparation and issuance of written questions common to all offerors, as well as those that are peculiar to individual offerors, allowing adequate time for development of responses by offerors.
(3) The Government team that will conduct the discussions. The contracting officer, together with the SEB Chairperson, will designate appropriate procurement, pricing, and technical personnel for the Government team in addition to identifying the individuals specified in the offeror's proposal requested to be present for the discussions.
c. The contracting officer, together with or on behalf of the SEB, will conduct written and/or oral discussions of the effort to be accomplished and the cost or price of the effort with all offerors determined to be within the competitive range. The discussions are intended to assist the SEB--
(1) In understanding fully each offeror's proposal and its strengths and weaknesses based upon the individual efforts of each offeror;
(2) In assuring that the intent and the points of emphasis of RFP provisions have been adequately conveyed to the offerors so that all offerors are competing equally on the basis intended by the Government;
(3) In evaluating the personnel proposed by each offeror; and
(4) In presenting a report to the SSO that makes the discriminators identified among proposals clear and visible.
d. In cost-reimbursement type contracts and all research and development contracts, the contracting officer shall point out instances where the meaning of some aspect of a proposal is not clear and instances in which some aspects of the proposal failed to include substantiation for a proposed approach, solution, or cost estimate. However, where the meaning of a proposal is clear and the SEB has sufficient information to assess its validity and the proposal contains a weakness that is inherent in an offeror's management, engineering, or scientific judgment or which is the result of its own lack of competence or inventiveness in preparing its proposal, the contracting officer shall not point out the weaknesses. The possibility that such discussions may lead an offeror to discover that it has a weakness is not a reason for failing to inquire into a matter where the meaning is not clear or where insufficient information is available, since understanding of the meaning and validity of the proposed approaches, solutions, and cost estimates is essential to a sound selection. Offerors should not be informed of the relative strengths or weaknesses of their proposals in relation to those of other offerors. To do so would be contrary to regulations which prohibit the use of auction techniques (see FAR 15.610(d)(3)). In the course of discussions, Government participants should be careful not to transmit information which could give leads to one offeror as to how its proposal may be improved or which could reveal a competitor's ideas.
e. In fixed-price type contracts other than for research and development, the specifications ordinarily describe the Government's requirements with more particularity than is possible in cost reimbursement or research and development contracting, so that less emphasis is placed on an offeror's introduction of scientific, engineering, and management innovations. The contracting officer, in written and/or oral discussions, shall point out instances in which some aspect of a proposal contains a weakness in relation to the Government's requirements. However, the contracting officer shall neither point out the relative strengths or weaknesses of a proposal in relation to those of other offerors nor transmit information which could give leads to one offeror as to how its proposal may be improved or which could reveal a competitor's ideas. The contracting officer shall point out price elements that do not appear to be justified and shall encourage offerors to put forward their most favorable price proposals, but shall not discuss, disclose, or compare price elements of any other offeror.
f. Although unusual, if the SEB and the contracting officer, after conducting written and/or oral discussions, determine that a proposal no longer has a reasonable chance of being selected for contract award, the proposal can be eliminated from the competitive range in accordance with FAR 15.609(b). The contracting officer shall notify the unsuccessful offeror in writing that its proposal is no longer eligible for award.
g. Normally, written or oral discussions are completed with each offeror in the competitive range in one round. In some cases, however, a single round of discussions prior to requesting BAFOs may be insufficient for a comprehensive evaluation, and multiple rounds may be required. In these cases, when discussions have been declared closed and BAFOs requested, the approval of the Associate Administrator for Procurement (or the Procurement Officer when the value of the procurement is less than $25 million) is required to reopen discussions (see 1815.611). Each subsequent round of discussions is subject to the same groundrules as the initial round (e.g., no discussion of weaknesses where such discussion is otherwise prohibited).
3. Best and Final Offers (BAFO's)
a. The contracting officer will issue to all offerors still within the competitive range a request for Best and Final Offers (BAFO's) in accordance with FAR 15.611. Oral requests for BAFO's shall be confirmed in writing.
b. A common cutoff date and time that allows each offeror a reasonable opportunity to support and clarify its proposal through submission of a written BAFO shall be established. An offeror may, on its own initiative, revise its proposal and make corrections or improvements until the established cutoff date.
c. Offerors should be cautioned to provide supporting documentation for any changes to their prior offers. Any revision received after the established common cutoff date must be considered late in accordance with FAR 15.412.
d. The cutoff date must be such as to permit adequate time for all offerors to submit revised proposals; particular care must be taken to ensure that there is no compression of time for the offeror with whom discussions were last held, i.e., if 2 weeks is adequate time for submission, then the time allowed should be 2 weeks from the date of last discussions.
4. Evaluation Findings
a. After consideration of all committee reports, information received from offerors through plant visits, written and/or oral discussions, and BAFO's, the SEB shall conduct a final evaluation of proposals.
b. The final evaluation must build on the SEB's earlier recorded findings. The purpose of the final evaluation is to determine the effects, if any, of discussions and BAFO's on the SEB's earlier Mission Suitability scores and adjectival ratings as well as on the initial evaluation of Cost, Relevant Experience and Past Performance, and Other Considerations. Therefore, a clear and logical audit trail shall be maintained for the rationale for changes in ratings and scores, including a detailed account of the SEB decisions leading to the final ratings and scores.
c. The final evaluation must represent the collective judgment of the SEB regarding its assessment of the offerors for each factor evaluated including its assessment of the Mission Suitability subfactors, elements, if any, and its associated ratings, scores, and findings. The adjectival ratings and numerical scores shall reflect the strengths, weaknesses, and discriminators the SEB finds in the proposals. In this way, the reasons for differences in adjectival ratings, numerical scores, and findings can readily be explained to and understood by the SSO.
d. All significant SEB evaluation findings shall be fully documented and incorporated into a written report which will serve as the basis for the selection decision by the SSO.
500 GENERAL
This Chapter describes the requirements for the written report and oral presentation to the SSO.
501 RESPONSIBILITIES AND
PROCEDURES
1. The SEB shall prepare a written report of its findings, signed by the Chairperson and all voting members of the SEB. It shall present its written report and make an accompanying oral presentation to the SSO. Guidelines for the written report and the oral presentation are set forth in Appendix C.
2. SEB Preliminary Presentation (Dry Run)
a. When the Administrator is the SSO, a preliminary presentation should be made to the Field Installation Director and to the Official-in-Charge of the cognizant Headquarters Program Office. When the Official-in-Charge of the cognizant Headquarters Program Office is the SSO, a preliminary presentation should be made to the Field Installation Director and the cognizant Headquarters Deputy Program/Staff Associate Administrator or the official responsible for the specific project. The dry run presentation is the same presentation prepared for the SSO. Attendance at the dry run shall be restricted to those personnel who are involved in the selection process or who have a valid need-to-know. For Headquarters dry runs, an attendance list will be issued by the SEB coordinator in the Office of Procurement (Code HS). Admittance to the dry run will be restricted to those individuals whose names appear on the attendance list. However, changes proposed 24 hours before the presentation will be considered. Persons not on the list will not be admitted without the prior authorization of the SEB coordinator. For field installation dry runs, the attendance list will be prepared by a designee of the Procurement Officer.
b. The following personnel or designees should attend the Headquarters dry run presentation:
(1) Cognizant program/ staff officials.
(2) Official-in-Charge of the cognizant Headquarters Program Office in those cases when the Administrator serves as the SSO.
(3) NASA Comptroller.
(4) Associate Administrator for Procurement and/or Deputy Associate Administrator for Procurement.
(5) General Counsel and/or Associate General Counsel (Contracts).
(6) Associate Administrator for Safety, Reliability, Maintainability, and Quality Assurance, when SRM&QA matters are involved.
(7) Field Installation Director.
(8) Field Installation Chief Counsel.
(9) Selected voting and nonvoting ex officio SEB members (no designees).
(10) Field Installation Procurement Officer.
(11) Headquarters procurement personnel responsible for support to the Headquarters cognizant Program/Staff Office.
(12) Other personnel as approved by the Official-in-Charge of the cognizant Headquarters Program Office and the Office of Procurement (Code HS).
c. The SEB Chairperson or designee is responsible for arranging and coordinating the SEB dry run with the SEB coordinator. The SEB dry run should take place far enough in advance of the presentation to the SSO to allow for changes or revisions to the presentation material, if necessary. The material used during the dry run presentation is considered to be "SEB Sensitive" and should be treated accordingly. Consequently, the SEB Chairperson is responsible for providing a projectionist, who has been SEB cleared, to present the material for both presentations.
d. The dry run presentation, as well as the presentation to the SSO, will ordinarily be made by the SEB Chairperson; however, if another official has been designated to make the final presentation to the SSO, that person shall also make the dry run presentation. The SEB presentation material shall be delivered to the SEB coordinator for control and distribution no later than 7 working days before the scheduled dry run presentation date. The SEB presentation material includes 10 copies of the SEB report and charts.
3. SEB Formal Presentation
a. The SEB shall present its evaluation findings to the SSO as the basis for a sound selection decision. This formal presentation should be attended by the same personnel or designees in attendance at the dry run, with the exception of the inclusion of the SSO at the formal presentation. Continuity of attendees assures a more efficient formal presentation in addition to limiting access to SEB sensitive data.
b. The cognizant procurement analyst in the Procurement Operations Division (Code HS) will function as the SEB coordinator and will perform coordination functions for the formal presentation when the Administrator or the Official-in-Charge of the cognizant Headquarters Program Office serves as the SSO. These functions include arranging the time and place of the presentation; assuring, with the concurrence of the appropriate personnel in the Office of the Administrator, proper attendance; and distributing SEB reports and graphic material.
4. The designated individuals to whom the SEB presentation(s) (dry run and final) are made will be responsible for ensuring that--
a. The requirements of this handbook and all applicable agency policies have been complied with in the solicitation and evaluation processes;
b. The written report and presentation accurately convey SEB activities and findings;
c. The oral presentation is arranged and conducted in a professional manner; it is complete and informative; and it can be concisely presented within the time allocated by the SSO; and
d. No changes (1) to established evaluation factors, subfactors, elements, weights, or scoring systems or (2) in the substance of the SEB's findings shall be made. They may, however, direct the SEB to reconvene to rectify procedural omissions, irregularities or inconsistencies, substantiate its findings, or revise the organization of the written report and/or the method of presentation.
5. All copies of the SEB reports and presentation material contain sensitive business information in addition to "SEB Sensitive" data and shall be adequately safeguarded throughout the source selection process. Physical safeguarding of sensitive data includes, but is not limited to--
a. Controlling the number of copies of proposals and SEB reports;
b. Restricting access to SEB work areas;
c. Securing SEB material under locked conditions or in vaults;
d. Minimizing SEB presentations other than the dry run and formal presentations; and
e. Limiting attendees at SEB presentations.
600 GENERAL
This Chapter provides guidelines relating to the source selection decision, notice and debriefing for unsuccessful offerors, source selection statement, and multiple selection decisions.
601 SOURCE SELECTION DECISION
1. The SSO shall use the evaluation factors, subfactors, and elements set forth in the solicitation to make the source selection decision. The SSO shall consider the SEB findings and the advice provided by cognizant line and staff management at the conclusion of the SEB presentation in determining which of the proposals submitted in response to the solicitation would prove most advantageous to the Government, all factors considered.
2. In accordance with NASA policy and FAR 15.613, upon source selection by the SSO, the contracting officer will proceed with negotiations leading to award of contract(s).
602 NOTICE AND DEBRIEFING
FOR UNSUCCESSFUL OFFERORS
1. a. When a proposal is no longer to be considered for contract award, the offeror will be promptly notified by the contracting officer, in writing, together, with a general, brief explanation of the major reasons in accordance with FAR 15.1002 and NFS 1815.613-71(b)(7). There are, generally, five points in the SEB process when it may be determined that a proposal is no longer to be considered for contract award:
(1) After determination that stated qualification standards, if any, have not been satisfactorily met.
(2) After identification of an unacceptable proposal in accordance with subparagraph 406.5 of this handbook.
(3) After determination of the competitive range resulting from the initial evaluation of proposals.
(4) After written and/or oral discussions resulting in a determination that a proposal does not stand a reasonable chance of being selected for award.
(5) After the selection decision by the Source Selection Official.
b. In the first four instances, the notice shall state that a revision of the proposal will not be considered. Post-selection notices shall be issued in accordance with FAR 15.1002(c) and 15.1003.
2. If any offeror requests a debriefing in writing, the offeror shall be formally debriefed and furnished the basis for the selection decision after the final source selection decision by the SSO. This debriefing should normally take place prior to contract award and be conducted in accordance with 15.1003. If the situation will not permit delaying the award in order to debrief unsuccessful offerors, the debriefing may be conducted after award.
603 SOURCE SELECTION STATEMENT
1. When the final source selection decision has been made by the SSO, a Source Selection Statement shall be prepared for the SSO's signature in a manner releasable to the competing offerors and the general public, if requested. When the Administrator or an Official-in-Charge of a Headquarters Program Office is the SSO, the Statement will be prepared by the Office of General Counsel. Source Selection Statements prepared at the field installation should be drafted by the Field Installation Chief Counsel or designee.
2. Source Selection Statements must describe the procurement; the SEB evaluation procedures; the substance of the Mission Suitability evaluation; evaluation of the Cost factor, Relevant Experience and Past Performance factor, and Other Considerations factor involved in reaching the selection decision. There should be coverage of unacceptable proposals, the competitive range determination, late proposals, or any other pertinent considerations applicable to the specific decision. It must be stressed that nothing can substitute for the use of good judgment. The Source Selection Statement must be self-sufficient and must reveal sound rationale for the selection clearly and succinctly, without revealing the scores involved, the proposed prices, or any other confidential business information.
3. Source Selection Statements shall be signed as soon as practicable after the final selection is made. The SSO may desire to have one or more key officials who participated in the selection decision concur on the Source Selection Statement prior to final signature.
4. As indicated in subparagraph 1, Source Selection Statements generally may be released to competing offerors and the general public; however, it is anticipated that the Statement will not always be available in final form or signed by the Source Selection Official at the time of debriefing of unsuccessful offerors. A draft of the decision portion of the Statement that has been reviewed by the SSO or a concurring official should be available to the designated Debriefing Official for guidance at the debriefing. In order to avoid any misunderstanding, the Source Selection Statement rationale contained in the draft Statement should have the approval of the SSO or a concurring official prior to any debriefing(s). This will enable NASA to demonstrate, for the record, that the SSO's selection rationale was not modified substantially (editorial changes can be made) as a result of the debriefings. If requested by an offeror a copy of the final Source Selection Statement will be furnished to the offeror when it is signed by the SSO, subject to the provisions of subparagraph 5.
5. a. Under some situations, multiple selections are made for the purpose of permitting a design or other competition to continue until a final selection is made for a single contractor to undertake full-scale development. Stating the strengths and weaknesses in the Source Selection Statement in these cases may result in the premature disclosure of innovative concepts, designs, and approaches. Release of these Selection Statements to competing offerors or the general public prior to final selection of concept(s) or contractor(s) for full-scale development could compromise the integrity of the competition by making possible a transfusion of ideas which could also inhibit offerors during the early phase from offering their best and most promising ideas for meeting the mission need.
b. Accordingly, unless prior approval is obtained through the Headquarters Procurement Operations Division (Code HS) with the concurrence of the Office of General Counsel, Source Selection Statements for the selection of alternative system design concepts subject to NMI 7120.4, Management of Major System Programs or Projects and NHB 7120.5, Management of Major System Programs or Projects Handbook, are not to be released to competing offerors or the general public, if requested, prior to the release of the Source Selection Statement for full-scale development.
c. A similar problem may occur in other procurements where competition continues but is not covered under NMI 7120.4 or NHB 7120.5. When possible, care should be taken to set forth the strengths and weaknesses and other information in a manner that will avoid this problem. However, if this is not feasible, the Statement should not be released except in accordance with the procedures of the Freedom of Information Act, 5 U.S.C. 552.
604 MULTIPLE SELECTION DECISIONS
1. While SEB procedures contemplate that the SSO will be in a position to select a single source for final contract negotiations subsequent to the SEB presentation, a variety of considerations may lead the SSO to direct that contract negotiations be conducted with two or more offerors. Such negotiations are to result in complete contract documents signed by the offerors which may be accepted by the agency upon final selection of the successful offeror by the SSO.
2. The SEB shall consult with the SSO, the contracting officer, and the negotiating team regarding negotiation positions, objectives, and information to be obtained during negotiations to assist the SEB in making its final evaluation and report after the negotiations. The objectives of negotiations are essentially the same as those where a single offeror has been selected for final contract negotiation and award; each offeror's correctable weaknesses should be pointed out and corrected during negotiations, using whatever technical and other information is known and which the Government has the right to use. Similarly, negotiations should be conducted to result in a fair and reasonable cost or price. Particular attention shall be given to any instructions which the SSO may have given when he/she directed the multiple contract negotiations.
3. The final contract negotiation process differs from the written and oral discussions previously held with offerors in the competitive range. Discussions have the specific function of obtaining information for evaluation and selection purposes, while the final contract negotiations have the additional function of presenting that information in contractually binding form. For this reason, it is essential that each offeror be brought to the most favorable terms that the negotiation process can produce, including technical and scientific approaches, management arrangements, and estimated costs (or fixed prices where applicable). The prohibition against auction techniques in FAR 15.610(d)(3) is equally applicable to these negotiations.
4. Upon completion of the negotiations and agreement on contract terms, the SEB shall conduct a final evaluation, focusing on a comparative analysis of the contracts negotiated, their relative strengths and remaining weaknesses, their estimated costs and fee(s)/prices and probable costs, and any other factors that might influence the selection. The evaluation must build on the SEB's earlier report and presentation to the SSO which resulted in the decision to have multiple contract negotiations conducted. The evaluation is to determine the effects, if any, of the contract negotiations on the SEB's earlier final Mission Suitability scores as well as on evaluation of Cost, Relevant Experience and Past Performance, and Other Considerations; however, an arithmetic rescoring is not to be accomplished. A summary of the results of the evaluation shall be prepared for presentation to the SSO including any changes in the SEB's assessment of each proposal. The revisions may be expressed in a narrative analysis including an appropriate adjectival rating.
5. a. Upon completing its evaluation of the results of contract negotiations, the SEB shall report its findings to the SSO. This is to be accomplished by oral presentation and supplemental written report. When the Administrator is serving as the SSO, this should be preceded by preliminary presentations to the Field Installation Director and to the Official-in-Charge of the cognizant Headquarters Program Office.
b. The report and presentation shall include the following:
(1) A summary review of the previous report and presentation.
(2) A brief discussion of significant weaknesses and strengths of the companies involved, as reported and presented in the previous report and presentation, with emphasis on key discriminators, if any.
(3) A discussion of how any instructions given by the Source Selection Official were carried out.
(4) The results of the negotiation, and the impact, if any, on the SEB's findings and conclusions in the previous report, including Mission Suitability, Cost, Relevant Experience and Past Performance, and Other Considerations, as appropriate.
(5) Discussions of any matters or areas of substance that arose during the negotiations and that were not present in the proposals or the earlier oral and/or written discussions or BAFO's.
6. The contracting officer should be present and should participate, as appropriate, in the presentation to the SSO. The contracting officer or designee will bring to the presentation copies of the contracts signed by the offerors.
TO:
FROM:
SUBJECT: Source Evaluation Board (SEB) for
Pursuant to Chapter 2 of the NASA Source Evaluation Board Handbook (NHB 5103.6), I hereby designate the following individuals to serve as members of the SEB for the purpose of evaluating proposals received in response to the solicitation for
:
Chairperson:
Name of individual, functional title, and organizational assignment.
Other Voting Members:
Names of individuals, functional titles, and organizational assignments.
SEB Recorder:
Name, functional title, and organizational assignment.
Nonvoting Ex Officio Members:
Name, functional title, and organizational assignment.
The SEB will conduct its business in strict accordance with the provisions of the SEB handbook. The SEB Chairperson is responsible for determining that each SEB member (both voting and nonvoting) is fully conversant with the instructions contained in the handbook. SEB duties will take precedence over other regular duties of the SEB members.
Attention of the SEB Chairperson and each SEB member is particularly directed to Chapter 2 of the SEB handbook which describes the roles of key participants in the SEB process, including cognizant line and staff management, the SEB, and the Source Selection Official (SSO). The importance of the SEB function to agency programs necessitates continual management involvement throughout the evaluation and selection process.
It is emphasized that the SEB report and presentation are the principal tools available to the SSO to perform a comparative analysis for making the final source selection decision and must be presented in sufficient depth to permit the intelligent weighing of alternatives. All proposals will be evaluated and reported in accordance with the SEB handbook. The SEB's written findings will give no consideration to elements which are extraneous to the objectives of this procurement.
Attention of the SEB Chairperson and the SEB is further directed to FAR 15.612(e) and Chapter 2 of the SEB handbook which prohibit the disclosure of information to anyone who is not also participating in the same evaluation proceedings. After receipt of proposals, all information contained in the proposals submitted for evaluation will be protected and will be made available only to the members (voting and nonvoting) of the SEB and to properly designated committees and panels on a need-to-know basis. The right to information on a need-to-know basis does not extend to the normal chain of supervision of any member of the SEB or to any individual having technical responsibility for the effort being evaluated except as specifically approved by the SEB Chairperson on a case-by-case basis. Individuals so designated by the SEB Chairperson will be notified, in writing, of the privileged character of proposal information.
Signature of Designating Date
Official
1. I, the undersigned, a participant in the Source Evaluation Board (SEB) proceedings for the competition of the
contract, certify that I will not discuss or reveal any information concerning these SEB proceedings to anyone who is not also participating in the same SEB proceedings, and then only to the extent that such information is required in connection with such proceedings on a need-to-know basis.
2. I further agree that at any time I discover that I have an interest in, or connection with, a company submitting a proposal for evaluation by the Board or advisory committee on which I serve, I shall promptly report, in writing, the fact of my interest or connection, and the nature of it, to the person who has appointed me to the Board or advisory committee, through the SEB Chairperson. I recognize that a reportable interest or connection includes the following:
a. Ownership of a company's securities by myself or my spouse.
b. A close family relationship to an official of a company submitting a proposal or participating as a subcontractor.
c. Any other interest in or connection with a company which might tend to subject NASA to criticism on the basis that such interest or connection would impair my objectivity in participating on the Board or advisory committee of which I am a member.
3. I further certify that I have read and understand NHB 1900.1, "Standards of Conduct for NASA Employees."
4. In addition, I fully realize that any breach by me of my obligation to safeguard and not disclose to unauthorized persons any information made available to me concerning the evaluation may result in appropriate disciplinary action, provided for by law or regulation, being taken against me by duly constituted authority.
Signature Date
I. INTRODUCTION
A. In preparing the SEB report and presentation, emphasis should be placed on substance. This Appendix C provides guidance on content and format of the SEB report and the oral presentation to the SSO. This guidance is designed to be appropriate for most procurements. If a specific procurement has peculiarities which cause the proposed format to be impractical, the SEB Chairperson may alter it in any manner that does not detract from the substance.
B. The findings of the SEB are presented in two parts, the written SEB report and an oral presentation. This Appendix sets forth the minimum requirements. The detail and use of additional schedules or other information are, of course, governed by the nature and scope of the subject being presented.
C. The SEB shall approve the written report and the graphic material to be used for the accompanying oral presentation. Viewgraph presentations are appropriate where the nature of the presentation permits. Copies of visual aids to be utilized for the oral presentation should be separately bound in a folder identified as "Briefing Charts" to accompany the written report.
D. A "Glossary of Terms" should be included in the SEB Report when appropriate to define acronyms or abbreviations.
E. Copies of the SEB report and related briefing charts shall be serially numbered and controlled by the Recorder and may be distributed or disclosed only to persons having responsibilities relating to the specific source evaluation proceeding involved, except as may otherwise be approved by the Administrator or designee.
II. WRITTEN SEB REPORT
A. Description of the Requirement
1. Procurement Description. Provide a narrative description of the technical requirement being procured together with its scientific objectives. Explain any follow-on effort which has been planned or for which program approval will be requested. Describe the relationship with other efforts in process or planned. Explain any particular technical complexities which had an important effect on the solicitation or the evaluation of proposals. Include a table of contents in the report.
2. Funding. State the funding applicable to the effort including:
a. The Government estimate for the procurement, if feasible.
b. The estimated amount to be obligated under the basic contract.
c. The estimated amount to be obligated under each priced option.
d. The estimated cost of any follow-on effort to be procured under separate contract, e.g., subsequent phases.
3. Procurement Approach
a. State the date that the applicable procurement plan was approved.
b. Discuss any special procurement considerations which applied to the procurement being evaluated, e.g., the use of phased procurements.
c. Explain how the use of the type of contract approved in the procurement plan will advance NASA objectives; give reasons and rationale for the selection of contract type, including the applicability or inapplicability of the various incentive concepts (award fee, cost, performance, schedule, and multiple).
B. SEB Roster and Chronology of Events
1. SEB Roster. Include the SEB designation letter and any changes to it. As a supplement, provide the names, functional titles, and organizational assignments of the chairpersons of any committees and panels used by the SEB.
2. Chronology. Provide a chronology of major events connected with the source evaluation, such as:
a. The date(s) the source list and the Statement of Work were received for consideration by the SEB.
b. The dates the evaluation factors, subfactors, elements, and their definitions for incorporation into the solicitation, the relative weights of factors and the qualification standards, if applicable, were approved.
c. The date(s) the SEB with the assistance of the cognizant procurement and program or project offices approved the source list and the RFP.
d. The date the RFP was issued.
e. The date and place of any preproposal conference.
f. The date(s) of any RFP amendment(s).
g. The due date for receipt of proposals.
h. The date(s) and disposition of any late proposal(s) received.
i. The date the SEB was convened for proposal evaluation.
j. The date of the competitive range determination.
k. The dates and places of discussions with each offeror in the competitive range.
l. The date the request for BAFO's was issued.
m. The common cutoff date for conclusion of discussions and receipt of BAFO's.
n. The date the SEB completed its findings.
o. The date of the presentation to the SSO.
3. Sources. Provide a composite list, in alphabetical order, of all sources solicited and sources submitting proposals by company name and address. The list should be footnoted to explain any code used. Include a list of team members and subcontractors for those companies that submitted proposals.
C. Evaluation and SEB Findings
1. Factors, Subfactors, Elements, and Weights
a. Qualification Standards. State any specific qualification standards included in the RFP, and explain why each was necessary.
b. Evaluation Factors, Subfactors, and Elements. State the evaluation factors used in the evaluation and their relative order of importance. State the Mission Suitability subfactors and elements, if any, their definitions, and the weights assigned to each. Explain the rationale for the proportionate weights assigned to each subfactor and element, if any, set forth relevant excerpts from the RFP which describe the evaluation factors, subfactors, and elements, if any, and their relative importance.
2. Evaluation Process
a. Discuss assignments made to committees and panels. Identify the factors, subfactors, and/or elements assigned for review and the rationale for assignment. Include any individual committee report necessary to understand the SEB results.
b. Discuss the adjective rating and numerical scoring methods and techniques used by the SEB.
c. Discuss the procedure used for the determination of the competitive range.
d. Describe the steps taken in verifying the offeror's relevant experience, past performance, and current capabilities, e.g., plant visits, customer checks, and audit reports.
e. Provide a summary of written questions common to all offerors and specific written questions addressed to particular offerors.
f. State the common cutoff date for conclusion of discussions and receipt of BAFO's.
g. Include an exposition of the SEB proceedings that is sufficient to verify proper procedures were followed and sufficient to bring out any procedural irregularities that might exist. Procedural problem areas, if any, are to be covered specifically.
h. Include copies of all letters addressed to or from the SEB regarding the SEB proceedings.
3. Initial Evaluation Findings
a. Summary Schedules. Provide the following schedules:
(1) A single schedule listing all proposals in descending order of scores received for Mission Suitability stating the assigned score for each subfactor and weighted element. Each proposal should also be classified with the appropriate adjectival rating which indicates the SEB's composite appraisal of the Mission Suitability evaluation. In addition, the appropriate summary or adjectival rating for Cost, Relevant Experience and Past Performance, and Other Considerations should be included as a part of this schedule.
(2) Provide summary charts of what the SEB considers to be the significant discriminators.
b. Statement of Findings
(1) Discuss briefly each proposal determined to be unacceptable and the rationale for the decision.
(2) Discuss each acceptable proposal in descending order of Mission Suitability scores. Include each offeror's estimated cost or price providing a breakdown by element (labor, material, subcontracts, overhead, G&A, and fee); provide the SEB's analysis and evaluation of the adequacy (for all offerors, including those not within the competitive range), realism (for all offerors determined to be within the competitive range), and most probable cost (for at a minimum, each proposal within the competitive range) of each respective cost proposal. Discuss the SEB's evaluation of Relevant Experience and Past Performance and Other Considerations. Include a discussion of the following considerations:
(a) Evaluation of the proposal as related to each factor, subfactor and element and to the approved evaluation plan in sufficient detail to permit examination of the findings of each evaluation phase and to trace discriminators to the final results. Where changes in rating or scoring have occurred during the process, a logical visible thread of the rationale for such changes in rating or scoring should be provided.
(b) Provide a discussion of the major and minor strengths and weaknesses of the proposal with an estimate of the potential for correction of weaknesses identified. Clearly list the strengths and weaknesses and identify them as major or minor, in addition to the narrative, to provide clear traceability throughout the evaluation process.
(c) Provide an analysis of key personnel.
(d) Discuss significant changes in the proposal, including the elimination of any correctable weaknesses, that would have to be negotiated after selection with a discussion of the negotiation cost objectives associated with those changes.
(e) Discuss the evaluation of the Cost factor, including the effects of the proposed cost on the technical and management effort.
(f) Provide a best estimate of the probable cost of performance, if selected, together with an indication of the confidence in the SEB's estimate.
(g) Provide any information or analysis that would be helpful to the SSO in determining the impact of cost in making a selection decision.
(h) Provide information to reflect the offeror's financial capability to perform the contract effort. A listing of possible information includes:
(i) Complete name and location of the organizational element proposing the effort.
(ii) Complete name and location of the parent corporation, if any.
(iii) Place or places of performance of the proposed effort.
(iv) Recent history of sales (of the particular division or entity involved), by customer, including industry and Government customers.
(v) Sales projections, by customer, for the period involved in the procurement.
(vi) Recent history of earnings of the division or entity involved in the procurement, if available.
(i) Present a brief analysis of the fee arrangement including any incentive arrangements proposed, e.g., how the rewards would be earned or lost, the benefit to NASA, and any changes to be sought in negotiations (target minimum or maximum fee levels, sharing formula, or ceiling) which will improve the coverage of the incentive toward the attainment of the NASA objectives. Indicate whether, and for what reason, any offeror took exception to or suggested an alternative to the arrangement contemplated in the RFP. Provide an analysis of the importance the SEB attaches to such an exception or alternative.
(j) Provide detailed information on Relevant Experience and Past Performance. Provide information on the offeror's response to the RFP requirements for data in this category, the evaluation of the data by the SEB, the sources or references contacted, etc.; summarize conclusions and highlight significant accomplishments or failures. Provide the rationale for the adjectival rating of the SEB.
(k) Discuss the evaluation of Other Considerations and the rationale for the adjectival assessment of each subfactor under this factor.
4. Competitive Range Determination. Discuss the competitive range determination and the rationale for the decision, summarizing the evaluation findings that provided the basis for the decision.
5. Written and/or Oral Discussions
a. Summarize the content of the written and/or oral discussions, the personnel included in the proposal who were invited to participate, and the list of attendees, including both Government and offeror personnel.
b. Discuss the time allotted for receipt of BAFO's and each proposal revision received.
6. Final Evaluation Findings. Summarize the BAFO revisions and provide a discussion of the evaluation of the revised proposals and any resultant changes in rating or scoring. Provide the rationale for any changes. Clearly identify from the previous list of major and minor strengths and weaknesses any revisions, additions, and/or deletions to provide clear traceability throughout the evaluation process.
7. Oral Presentation
a. The SEB Chairperson is normally responsible for conducting the presentation to the SSO. It is the Chairperson's function to convey concisely and accurately the results of the SEB deliberations to permit an informed and objective selection of the best source for the particular procurement.
b. As a general rule, the SEB Chairperson's prepared oral presentation should not exceed an hour to be followed by a question and answer period. Copies of the viewgraphs to supplement the SEB report forwarded to the SSO are to be available to those attending the oral presentation. Relevant backup material is to also be available at the presentation.
c. The main thrust of the oral presentation is to focus upon issues and problems identified by the SEB's findings and to highlight the reasonable alternative choices available to the SSO. This presentation must include an explanation of any applicable qualification standards; evaluation factors, subfactors, and elements; the major strengths and weaknesses of the offerors; the Government estimate, if applicable; the offerors' proposed cost/price; the probable cost; the proposed fee arrangements; the results of written and/or oral discussions; the BAFO's; the final evaluation findings; and the final adjectival ratings and scores. These aspects of the report are central to its meaning, and must be reviewed in the oral presentation despite the redundancy to the written report.
d. The presentation shall clearly indicate any discriminators. This requires the presentation of scores in enough detail to provide an adequate basis for the SSO to assess the validity of the judgments made by the SEB. This detail shall extend at least to all levels of subfactors and numerically scored elements.
e. A suggested progression of charts follows. Brevity and understandability are key. Charts should highlight the significant aspects of the comprehensive written report. They should prompt discussion. Sample charts are not included in this handbook since these tend to lead toward unnecessary or even inappropriate standardization. As appropriate, the Associate Administrator for Procurement will, from time to time, disseminate sample charts illustrative of approaches or techniques of exceptional merit and usefulness.
(1) Identification of the Procurement. This chart should identify the installation, the nature of the services or hardware to be procured, some quantitative measure including the Government estimate for the procurement, and the kind of contractual arrangement planned. Detailed objectives of the procurement should be avoided.
(2) Background. This item is useful to identify any earlier phases of a phased procurement or, as in the case of continuing support services, to identify the incumbent and any consolidations or proposed changes from the existing structure.
(3) Evaluation Factors, Subfactors, and Elements. An explanation of any qualification standards and the evaluation factors, subfactors, and, if utilized, elements, along with assigned weights, is an important part of any presentation of SEB findings. The relative order of importance of the evaluation factors and, within Mission Suitability, the numerical weights of the subfactors and, if utilized, numerically scored elements should also be presented. The adjectival scoring system utilized as an aid in evaluation of Mission Suitability should be presented.
(4) Sources. This chart will indicate the number of potential offerors solicited, the number of potential offerors expressing interest, e.g., attendance at a preproposal conference, and the identification of offerors submitting proposals in response to the solicitation. Small businesses, small disadvantaged businesses, and women-owned businesses can be identified here. In addition, a chart must be included for each offeror indicating the Chief Executive Officer, the location of the offeror, some representative products, and similar information for each major subcontractor proposed.
(5) Summary of Findings. The summary chart shall, as simply as possible, list the initial Mission Suitability ratings and scores, the final Mission Suitability ratings and scores, the offerors' proposed costs/prices, and the SEB's assessment of the probable costs. In addition, any clear discriminator, problem, or issue which could have an effect upon the selection should be introduced at this time. The determination of competitive range must be addressed here.
(6) Strengths andWeaknesses of Offerors. These charts can be among the most valuable presented, and its preparation could initially assist the SEB Chairperson in distilling the essence of the SEB's findings. Certain guidelines should be followed in the preparation of these charts:
(a) Only the major strengths and weaknesses of individual offerors should be selected for presentation. The significance of strengths and weaknesses is lost when exhibited in long lists without distinction as to importance.
(b) The strengths and weaknesses should be directly related to the evaluation factors, subfactors,and elements.
(c) Strengths and weaknesses should be as clear as charting techniques and limitations will permit. Care should be taken to avoid an analysis of strengths and weaknesses based totally on numerical scores. This practice limits the SEB and the SSO in their evaluation and selection. For example, not all major weaknesses could be considered of equal value--some are correctable during negotiations and some are not.
(d) It is im portant to indicate the significance of major weaknesses. Can they be corrected during negotiations? If so, at what cost?
(e) The results and impact, if any, of written and/or oral discussions and BAFO's on ratings and scores must be clearly indicated.
(f) Key personnel charts must be included in the presentation, providing a list of the key personnel for each offeror together with, as appropriate, their educational background, general background, and any pertinent details applicable to this procurement.
(7) Final Mission Suitability Ratings and Scores. This chart is to summarize, as simply as the material will permit, the evaluation subfactors and elements, the maximum points achievable, and the scores of the offerors in the competitive range.
(8) Final Cost Evaluation. This chart summarizes probable costs associated with each offeror including proposed fee arrangements. The data should be as accurate as possible; therefore, SEB adjustments to achieve comparability should be shown. Further, the presentation of this chart should include the measure of confidence the SEB has in the costs of the individual offerors, noting the reasons for low or high confidence.
(9) Relevant Experience and Past Performance. This chart should reflect the summary conclusions. This information is to be supported and amplified by specific case data with particular emphasis on exemplary or inferior performance and its potential bearing on the instant procurement.
(10) Other Considerations. This chart lists and discusses the Other Considerations addressed in the RFP, providing an appropriate adjectival rating for each of the Other Considerations subfactors.
(11) Special Interest. This chart should include only information of special interest to the SSO that has not been discussed elsewhere, e.g., procedural errors or other matters that could have an effect on the selection decision.