49 C.F.R. Appendix D to Part 236—Independent Review of Verification and Validation
Title 49 - Transportation
(a) What is the purpose of this appendix? This appendix provides minimum requirements for independent third-party assessment of product safety verification and validation pursuant to subpart H of this part. The goal of this assessment is to provide an independent evaluation of the product manufacturer's utilization of safety design practices during the product's development and testing phases, as required by the applicable railroad's RSPP, the product PSP, the requirements of subpart H of this part, and any other previously agreed-upon controlling documents or standards. (b) What general requirements apply to the conduct of third party assessments? (1) The supplier may request advice and assistance of the reviewer concerning the actions identified in paragraphs (c) through (g) of this appendix. However, the reviewer should not engage in design efforts, in order to preserve the reviewer's independence and maintain the supplier's proprietary right to the product. (2) The supplier shall provide the reviewer access to any and all documentation that the reviewer requests and attendance at any design review or walkthrough that the reviewer determines as necessary to complete and accomplish the third party assessment. The reviewer may be accompanied by representatives of FRA as necessary, in FRA's judgment, for FRA to monitor the assessment. (c) What must be done at the preliminary level? The reviewer shall evaluate with respect to safety and comment on the adequacy of the processes which the supplier applies to the design and development of the product. At a minimum, the reviewer shall compare the supplier processes with acceptable methodology and employ any other such tests or comparisons if they have been agreed to previously with FRA. Based on these analyses, the reviewer shall identify and document any significant safety vulnerabilities which are not adequately mitigated by the supplier's (or user's) processes. Finally, the reviewer shall evaluate the adequacy of the railroad's RSPP, the PSP, and any other documents pertinent to the product being assessed. (d) What must be done at the functional level? (1) The reviewer shall analyze the Preliminary Hazard Analysis (PHA) for comprehensiveness and compliance with the railroad's RSPP. (2) The reviewer shall analyze all Fault Tree Analyses (FTA), Failure Mode and Effects Criticality Analysis (FMECA), and other hazard analyses for completeness, correctness, and compliance with the railroad's RSPP. (e) What must be done at the implementation level? The reviewer shall randomly select various safety-critical software modules for audit to verify whether the requirements of the RSPP were followed. The number of modules audited must be determined as a representative number sufficient to provide confidence that all unaudited modules were developed in compliance with the RSPP. (f) What must be done at closure? (1) The reviewer shall evaluate and comment on the plan for installation and test procedures of the product for revenue service. (2) The reviewer shall prepare a final report of the assessment. The report shall be submitted to the railroad prior to the commencement of installation testing and contain at least the following information: (i) Reviewer's evaluation of the adequacy of the PSP, including the supplier's MTTHE and risk estimates for the product, and the supplier's confidence interval in these estimates; (ii) Product vulnerabilities which the reviewer felt were not adequately mitigated, including the method by which the railroad would assure product safety in the event of a hardware or software failure (i.e., how does the railroad assure that all potentially hazardous failure modes are identified?) and the method by which the railroad addresses comprehensiveness of the product design for the requirements of the operations it will govern (i.e., how does the railroad assure that all potentially hazardous operating circumstances are identified? Who records any deficiencies identified in the design process? Who tracks the correction of these deficiencies and confirms that they are corrected?); (iii) A clear statement of position for all parties involved for each product vulnerability cited by the reviewer; (iv) Identification of any documentation or information sought by the reviewer that was denied, incomplete, or inadequate; (v) A listing of each RSPP procedure or process which was not properly followed; (vi) Identification of the software verification and validation procedures for the product's safety-critical applications, and the reviewer's evaluation of the adequacy of these procedures; (vii) Methods employed by the product manufacturer to develop safety-critical software, such as use of structured language, code checks, modularity, or other similar generally acceptable techniques; and (viii) Method by which the supplier or railroad addresses comprehensiveness of the product design which considers the safety elements listed in paragraph (b) of appendix C to this part. [70 FR 11107, Mar. 7, 2005]
Title 49: Transportation
PART 236—RULES, STANDARDS, AND INSTRUCTIONS GOVERNING THE INSTALLATION, INSPECTION, MAINTENANCE, AND REPAIR OF SIGNAL AND TRAIN CONTROL SYSTEMS, DEVICES, AND APPLIANCES
Subpart H—Standards for Processor-Based Signal and Train Control Systems
Appendix D to Part 236—Independent Review of Verification and Validation
