Accepting your FSTD - On Site Acceptance (OSA)

Introduction

In 2020 we took a look at accepting your Flight Simulation Training Device (FSTD) at the Training Device Manufacturer’s (TDM) facility, Factory Acceptance (FA). Following on from that we thought it was about time to complete the process and look at On Site Acceptance (OSA) once the device is at the final location.

What issues need to be considered?

Whereas Factory Acceptance (FA) is, arguably, not actually 100% needed OSA most definitely is. Within the EASA jurisdiction the operator should send the competent authority confirmation that a FSTD being put forward for initial evaluation has been already evaluated by the operator and it meets the relevant standards (See AMC1 ORA.FSTD.200, application part C). The FAA regulations are, if anything, more demanding (FAA 14 CFR Part 60, section §60.15 Initial qualification requirements). As we have talked about in previous blogs, the authorities are becoming increasingly adamant that the operator is responsible to accept FSTDs, not the authorities. So OSA is a critical point in the project.

Acceptance and Evaluation

During OSA though it is not always recognised there are in fact two distinct, though intertwined, activities going on at the same time. One, acceptance, between the TDM and the operator/purchaser. This is a contractual confirmation that the purchaser has got what they ordered, one element being successful qualification. The second, preparation for initial evaluation, is the joint effort between purchaser and TDM to prepare the device for evaluation.

Roles and Responsibilities

OSA is in the phase of the programme where the roles change; it is the phase during which the operator needs to assume the responsibility for the total management of the device. Once Ready For Training is achieved the TDM’s team, apart from any field service representative that may have been negotiated, will be off to the airport as soon as they can. So management of discrepancies, maintenance, scheduling, configuration control and QTG management all need to transition from the TDM to the operator.

In particular it needs to be fully appreciated that the final responsibility to decide if the device is ready to be presented to the competent authority rests with the operator, in particular the Accountable Manager.

Teamwork

As with FA, a successful OSA relies on good cooperation between the TDM and the operator. Indeed if the competent authority arrives to carry-out the initial evaluation and finds that the TDM and operator are at loggerheads, disputing the condition of the FSTD, it will not bode well for the result of the evaluation. The OSA needs to be carried out in an open, honest, environment with mutual respect and trust.

Who Should be In Your On Site Acceptance Team?

Ideally, and preferably, the same team who conducted the FA should also conduct the OSA. Whilst a lot of the testing will be objective (ATMs and QTG) there is still a large element of subjective testing involved. There is nothing more guaranteed to alienate the TDM’s team than having to re-do subjective tuning because a “new” evaluation pilot has a different point of view, it is also an extreme waste of time.

EASA expects the evaluation team to be specified on the application for qualification and their qualifications to be listed. One of these should be a nominated pilot who needs to supply their pilot's licence number.

The TDM will probably send their Project Manager, a QTG specialist and a test engineer/project engineer; some may also send their own evaluation pilot but with the pressure on budgets that is becoming less frequent. They will also normally ensure there is a hardware technician available in case there are failures during this period, often the field service representative fulfils this role.

The operator will need to increase the involvement over and above the team they used for the FA, recognising that some roles may be combined in the operators' organisation, they need to ensure the Accountable Manager, Compliance Manager, Head of Training and Maintenance Manager (if not part of the original FA team) are fully engaged and prepare themselves for the competent authorities evaluation.

Plan Ahead

In the run-up to the competent authority’s visit and subsequent Ready For Training (RFT) there is a lot to do and careful planning is key to success. With a lot of the competent authorities currently struggling to meet the demand for initial and recurrent inspections you will need to apply well in advance of your desired date, hence a robust and realistic schedule is essential. It will need to allow time for the QTG to be run and submitted at least 30 days in advance, give time for maintenance team training, snag clearance.

Effective Communication

Effective communication at all times between your team and the TDM’s people is fundamental to success. The nature of FA means people are often working long shifts around the clock, during weeks and over weekends and so keeping in touch, maintaining shared objectives and a set of defined priorities can be difficult. Daily pre-briefings and debriefings on either side of FA sessions are a common and effective way to exchange information.

What's different to the FA

In the factory your sole focus is on the device itself. Once it is installed the focus needs to be broadened to cover not only the device but the installation as well. In this regard remember it may not only be the TDM you have to carry-out an acceptance with. You will most likely have contracted other suppliers to carry out building adaptations and services supply; make sure these are included in your testing. You should also make sure you confirm the delivery of all deliverables from the TDM, this is often overlooked. In particular check;

• Integration with the building's fire and security systems has been completed.

• Spare parts have been delivered and are the correct ones.

• All technical publications are delivered, of a usable quality and relevant to your FSTD.

• All special tools are delivered and work.

• All software tools are delivered and work.

• Additional tools have been delivered and work, these can include lesson plan editors, briefing/debriefing tools, flight crew monitoring systems.

What Sort of Testing Should We Do During OSA?

As with FA there is no definitive answer to this, however with OSA your prime objectives are to be able to prove to the competent authority that the device meets the applicable standards and make sure the device is suitable for your training needs. Remember to ensure the authority requirements, see above, have been covered. However, we can give some guidelines

• Check all (any) corrections for discrepancies that were found during FA have been corrected.

• Assuming that all the Acceptance Test Manuals (ATM) were run during FA there should be no need to re-run a full set during OSA. However, it is highly recommended that, where discrepancies were raised during the running of ATMs in FA, the whole section of the applicable ATM is re-run to test the corrections (not just the single test that failed).

• Run as many of your lesson plans as you can.

• The final running of the QTG (and production of the Master QTG (MQTG). As we have indicated in previous blogs, it is advisable to ensure the QTGs are run using the delivered tools and by, or at least observed by, the operators' personnel (see our blog on QTG)

As with the FA, the scope and nature of testing for OSA should always be discussed and agreed with the TDM well before the start of OSA, typically after FA when the state of readiness of the FSTD should be known.

Problem Recording and Tracking

As we discussed in the blog on Factory Acceptance the TDM should have provided you with a database system to record and subsequently track all problems identified during acceptance. To work properly, these databases must be accurate in content and kept updated daily. However, as discussed above, the operator needs to have transitioned to their own tracking system by the time of the evaluation. That is not to say that the TDM’s system will not be used anymore, but the operators' system has to become the prime record for device compliance; the TDM’s reference being recorded in the operators' system.

Software and Hardware Configuration Control

Unfortunately, the pace of the modern world does not allow development and testing phases to happen serially. These activities are forced to run in parallel, such that FA testing often happens all day long and then bugs are found, reported, and hopefully fixed overnight. Problems can multiply exponentially if the same software load is being shared across several FSTDs in build and test at the same time. Multiple development and testing loads are required in order to do this parallel working successfully and the only way this can be maintained is through effective configuration control.

Arinc has a standard for this!

As with most aspects of FSTD purchase, qualification and support, Arinc, in the shape of the FSEMC, has produced an excellent guidance document for the whole acceptance process (Arinc report 438, Guidance For Acceptance Of Flight Simulation Training Devices). Section 3.7 covers OSA.

Retention of Payments

We have seen a trend that operators want to withhold a percentage of the sale price against the clearance of any outstanding discrepancies. First caution on this policy is to check the sales contract, to do so you really ought to have a contractual basis on which to do so.

On the face of it, this seems a very sensible precaution but we are not convinced. If this is discussed during contract negotiations and put into the contract we can assure you the TDM will factor the risk of not receiving the payment into the sales price; you will pay more. Moreover, we advise to be realistic, all FSTDs have some issues that will, frankly, never be fully resolved; and it doesn’t actually matter. As long as the FSTD meets the qualification standard and is capable of conducting your training is there really a problem?

Lastly, ask yourself the question that, if you feel you need to have a retention, have you chosen the correct TDM?

How can Sim Ops Help?

Between them, members of the Sim Ops team have literally thousands of hours of amassed experience running and participating in Acceptances all around the world, across every type of FSTD imaginable. We have seen many examples of best (and worst) practices in the industry so our team can help guide you safely through the minefield and avoid many of the pitfalls that await those that simply rush in without careful forethought and planning. Read more about our FSTD acquisition and evaluation services.