FAA Has Taken Steps To Validate Its Air Traffic Skills Assessment Test but Lacks a Plan To Evaluate Its Effectiveness

What We Looked At FAA relies on its highly-trained Air Traffic Control Specialists (ATCS) to safely and efficiently manage the flow of air traffic across the National Airspace System. In 2012, FAA undertook a review of the ATCS hiring process that identified the need to update its ATCS pre-employment test. In response, FAA deployed the Air Traffic Skills Assessment (ATSA) test in November 2016. The National Defense Authorization Act for Fiscal Year 2020 mandated that the Office of Inspector General (OIG) assess the assumptions and methodologies used to develop the ATSA. Accordingly, we initiated this audit to assess FAA’s (1) validation of the assumptions and methodologies used to develop the air traffic controller pre-employment test and (2) administration of the test and its plans to evaluate the test’s effectiveness in identifying successful ATCS candidates.   What We Found FAA has taken steps to validate the assumptions and methodologies used to develop the ATSA. FAA worked with a human resource consulting organization, APTMetrics, to identify the critical skills, abilities, and other characteristics (SAO) required to perform important ATCS responsibilities. FAA then worked with APTMetrics, NATCA, and various subject matter experts to ensure that the ATSA evaluates the SAOs required for entry level ATCS. Also, the ATSA identifies sufficient applicants to meet FAA’s trainee hiring goals but the Agency has not established a plan for evaluating the ATSA’s effectiveness. As a result, it is unclear whether the test is effective at helping the Agency identify successful ATCS candidates. Furthermore, FAA’s lengthy hiring process and COVID-19 restrictions have slowed FAA’s onboarding and training activities for newly hired ATCS. As a result, few applicants who have taken the ATSA and were selected as trainees have completed their training and obtained certification.   Our Recommendations FAA concurred with our recommendation that the Agency evaluate the ATSA’s effectiveness in identifying successful ATCS candidates. We consider the recommendation resolved but open pending completion of the planned actions.