The following report was released by the Government Accountability Office on November 13, 2013.

TSA Should Limit Future Funding for Behavior Detection Activities

• 99 pages
• November 8, 2013

Available evidence does not support whether behavioral indicators, which are used in the Transportation Security Administration’s (TSA) Screening of Passengers by Observation Techniques (SPOT) program, can be used to identify persons who may pose a risk to aviation security. GAO reviewed four meta-analyses (reviews that analyze other studies and synthesize their findings) that included over 400 studies from the past 60 years and found that the human ability to accurately identify deceptive behavior based on behavioral indicators is the same as or slightly better than chance. Further, the Department of Homeland Security’s (DHS) April 2011 study conducted to validate SPOT’s behavioral indicators did not demonstrate their effectiveness because of study limitations, including the use of unreliable data. Twenty-one of the 25 behavior detection officers (BDO) GAO interviewed at four airports said that some behavioral indicators are subjective. TSA officials agree, and said they are working to better define them. GAO analyzed data from fiscal years 2011 and 2012 on the rates at which BDOs referred passengers for additional screening based on behavioral indicators and found that BDOs’ referral rates varied significantly across airports, raising questions about the use of behavioral indicators by BDOs. To help ensure consistency, TSA officials said they deployed teams nationally to verify compliance with SPOT procedures in August 2013. However, these teams are not designed to help ensure BDOs consistently interpret SPOT indicators.

TSA has limited information to evaluate SPOT’s effectiveness, but plans to collect additional performance data. The April 2011 study found that SPOT was more likely to correctly identify outcomes representing a high-risk passenger—such as possession of a fraudulent document—than through a random selection process. However, the study results are inconclusive because of limitations in the design and data collection and cannot be used to demonstrate the effectiveness of SPOT. For example, TSA collected the study data unevenly. In December 2009, TSA began collecting data from 24 airports, added 1 airport after 3 months, and an additional 18 airports more than 7 months later when it determined that the airports were not collecting enough data to reach the study’s required sample size. Since aviation activity and passenger demographics are not constant throughout the year, this uneven data collection may have conflated the effect of random versus SPOT selection methods. Further, BDOs knew if passengers they screened were selected using the random selection protocol or SPOT procedures, a fact that may have introduced bias into the study. TSA completed a performance metrics plan in November 2012 that details the performance measures required for TSA to determine whether its behavior detection activities are effective, as GAO recommended in May 2010. However, the plan notes that it will be 3 years before TSA can begin to report on the effectiveness of its behavior detection activities. Until TSA can provide scientifically validated evidence demonstrating that behavioral indicators can be used to identify passengers who may pose a threat to aviation security, the agency risks funding activities that have not been determined to be effective. This is a public version of a sensitive report that GAO issued in November 2013. Information that TSA deemed sensitive has been redacted.

…

The Aviation and Transportation Security Act established TSA as the federal agency with primary responsibility for securing the nation’s civil aviation system, which includes the screening of all passengers and property transported by commercial passenger aircraft. At the more than 450 TSA-regulated airports in the United States, all passengers, their accessible property, and their checked baggage are screened prior to boarding an aircraft or entering the sterile area of an airport pursuant to statutory and regulatory requirements and TSA-established standard operating procedures. BDA, and more specifically, the SPOT program, constitutes one of multiple layers of security implemented within TSA-regulated airports. According to TSA’s strategic plan and other program guidance for the BDA program released in December 2012, the goal of the agency’s behavior detection activities, including the SPOT program, is to identify high-risk passengers based on behavioral indicators that indicate “mal-intent.” For example, the strategic plan notes that in concert with other security measures, behavior detection activities “must be dedicated to finding individuals with the intent to do harm, as well as individuals with connections to terrorist networks that may be involved in criminal activity supporting terrorism.”

TSA developed its primary behavior detection activity, the SPOT program, in 2003 as an added layer of security to identify potentially high-risk passengers through behavior observation and analysis techniques. The SPOT program’s standard operating procedures state that BDOs are to observe and visually assess passengers, primarily at passenger screening checkpoints, and identify those who display clusters of behaviors indicative of stress, fear, or deception. The SPOT procedures list a point system BDOs are to use to identify potentially high-risk passengers on the basis of behavioral and appearance indicators, as compared with baseline conditions where SPOT is being conducted. A team of two BDOs is to observe passengers as they proceed through the screening process. This process is depicted in figure 1.

According to TSA, it takes a BDO less than 30 seconds to meaningfully observe an average passenger. If one or both BDOs observe that a passenger reaches a predetermined point threshold, the BDOs are to direct the passenger and any traveling companions to the second step of the SPOT process—SPOT referral screening. During SPOT referral screening, BDOs are to engage the passenger in casual conversation—a voluntary informal interview—in the checkpoint area or a predetermined operational area in an attempt to determine the reason for the passenger’s behaviors and either confirm or dispel the observed behaviors. SPOT referral screening also involves a physical search of the passenger and his or her belongings. According to TSA, an average SPOT referral takes 13 minutes to complete. If the BDOs concur that a passenger’s behavior escalates further during the referral screening or if other events occur, such as the discovery of fraudulent identification documents or suspected serious prohibited or illegal items, they are to call a LEO to conduct additional screening—known as a LEO referral—who then may allow the passenger to proceed on the flight, or may question, detain, or arrest the passenger. The federal security director or designee, regardless of whether a LEO responds, is responsible for reviewing the circumstances surrounding a LEO referral and making the determination about whether the passenger can proceed into the sterile area of the airport.

…

Available Evidence Does Not Support Whether Behavioral Indicators Can Be Used to Identify Aviation Security Threats

Meta-analyses and other published research studies we reviewed do not support whether nonverbal behavioral indicators can be used to reliably identify deception. While the April 2011 SPOT validation study was a useful initial step and, in part, addressed issues raised in our May 2010 report, it does not demonstrate the effectiveness of the SPOT indicators because of methodological weaknesses in the study. Further, TSA program officials and BDOs we interviewed agree that some of the behavioral indicators used to identify passengers for additional screening are subjective. TSA has plans to study whether behavioral indicators can be reliably interpreted, and variation in referral rates raises questions about the use of the indicators by BDOs.

Share this: