Analytics That Allow You To See Beyond The Cloud. By Alex Huang, Ph.D., Head of Aviation Analytics Services, The Weather Company, an IBM Business

Transcription

1 Analytics That Allow You To See Beyond The Cloud By Alex Huang, Ph.D., Head of Aviation Analytics Services, The Weather Company, an IBM Business

2 Table of Contents 3 Ways Predictive Airport Analytics Could Save Airlines Millions 2 Problem 3 Challenges 4 Predictive Airport Analytics 5 Solutions 6 Airport Congestion Prediction 6 Airport Configuration Prediction 8 Airport Taxi Time Prediction 11 Predictive Airport Analytics in Summary 13 1

3 DFW Predictive Airport Analytics in Fusion 3 Ways Predictive Airport Analytics Could Save Airlines Millions Unanticipated changes in weather and associated impacts on airport operations can result in significant delays and costs. As such, airline operators with early insight into forecast weather impacts on airport operations can save millions of dollars by optimizing flight plans and fuel contingencies. The Weather Company, an IBM Business (Weather) makes this possible by combining big data analytics and machine learning methods with highresolution weather forecasts and current flight data to skillfully predict airport operational conditions for a 12-hour period - up to five times sooner than previously possible. This white paper will focus on how Predictive Airport Analytics provides opportunities for airlines to improve advanced decision-making. There are three types of analytics on this journey to organizational optimization: 1. Descriptive Analytics 2. Predictive Analytics 3. Prescriptive Analytics Descriptive analytics is the reporting of current or historic facts. For example, descriptive analytics could tell us which way the wind is blowing or provide historical data such as average taxi times. This type of analytics provides the foundation for airline operations today. The second type of analytics is predictive, which organizes and then predicts possible outcomes based on descriptive data and data modeling. Predictive analytics recognizes patterns and their impact on future operations. 3 Types of Predictive Airport Analytics include: 1. Airport Congestion 2. Runway Configuration 3. Taxi Times For example, if we know that an airport will change its runway configuration when the wind changes direction, and we know the wind will change to the north after noon, then we can predict a runway configuration change. Runway configuration changes cause approach patterns and taxi times to change. These future changes and the subsequent delay propagation have long reaching impact on airline operations and provide a significant opportunity to lower costs if anticipated earlier. The third type of analytics is prescriptive analytics, which provides automated recommendations or actions to mitigate the predicted impact. 2

4 Problem Most airlines concentrate a majority of their flight operations on just a few hub airports. Figure 1 demonstrates the airport hub distribution for the top 14 US carriers. In fact, among the top three airlines, on average, 41 percent of each carrier s departures occur from just four hubs. As a result, when weather mixes with congestion at these hub airports, the impacts can be felt long after the original problem has been resolved. Today, airlines rely on government advisories such as the FAA s Ground Delay Program (GDP) to understand potential impacts to airport capacity. GDPs are implemented to control air traffic volume at airports where the projected traffic demand is expected to exceed the airport s acceptance rate. It should come as no surprise that the most common reason for changes in acceptance rate is weather. Airline operations are often notified of changes only one to two hours in advance of flight arrivals and departures, causing a cascade of fuel contingency miscalculations and delays. Figure 1. Airline Departure Share from Hub Airports 3

5 Challenges Airline dispatchers must calculate the right amount of fuel for each flight to ensure performance and minimize cost. Since delays and congestion are not always understood in advance, airlines often over or under-fuel their planes. Over-fueling causes excessive fuel burn due to carrying extra weight of the fuel itself. In the most costly scenarios, flights are forced to return to gates or divert to alternative airports due to insufficient fuel reserves. When unforeseen events such as excessive taxi time or airborne holding occur due to congestion, airlines can face substantial expenses. Unnecessary fuel costs are just the tip of the iceberg. Air Traffic Control (ATC) coordinators are responsible for the traffic flow at all major US airports. They assess operational conditions, examine real-time impact to flight operations, and determine strategies to avoid hazardous situations. When a major airport experiences extended departure queues, ATC coordinators estimate whether the queue is temporary or will last for an extended period of time since the actions are vastly different for the two scenarios. Coordinating with dispatchers, ATC coordinators also need to know when passengers will miss downstream connections. With limited knowledge about future airport operational condition (such as runway configuration, airport congestion, and taxi times) dispatchers, schedulers and coordinators are often forced into guessing future conditions based on experience. As a result, they regularly use overly conservative estimates. 4

6 Predictive Airport Analytics While delays cannot be avoided, they are predictable. Airport congestion level, runway configurations and taxi times can be anticipated far in advance of changes; allowing airlines to mitigate the impact of changing airport condition. For this to be effective, airlines must use predictive analytics to prescribe operational efficiency. Predictive analytics provide a forecast of future events (e.g., airport capacity, runway configuration and airport taxi time) based on historical data describing the behaviors of such events. These forecasts provide actionable insights for airlines to understand future Key Performance Indicators (KPIs) at major hub airports. Focusing on three areas of airport analytics, airlines can influence critical decision-making: 1. Airport Congestion 2. Runway Configuration 3. Taxi Time Predictive Airport Analytics are driven by a combination of machine-learning models, decision-trees, and queuing models. It is available via The Weather Company s airline decision support platform, Fusion, as well as via an API. The Weather Company Weather Data Figure 2. Illustrates the key components and process flow for Predictive Airport Analytics A predictive airport analytics model is created leveraging supervised machine-learning algorithms. It is trained and built on historical weather, flight and airport operational data. The accuracy of the predictive analytics models depends on two key factors: 1. Formulation and accuracy of the predictive model 2. Precision of the weather forecast As shown in Figure 2, both flight data and weather forecast data are two of the critical components in forecasting future airport operational conditions. 5

7 The formulation of the predictive model needs to account for key elements of an airport s operation. Once the model is validated and back tested, proprietary numerical weather predictions are used in conjunction with flight and airport data, synthesized from several different sources. The precision of the weather forecast is also imperative to the accuracy of the predictive model and adapts as the weather changes and real-time flight data reflects decisions made by airlines. Solutions The Weather Company leverages a proprietary, high-resolution, precise numerical weather model and backs this up with data science to predict airport congestion, runway configuration, and taxi time for a rolling 12-hour period. AIRPORT CONGESTION PREDICTION Airport congestion is a leading cause of enroute and ground delays, especially when it is unanticipated. The Weather Company airport congestion predictions provide early insight into future congestion, empowering decision-makers to take action to reduce the impact on operations. The 12-hour Airport Congestion Prediction comprises the following key components: 1. Airport Capacity Estimation 2. Future Flight Demand Estimation 3. Demand/Capacity Imbalance Estimation The first component, airport capacity estimation, determines airport capacity based on historical weather impact, airport arrival rates (AAR), airport departure rates (ADR), flight demand, as well as several other proprietary factors. A rule-based decision-tree model is used to derive future AAR and ADR. The second component, flight demand estimation, is used to identify flight demand based on the latest Estimated Time of Arrival (ETA) and Estimated Time of Departure (ETD) for flights in the given period. Once an airline and/or air traffic control modifies ETA and ETD, demand will reflect the latest updates. The third component, demand capacity imbalance estimation, uses a single server queuing model to assess flight demand and airport capacity imbalance. The model assesses whether the airport capacity can handle the cumulative flight demand for each time interval. The cumulative flight demand in each time interval is defined as the sum of the scheduled flight demand for that time interval, plus the flight demand that is not fulfilled from the previous time interval. As a result, airlines equipped with the analytics can have a clear indicator of the airport departure and arrival congestion. The model refreshes every 15 minutes with updated demand and congestion data, taking into account airlines that are taking action as a result of the predictions. For the example in Figure 3, the airport congestion prediction chart for SFO shows both cumulative flight demand and airport capacity values for each time period. On the top of the chart, a traffic light color system provides a quick glance into the congestion level at the airport. The light green line indicates a low congestion level at SFO and the scale is 6

8 shown to the right of the y-axis. A measure of 10 indicates that the airport is expected to be at its full capacity (100% utilization of the capacity). In Figure 3, the model predicts SFO will have departure flow congestion between 19:00 UTC and 22:00 UTC. Delays are likely to occur because flight demand during this time period exceeds the airport capacity. Airlines and ATC might choose to delay or cancel flights to avoid airborne holding or extended taxi times. Figure 3. SFO Airport Congestion Prediction Chart in Fusion Airport congestion prediction performance is assessed using a zone accuracy concept. If the predictions and the observations both fall within the same zone, then the prediction is considered to be accurate. The congestion level is divided into three zones: 1. Under 80% 2. Between 80% and 100% 3. Above 100% The airport congestion prediction performance during the three time durations are shown below. 7

9 As shown in the table above, the average accuracy rate using the zone accuracy concept for the airport congestion prediction is 94%. Among the 77 airports, the 10th percentile still has an accuracy rate of 80%. This shows that, on average, 69 airports with airport congestion prediction analytics maintain an accuracy rate higher than 80%. In addition, major airports hourly prediction accuracy rates between 3/18/2015 and 3/23/2015 remain consistent for all of the prediction time horizons (as shown in Figure 4). This implies the prediction results can be trusted up to 10 to 12 hours out. Figure 4. Airport Congestion Prediction Analytics Accuracy Rates at Major Hub Airports AIRPORT CONGESTION PREDICTION Airport configuration prediction provides decision-makers with early insight into the layout and timing of runway configuration changes, which can aid flight and approach planning. Airport configuration prediction uses a two-step machine-learning model. The ultimate goal is to predict an airport s runway configuration layout and the timing for any change in the configuration. The prediction is based on historical ATC behavior including predictable changes based on the weather. Since each airport has unique operational behaviors, individual models have been built for each of the 29 US airports covered. In general, models are trained using the last two full years of historical data. In cases where airports have had permanent runway layout changes during the two-year time span, fewer months have been used to ensure true historical behavior matches the latest runway configurations. Temporary runway closures most often communicated using NOTAMS are not presently included in the modeling. 8

10 Figure 5. BOS Airport Runway Configuration Prediction Chart in Fusion Figure 5 shows an airport runway configuration prediction in WSI s Fusion. The representation is divided into two parts. The top part provides the actual runway layout in graphical form and displays runway usage for the selected hour. Wind direction forecast is also shown. The bottom part of the display includes a table of runways for each hourly interval. Different color codes indicate the type of operations (departure, arrival, dual, or no use) and possibility of the runway s use. As the example in figure 4 indicates, BOS is using both Runway 04L and 04R for arrivals and departures, and Runway 09 for departures only between 19:00 UTC and 23:00 UTC. Starting from 00:00 UTC, the configuration is going to change to the opposite direction using Runway 22L and 27 for arrivals and Runway 22R for departures. Airlines can use this information to adjust flight patterns to minimize holding time and optimize fuel contingencies. For airport runway configuration prediction, True Positive (TP) rate and False Positive (FP) rate are monitored. That is, for each runway at an airport, if the model correctly predicts the runway s usage (arrival or departure or no use), then it is considered to be a hit. The hit rate at each airport is determined using the following formula: True Positive (TP) Rate = (Number of Runways Correctly Classified as in Use) (Total Number of Runways that were in Use) A False Positive (FP) rate is also maintained to examine the chance of the model falsely predicting a runway s use. 9

11 The FP rate is calculated using the following formula: False Positive (FP) Rate = (Number of Runways Incorrectly Classified as not Used) (Total Number of Runways not in Use) Figure 6 and Figure 7 shows the TP and FP rates for arrival operations at Dallas Fort-Worth (DFW) and LaGuardia (LGA) between June 26 and July 31, Figure 6. True Positive Rate and False Positive Rate for Arrival Runway Configuration Predictions at DFW Figure 7. True Positive Rate and False Positive Rate for Arrival Runway Configuration Predictions at LGA 10

12 Between June 26 and July 31, 2015, the average departure runway configuration TP rate for the 29 airports covered was 79.79% across a 12-hour time horizon and the standard deviation was 13.26%. The average arrival runway configuration TP rate for the same 29 airports is 79.11%, with a standard deviation of 12.29%. The average departure runway configuration FP rate was 14.84% with a standard deviation of 11.77% and the average arrival runway configuration FP rate was 16.52% with a standard deviation of 12.54%. This prediction accuracy is statistically sufficient for airlines to visualize the flow directions in and out of airports. In addition, both TP and FP rates are subject to an airport s runway utilization. For example, if an airport has a temporary runway closure, the model will yield a lower TP rate due to the closure. For example, Runway 4L/22R at JFK is closed between April and September As a result, the runway configuration during this time period is fairly different from the past two years. Leading to, JFK s TP rates are the lowest among the 29 airports. AIRPORT TAXI TIME PREDICTION Airport taxi time prediction analytics take the guesswork out of delay propagation. Delays backup gate arrivals and takeoffs. Taxi time prediction leverages individual flight records from over ten of the most active U.S. carriers over the last two full years at 29 of the busiest U.S. airports. In addition to the airport specific taxi in and taxi out time predictions, 34 additional models were leveraged to account for the measurable impact on individual airlines taxi times at their hub airports. Similar to the airport runway configuration model, the taxi time prediction does not account for temporary runway closure and it is advised that airlines examine NOTAMS for the latest runway closure information. For taxi time, historical flight records published by the FAA are also used to verify the model s predictions. A verification method is currently being tested to examine predicted average deviation from true average. This robust combination of inputs provides expected taxi in and taxi out time predictions. In addition, the model also provides a distribution band to indicate the potential spread of the taxi time distribution. For example, if the predicted taxi out time average is ten minutes, and the model displays a time band between eight minutes and 25 minutes, you can 11

13 conclude that it is likely you will have higher than the ten minute average taxi time. However, if the band is between eight minutes and 13 minutes wide, you can be confident that it is likely the taxi out time is going to be closer to the ten-minute average at that given hour. For example, Figure 8 shows the airport taxi out and taxi in time predictions for ATL. We can observe the taxi out time is raised slightly from 22:00 UTC to 00:00 UTC and drops below fifteen minutes afterward. Since the distribution band for the taxi time is fairly tight, we can further conclude the model is confident in the taxi time predicted over this period. Using this data, airline operations can anticipate delays that may affect incoming or departing passenger connections. Figure 5. ATL Airport Taxi Out and Taxi In Time Prediction Chart in Fusion 12

14 Predictive Airport Analytics in Summary Early insight into congestion, runway configuration changes and taxi time at key airports is vital to optimizing flight operations. Each airline should evaluate its current decisioning process around dealing with airport congestion, runway configurations and taxi times. What data is used today to make key decisions? How is data usage standardized across the fleet? How do you simplify data access to make the right decision in real-time with so much information available? WSI Predictive Airport Analytics transform precise weather forecasts and flight data to skillfully predict changes in key airport operational conditions for the next 12 hours, empowering airlines to proactively implement strategies such as flight swaps, fuel contingencies and alternative routing. By bringing together the latest in big data analytics, machine-learning, and precise weather forecasting to deliver statistically accurate airport operations predictions, WSI Predictive Airport Analytics transforms vast amounts of information into timely and actionable insights to sharpen decision-making and improve efficiency and performance. 13

15 business.weather.com USA.2FLY