For two decades, NetForecast has provided Wi-Fi performance measurements for terrestrial and mobile Internet services including for customers like Comcast. But in 2018, it started to get involved in measuring inflight connectivity, both air-to-ground (ATG) and satcom-based solutions. Now the woman-owned company’s client base includes airlines and aero ISPs. It also has several airline trials in place and more in the offing including arrangements secured by management at the APEX/IFSA EXPO in Long Beach.

RGN sat down with NetForecast president Rebecca Wetzel at the show to learn more. “We decided that we had done very well with our terrestrial clients, and we wanted to find other areas where there were pain points that we thought that we could help solve, and places that we could add value,” she said about the firm’s decision to enter aviation several years ago.

“So, we became involved with the Airline Passenger Experience Association. We went to one of their TECH conferences and we got involved, almost immediately, in a Connectivity Working Group sub-group that was tasked with what to measure in terms of the passenger experience with Wi-Fi onboard aircraft. And so, we helped to create a specification for what to measure. We worked together with the airlines and with service providers and vendors and we created a spec which is now publicly available, and we use that spec to actually develop a service around it, to provide the measurements for inflight WiFi so that people could really quantify the passenger experience.”

Quality of Service (QoS) is a common measurement for performance of a connection. But while NetForecast measures QoS, it also measures the actual passenger experience, and scores it accordingly.

Wetzel explained:

What we’ve done is we have created models for how applications behave over networks, and we categorize applications by class so that you have messaging, you have interactive, you have streaming. And so, we have a bunch of categories of applications and then we see what those applications need in terms of support from the network.

And so we know what the QoS is. We measure the QoS in the plane from a portable electronic device. In most cases it’s the flight attendants [who] can have the app running on their electronic devices and then we measure the performance of the network from the passenger’s perspective.

And then we can translate that or match that to what the applications need [in order] to provide a good customer experience, and then we can score that. We can give it a Quality of Experience (QoE) score for the flight for the various applications, and then we can roll that up into a Quality of Experience score for the entire flight.

Global IFC standards group Seamless Air Alliance is now getting more involved in IFC performance measurements, and has even established a lab for members who wish to measure service quality.

But Wetzel told RGN that NetForecast’s work in measuring the actual passenger experience effectively in the seat and providing that independent viewpoint is, to her knowledge, something that no one else is doing.

Scoring various applications and providing an overall QoE score for the entire flight requires NetForecast to look at the entire chain, starting at the passenger seat all the way to the servers being accessed by the passenger, through the onboard Wi-Fi system, through the satellite or ATG pipe down to where the data reaches the ground and then beyond.

“So we look at peering points,” she said. “We know exactly how that traffic is routed and what the experience is to the destination and back. So, we can look at latency, that it gets contributed by the satellites, for example, latency that gets contributed on the ground. We can look at where is it the passengers in a particular region might be most likely to go to on the ground in terms of looking for content. And then we can customize our tests to be able to give a real sense of what that experience is going to be like to go to a server that is going to be, perhaps, in a geography that they’re likely to be going to as a destination.”

This gives the airlines “some grounding in making decisions and and fixing problems that you might not even have known were there, because we can, for example, provide data regionally; we can show where there are gaps, where there are areas of coverage that might be needing attention … because we can look at the overall scores and then we can look at the underlying data and dig down, dive into it to figure out where there are problems…And unlike other possible solutions, we look at the customer experience from end to end.” Any gaps in coverage — a common IFC issue — are factored into the scores.

Armed with NetForecast’s QoE scores, airlines are then better equipped to hold aero ISP’s proverbial feet to the fire to meet their SLAs.

Said Wetzel:

What we’re finding is that the airlines want to know this. They’re hearing that their service providers are saying ‘look we’re nearing our SLAs, here’s our data that shows that everything is fine’. [But] the customers, the passengers are complaining and we’re able to independently provide information to the airline that helps them understand why there’s this dissonance.

And in a lot of cases, airlines are trying to put in place the best possible passenger experience and they have choices to make of service providers or perhaps a service provider is upgrading to a different service and they can look at the ‘before and the after’ or ‘compare the A to the B’ and see which is going to be the best for their particular passengers.

Participating in the APEX Connectivity Working Group, which enabled NetForecast to talk directly to airlines and get their feedback, was “absolutely essential” to the firm’s work to spool up and understand the aviation environment, and serve the sector.

“[W]e were only able to understand that by working with the industry, in collaboration with the industry, to come up with this [APEX] spec and so that was the best possible way that we could really learn the ropes and make sure that what we came up with for a solution was going to meet the needs of the industry and it’s been very successful. And we found a great deal of interest. A lot of airlines as well as service providers are coming to us and wanting to use our services to help them to better serve their passengers.”

She revealed that one of NetForecast’s airline customers “have said ‘we want data on every flight we take and we want to be able to see what that passenger experience is like’ because they care. And it’s increasingly important because passengers are making buying decisions based on this.”

Since the early days of satellite-supported broadband IFC, GEO satellites have been in play. But industry is moving towards hybrid networks that can also involve LEO and MEO satellites and ATG. Indeed, a multi-pipe reality was touted by many stakeholders on the show floor. But this adds a certain amount of complexity to measuring PaxEx. Moreover, today’s hyper-connected younger passenger has high expectations for IFC. Wetzel described how NetForecast has already put the building blocks in place to measure onboard Internet powered by hybrid networks:

“One of the things that we need to do when we’re looking at each of these network types — and we are working with a variety of technologies — is we need to benchmark each one of them because what we need to look at is what’s the best that that network can deliver or what does it routinely deliver? And then we can look at when does it fall below what it should be delivering. And those thresholds are very important. We use a methodology called Apdex, Application Performance Index, and this was a technology or a methodology that was developed by our founder, Peter Sevcik. And what Apdex does is it allows you to set a threshold for a tolerating experience and then a frustrating experience and once you’ve determined what those thresholds are, there’s a formula that allows you to turn the Quality of Service metrics into Quality of Experience metrics and that’s where we know what those thresholds are from various application types. And then we can turn those into the Quality of Experience scores based on what it is that the passengers are trying to do and then for the entire flight,” she explained.

“So all of that [and] yes it does get complicated, but you need to really measure the network against itself in a lot of cases and then we can look at the hybrid situation. And then you can look at how the experience changes during the course of a flight or, you know, it can expand into any kind of transportation such as maritime. You know, with a ship you’re going to be changing possibly from the GEO [satellites] to maybe a MEO and a LEO and maybe ATG. And so, we can map that journey and come up with Quality of Experience scores along the trip.”

To that end, NetForecast is now getting involved in maritime, and some of its work will be informed by its aviation work. “I have to say it has been a very interesting and exciting time to be doing this, and there is a real need. We are finding a hunger for the data,” she concluded.

The videotaped interview is available below.

 

Featured image credited to Reilly Oatridge