Cockpits & Dashboards: Aviation Lessons for Measuring What Matters

This article was generated with the assistance of AI to help our volunteer team share valuable insights efficiently. While we've reviewed the content for accuracy, please let us know if you spot any errors or have additional insights to contribute!

Missed our recent FEDSA session with Fiona McCarthy? No worries. This talk explored a challenge many product teams, creators, and business owners face: having access to more data than ever, while still struggling to understand what really matters.

Drawing on more than 20 years of experience in aviation, Fiona compared modern analytics dashboards with aircraft cockpits. In aviation, pilots are not shown every available piece of information at once. Instead, cockpit systems prioritise the data needed to make good decisions in the moment.

Digital dashboards, by contrast, often overwhelm us with clicks, reach, impressions, followers and engagement rates without explaining what those numbers mean or whether they reflect genuine trust, value or long-term relationships.

Fiona argued that the problem is not necessarily a lack of data. It is a design problem: we have not always chosen the right information to surface.

A central idea in the talk was Goodhart’s Law: when a measure becomes a target, it stops being a useful measure.

Reach may begin as a way to understand how many people encountered a piece of content. Once it becomes a target, however, organisations learn to inflate it through advertising, bots or algorithmic tactics.

The same pattern can affect other familiar metrics:

The metrics are not always wrong, but they are incomplete. They tell us what happened without necessarily explaining what it meant.

Aviation faced a similar problem as aircraft became more complex. Adding more instruments and alerts increased pilots’ cognitive load and, in some cases, contributed to incidents.

The solution was not to stop collecting data. It was to prioritise what the person in the cockpit needed to see.

For a pilot, altitude, attitude and airspeed provide essential information. Other data still exists, but it remains in the background until it becomes relevant.

Fiona suggested that analytics products should follow the same principle:

What does the person using this dashboard need to see to make a good decision right now?

In digital products, there are two people in the seat: the practitioner building or managing the experience, and the audience deciding whether it is worth their time and trust.

Fiona shared several examples showing how standard analytics can fail to capture the human meaning behind an event.

A small flower shop discovered that another company was appearing in search results using its name and attracting potential customers. The owner shared an emotional video, and the local community responded by supporting the business.

A conventional dashboard could record impressions, shares and engagement for both the harmful search activity and the community response. What it could not explain was that one damaged the business while the other helped rescue it.

In another example, a spontaneous interaction with a traffic officer generated millions of views. The content appeared to be about a car, but audiences shared it because of the warm interaction between people from different backgrounds.

Fiona described this as bridging social capital: trust forming across communities and groups. The dashboard could report that the video performed well, but it could not explain why it resonated.

She also discussed content creator Chad Nathan, whose audience helped raise a significant amount of money for a homeless amputee. The response did not come from a single viral post. It came from years of consistent work, follow-through and trust.

Traditional analytics captured the result, but not the accumulated trust that made it possible.

Fiona proposed several ideas for measuring deeper audience relationships.

This would distinguish between someone who returns because they genuinely value the content and someone who returns because they were prompted by an advert, notification or reminder.

The difference matters because voluntary return signals commitment, while prompted return may only show compliance.

A quick reaction or emoji requires very little effort. A thoughtful comment, question or discussion shows that someone has considered the content and chosen to contribute publicly.

Treating these actions as equivalent engagement misses the depth of the audience’s involvement.

Trust develops through consistent performance, transparent processes and an aligned purpose. A useful metric could examine how voluntary returns, meaningful conversations and unincentivised sharing develop over time.

This would help organisations understand the strength of their audience relationships, rather than focusing only on individual campaigns or posts.

The session raised an important challenge for designers, engineers, founders and content creators: rather than asking how much data we can collect, we should ask what information helps us understand the people behind the numbers.

Watch the full event to hear Fiona’s aviation stories, case studies and ideas for building more meaningful analytics dashboards on our YouTube channel.

Join our Discord for discussion and updates, and subscribe to our monthly newsletter to stay in the loop.