How ESPN Engineered Live Players on The Simpsons Funday Football Broadcast

Using Sony’s Hawk-Eye player tracking technology that Design News covered previously, the NFL and ESPN were able to broadcast an animated Simpsons version of the Monday Night Football game between the Dallas Cowboys and the Cincinnati Bengals in real time alongside the live video broadcast of the game.
Leaving some of us to wonder, “How did they do that?”

Unsurprisingly, through the application of a lot of cool technology. As discussed in the Hawk-Eye article, Sony’s technology blankets the field with cameras and brings all their images together over a Cisco network.

Using Sony’s Beyond Sports Technology, to track individual players and their movements, ESPN was able to depict the football teams as Simpsons characters playing at the fictional Atoms Stadium in the show’s Springfield hometown. This Simpsons Funday Football alternate viewing presentation streamed live on Disney+ and ESPN+, and, on mobile, with NFL+ while the traditional Monday Night Football telecast aired on ESPN, ABC, and ESPN Deportes. This broadcast was the second consecutive year ESPN, Disney, and the NFL have presented a Funday Football telecast, following last year’s showing of a Toy Story-themed version of the Falcons-Jaguars game from London. 

Sony’s Beyond Sports analyzes, validates, enhances, and translates player movement to a 3D environment that streams to the CGI platform in real-time. Hawk-Eye Innovations’ optical tracking adds dynamic player and character movement including detailed limb movement and mannerisms captured by its cameras.  

Related:Super-Smart Super Bowl Replay Technology

Additionally, commentary was provided by ESPN’s In-House NFL Personalities and Simpsons enthusiasts Mina Kimes and Dan Orlovsky, who will provide football strategy discussion with Drew Carter calling the game action. They were animated in the style of the Simpsons and employed ESPN Edge Innovation Partner’s Meta Quest Pro headsets to experience the game from Springfield thanks to VR technology and Sony’s Beyond Sports’ Virtual Commentator Technology.

The ability to deliver a broadcast of animation created on the fly using data from players comes from “the constant and rapid progress with the technology that, again, started with single point tracking just a couple of years ago and relatively basic movements to where we are today with blended tracking,” said Phil Orlins, ESPN’s vice president of production in a pre-broadcast press conference.

“Our 3-D simulation is based on positional tracking data, firstly,” elaborated Nico Westerhof, Beyond Sports co-founder and chief technology officer. “We actually have two sources now coming together. We have the single-point tracking data that’s provided by NGS, which is active tracking. It’s trackers that are in the shoulder pads of the players. It’s a very stable feed that tells us exactly the position for a player.”

Related:Super Bowl 2024 Scores Rout for Tech

Using this data alone, the animation could show the location of the players on the field as they move around and the location of the ball. “We just know where they are,” pointed out Westerhof, “but they don’t hold the ball. They don’t throw the ball. They don’t catch the ball. This is stable, but limited, in terms of viewing experience.”

The next step is to blend in detailed information about players’ individual movements that is captured by sensors on their bodies. “The second data source is skeletal data tracking and limb tracking data,” he said. “And there we actually have 29 points per player so we can get a lot closer to the actual character’s movement and the detailed movement that these characters have.”

Finally, for the Simpsons-themed broadcast, ESPN blended in Simpsons characters as players on the field from time to time. “This takes a whole other-level processing,” Westerhof said. “We’ll use Lisa (Simpson) for an example. As she catches the ball, Lisa is much smaller than the rest of the players. So, in real life, the ball would go over her head, but now with data processing we can actually take the ball, make it go exactly into her hands. So for the viewer, it looks still believable, and it all makes sense.”

Related:He’s One of Us: NFL Quarterback is a Math Major at ODU

That is, if a fictional cartoon character from the longest-running scripted TV show in history catching passes from NFL hero Joe Burrow makes sense. But the broadcast puts a new spin on what can look like a giant rugby scrum to young viewers. And it also can amaze older viewers who can appreciate the difficulty of the technology that makes the show possible.