“I didn’t think it would work” – An Interview with uniqFEED CSO and Founder, Ralf Dragon
Today, virtual advertising is used in some of the most high-profile sports in the world. It has left the realm of “niche” technology and is part of the wider sports business conversation. However, there is still some way to go until widespread adoption is achieved. The market is still figuring out what it wants and needs, and the technology providers are working hard to develop solutions that deliver on these wishes.
uniqFEED was founded back in 2015, when virtual advertising technology was still in its infancy and any existing providers offered hardware-based solutions. uniqFEED founders Ralf Dragon and Lukas Gysin spotted a gap in the market for a software-based solution, which would eliminate not only the requirement for additional hardware within the stadium, but also the need for any additional personnel even within the vicinity of the venue. The idea was sound, and the potential benefits were clear, but was a technology like this even feasible? Could virtual content be inserted into a live sports broadcast using software only? uniqFEED founder Ralf Dragon, thought so. With a PhD in visual object motion analysis and segmentation, he was ideally placed to make this call. Here we speak directly to him about his early thoughts about virtual advertising technology, the challenge of meeting the needs of the market, and the possibilities for the future.
When he initially became aware of virtual advertising, Ralf says that he was unsure about whether a widespread adoption of the technology was likely or even possible:
“I think I first became aware of it in 2006 when I was about to finish my diploma. It was around the same time that I heard about large scale 3D reconstruction and worked on an automatic voice translation project. My opinion about virtual advertising and those two other projects was that it would be really cool if these were to become reality, but also that it might be unrealistic to get the technologies to work in practice.”
Ralf’s initial skepticism about virtual advertising and other new technologies was confronted, however, when he started to see these other conceptual ideas coming to life. One such example was Google Street View, which was released in 2007, demonstrating that “theoretical ideas could very quickly become reality”.
This new optimism set the wheels in motion for realizing a new, entirely software-based virtual advertising solution. Ralf partnered with Lukas Gysin to found uniqFEED in 2015 and bring a new solution to the market.
“Hardware-based solutions had existed already for quite some time but their development had seemed to stagnate,” was Ralf’s observation of the market back in 2015. At the time these solutions typically required additional equipment to be installed within sport venues – special LED boards, for example, or adaptations to camera equipment. This made the process of implementing virtual advertising expensive, intrusive and difficult to deploy at scale.
As rightsholders became aware of the possibilities offered by virtual advertising technologies in terms of increased sponsorship revenues, it became clear that there was market for a software-only solution. “However,” says Ralf “…one has to make the technology work
There was no blueprint for a software-based product at the time. Today, software-based virtual advertising technology is based on a series of complex algorithms which enable the system to recognize the area within the television frame that needs to be augmented with virtual content and to ensure that this content remains in the correct position as the camera pans and zooms. The system also needs to “understand” what is foreground and what is background with the frame, to ensure that players/balls/other items that pass in front of the LED boards, are not obscured by the virtual insertions.
“In the years before uniqFEED was founded, there were tremendous advancements in GPU (Graphics Processing Unit) performance and all of a sudden it was possible for many algorithms to be processed in real-time. I realized, as I considered the technical challenges of the past, that the timing was actually exactly right to implement virtual advertising as a GPU-powered software-only product.
The initial concept was only the first hurdle, however. Ralf and Lukas were setting out to do something entirely new, with many unforeseen and unexpected challenges along the way.
“I thought that camera tracking, particularly in sports like Football and Ice Hockey, would be difficult.” Ralf says, referring to the process of ensuring that the virtual content remains in place as the camera moves. However, this turned out not to be as problematic as predicted, and uniqFEED is now widely considered as having some of the most effective tracking in the industry.
What posed more of a problem, were the segmentation algorithms (known more familiarly in the industry as keying). Ralf Dragon optimistically predicted that virtual advertising would quickly supersede LED advertising. But this turned out not to be the case. Fully animated LED boards are still commonplace among many major leagues and sports, particularly in football, and overlaying these with virtual content is a lot more challenging than overlaying a blank or static background.
“In the meantime, it became clear that LED boards would be more and more widespread and that our solution would need to be non-intrusive. So, we had to somehow overcome the challenge of replacing complex animated backgrounds”
The growing team of developers at uniqFEED rose to the challenge to develop uniqFEED’s current “deep segmentation algorithm, which enables the segmentation of players and objects that pass in front of dynamic LED backgrounds.” Ralf says that he could not have foreseen that uniqFEED’s system would be able to do this so efficiently and reliably in so short an amount of time, calling the feat “truly incredible!”
This pivot to a new method for segmentation/keying also had unexpected benefits for the continued development of uniqFEED’s virtual advertising solution for different sports, making the technology “more robust and laying a foundation for overcoming increasingly challenging sports environments.” Some of these challenges include managing difficult weather conditions, such as rain and snow, and training the system to react to changes in light and shade.
Another challenge that quickly became apparent as uniqFEED began to break into the market, was the issue of integrating into broadcast workflows, and how the technology could potentially be blocked by the broadcasters themselves, regardless of the rightsholders’ keenness to implement the technology based on commercially driven motives.
“It started to become more and more evident, that even if our technology would unlock massive new revenues, it was going to be difficult to change existing workflows”.
The first priority was to ensure the development of a solution that works downstream of broadcast production, between content creation and distribution, ensuring no intrusion on the host broadcaster’s production process. However, this would not be the only requirement from the broadcast side…
“These days, many of our clients require us to work with the dirty world feed [this refers to the version of the world feed that already includes match graphics such as score bugs and players statistics]. Fortunately, we have been able to react and adjust our architecture to render on the dirty world feed, allowing us to integrate into more setups and thus to overcome what could otherwise have been literal “deal breakers”.
It increasingly became clear to Ralf, Lukas and the growing team at uniqFEED, that as the technology continued to develop, so too would the needs of the market.
“Since the beginning, our focus has always been on solving challenges, one after the other.” Initially, Ralf says that the team at uniqFEED were rapidly developing new product features, but that these didn’t always come together to form a cohesive market-ready product. “It was evident that we needed to constrain our excitement and focus our ideas on developing what the market is really looking for…. to invest more time and effort into testing, market evaluation and planning” However, he says that while the team are now more focused on creating the ideal virtual advertising solution that the market wants and needs, “I am happy that we have been able to preserve the adaptability, passion for new ideas and problem-solving attitude that we have had from the very beginning” .
Finally, Ralf briefly touched on his thoughts on the potential for the technology in the future. Like many, he sees virtual advertising technology evolving to emulate advertising platforms like Google’s AdSense, which displays personalized advertising to individuals based on demographic data and user behavior online. “Creating localized feeds is impressive”, he says “but the potential is to further revolutionize how live sport is monetized. Using our technology, we will [in the future] be able to personalize ads… which in turn means an increase in revenue for the rightsholder by expanding beyond only sponsorship and into mass-targeted advertising. Advertisers will have increased flexibility when designing their campaigns, as the ads could be sold per view, rather than per league or per season”.
What is clear, from this interview with Ralf, is the future is never certain. That unforeseen challenges will occur and that the continuously evolving market will be what determines where the technology will go next. What is also clear is that even though the uniqFEED team have already achieved well beyond what Ralf originally thought was possible, the job is far from done and the team have the pride, passion and drive to make virtual advertising even better in the future, for the benefit of all professional sports (and maybe even beyond!).
Interested in learning more about virtual advertising? Get in touch with the team.