Dota 2 Tooltips
Info
Release: April 2021
Team: Shannon Scotten
Overview
This extension allows viewers on Twitch to interact with Dota UI right on the stream and read about abilities, items, talents and more; exactly how they would do it in the game directly.
Perhaps one of my most complex projects to date. It was the perfect way to spend my weeks of lockdown during the pandemic. Shannon always wondered why such a tool didn’t exist in the community — I took it as a challenge and got to work.
The extension really put my skills to the test. I was used to working with real-time game integration, and extending UI systems, however the challenge here was much greater since I knew immediately that the UX standards of this app will be high. If we wanted streamers to use it, we had to make sure it was seamless to setup with just a few clicks.
After I coded the first prototype, Shannon demonstrated the extension on his stream and from there we just kept adding more features and iterating based on viewer feedback. The extension quickly took over the whole Dota 2 section on Twitch.
Is is now the #1 most used extension for games on Twitch.
Twitch Extension · UI/UX · Game State Integration · OpenCVfor the community
Logo with title
Grubby's stream with the Dota 2 Tooltips extension enabled
Tooltips are precisely styled to match their ingame design
The extension also works for mobile viewers
Dota 2’s GSI provides a way to get real-time data on the hero you are currently playing with. However, one of the unique challenges I faced was finding a way to fetch the list of all the other heroes in the game.
Shannon and I considered many solutions, like having chat commands to allow mods to set this information manually. Or having the streamer add a Steam bot to their friends list, which could spectate their game and parse this info. However, all those solutions added friction to the setup. Finally, we decided to give computer vision a try.
Using OpenCV, I was able to parse out the hero list from the stream thumbnail. With some adjustments and optimizations around image processing to make parsing as quick as possible, we started scanning any stream that had our extension installed. The backend would relay this information to the extension so viewers could then click on any hero in the top HUD!
With limited APIs, OpenCV was a perfect fit
The International Edition
I worked together with Valve to deploy the extension for the largest Dota 2 event of the year The International 10. Having access to full game stata data allowed me to create features specifically tailored for watching tournament matches. Aside from a thematic restyle, the TI edition of the extension also included an item board, fully browsable item shop, tournament standings, in-game predictions and cheering.
The standards at TI are equal between all languages so the extension was localized to support all official broadcast languages: Russian, Chinese, Spanish & Portuguese.
In 2022, we partnered up again to bring the extension for the 11th iteration of The International. Adding even more features: dropdown stats, draft tooltips, graphs and predictions for mobile viewers!
