Thats really neat to have a family connection to the track! I've only been to a handful of big tracks here in California but one of the great things about iRacing is the opportunity to get familiar with racetracks all around the world.
As far as PitGirl and the VoiceAttack / Twitch / iRacing / OBS integration goes its been a lot of fun to develop and use while driving.
These are two of the core pieces that I did not develop:
The Digital Race Engineer, or DRE -
https://www.thedigitalraceengineer.com This is where PitGirl was born. I started using the Digital Race Engineer in 2016 when I was pretty new to iRacing and VR. I found that fumbling with the controls to try and set fuel for pitstops was a nightmare and a viewer suggested this as a potential solution. At that time, DRE was a commercially developed plugin for Voice Attack that integrated with the iRacing SDK. Being able to ask the system questions about what was happening in the race (like fuel requirements) and particularly being able to tell it what to do - “Pitgirl, set fuel to 3.7 gallons” was a a game changer. The developer still offers the legacy VoiceAttack system, but has gone on to develop a stand alone version. I’m deeply dependent on VoiceAttack for my other systems, so I don’t see myself switching, but encourage anyone to take a look and use DRE.
Its great.
Speech Chat -
https://www.speechchat.com/ I use Speech Chat to hear the Twitch Chat. In the early days, everyone on the stream heard the voice, but people did troll it. They would push the limits of good taste to see what they could get it to say, put in huge numbers, post URLs, and stuff like that. Once I set it so that I’m the only one who hears the voice, all that stopped. In my experience, hearing the chat and responding ends up being much more like having a conversation and something I can do while racing. I have had windows for the the chat posted in VR, but in practice any time I tried to actually suffer the distracted driving of reading it would end in tears.
For a long time I just used VoiceAttack and DRE to handle the spectator view on my stream. I had custom voice commands to change the cameras, do simple replays, and manage the scenes for the stream in OBS. Those commands were cool and unique, but all really boiled down to VoiceAttack sending keystrokes to iRacing and OBS and the whole thing was terribly fragile. It had no idea if commands were successful, or what state things were in, and I had to reset it often when it went off the tracks.
Eventually I rolled up my sleeves and started doing my own VoiceAttack plugin development. It was the first time I had used C# and the first non trivial software development I had done for a couple decades. I have a computer science degree from before the turn of the century, and have had a career in video games as a designer - but have not been an active programmer for a long time. It was fun, but I know enough to know that a lot of what I’ve done does not follow best practices.
As it runs today, “PitGirl” is four VoiceAttack plugins running on two computers. There is my Driving Rig PC and the Stream PC. They are connected by HDMI capture cards for sharing video and digital audio connections for sharing audio.
On the Rig PC, VoiceAttack runs the Digital Race Engineer and a PitGirlSpeechCoordinator plugin I wrote. The speech coordinator has web sockets to get requests from the StreamPC plugins for when they want to say something and does its best to keep PitGirl from talking over herself. It knows when the DRE system is speaking, and can delay or decline requests to speak from plugins running on the StreamPC.
On the Stream PC VoiceAttack runs a PitGirlVAPlugin and TwitchVAPlugin.
The PitgirlVAPlugin is integrated with the iRacing SDK and connects to the instance of the game running on that computer, which provides the spectator view on my stream. This plugin provides command support to VoiceAttack to do direct integration with iRacing and OBS. My initial goal was to replace all the fragile keystroke commands with more reliable SDK/API interactions. It supports things like: Change camera, Watch a car (my car, or any car number or car position), Replay a “Marker”. That let me build higher level VoiceAttack speech recognition commands for things like:
PitGirl, set camera to chase
PitGirl, checkout the car ahead
PitGirl, show most excitingIt also has a loop that is constantly monitoring the iRacing telemetry and records these Marker events for any time anyone changes position, goes off track, broadcasts on the radio, etc. So rather than just relying on the iRacing next/previous incident access to the replay data, I have a whole data structure full of everything that the system recorded and can build high level commands like:
PitGirl, replay recent markers for car 5The system will do an ad hoc replay of anything that car experienced in the last three minutes, doing nice OBS scene transactions, random camera selections, and voice overlay describing what happened.
The plugin also maintains html/javascript overlay elements for the stream so we can see which driver, car number, lap and position data both for real time when the spectator is watching the live race as well as historically so its still accurate when watching the replays.
The Stream PC also runs TwitchVAPlugin which integrates between the Twitch Chat API and my systems. This makes it so viewers can make requests to similar PitGirl commands that I use verbally, just from the Twitch Chat. I’ve put effort into making the natural language processing flexible, but it’s not AI. Users can say things like:
!pitgirl set camera to cockpit
!pitgirl checkout the car in position 4
!pitgirl replay overtakes for car 3 with the chopper cameraIt works on dictionaries for commands: camera, replay, checkout, watch
And dictionaries for terms that relate to one or more commands: car number, camera name, marker type, etc.
The system scans what the user types in looking for matches in the command and terms dictionaries and builds a weighted list of the results. If there are matches, it runs the highest weighted result. That means the order of words doesnt matter and I can manage the weights of the commands/terms to try and prioritize expected results. So in the examples above, the command that asked for a replay but also specified a camera will run the replay command but knows that there was also a camera parameter to include as well. The real goal was to try and make it so viewers on the stream had interactive things they could ask PitGirl to do and have a feeling of agency and involvement. And they could help with the race direction for the stream broadcast and I could focus more on driving.
The most recent addition is the OpenAI ChatGPT integration.
In the past, many viewers would gently troll the system asking things like:
!pitgirl what does kris do after the race?
!pitgirl whats your favorite ice cream flavor?I never felt motivated to try and script responses to those questions, but when I started playing with ChatGPT and realized the API was available, that seemed like it would be a fun thing to integrate.
So now when someone enters a !pitgirl command into the twitch chat that does not match one of the existing commands and they use a question word (who, what, when, where, why, how, can, is, etc) it gets set as a question for OpenAI.
There are special users (I trust) who get their questions send directly. They can ask things and she responds. For most users, it poses the question and I can hear what asked and have a VoiceAttack command I can use to send it:
PitGirl, ask the Twitch Chat question.I can also just verbally interact with the system:
PitGirl, let me ask you a questionShe responds:
What would you like to ask?It uses free-form dictation to get a question and send it off. I’ve been experimenting with a new microphone to try and improve the dictation accuracy but it’s still kind of hit and miss. It’s best when the viewers ask the complex questions.
Topic: PitGirl - my sim racing virtual assistant. iRacing, VR, Twitch, and now ChatGPT (Read 2418 times)