Author Topic: OpenAI API Plugin for Voiceattack (ChatGPT)  (Read 47425 times)

Killerwhale

  • Newbie
  • *
  • Posts: 5
Re: OpenAI API Plugin for Voiceattack (ChatGPT)
« Reply #30 on: April 18, 2025, 01:29:10 AM »
Hi Sempler, yes I would like to create simple profile that looks like this in the attachment. Basically it says in the when I say box Ask AI * , then after that I created text value to capture it in a variable named Question.  so far good but when it comes to execute external plugin, of course I select the openai plugin but the rest, I am unsure how to populate.  for example the plugin context, small integer, text variable and etc.



SemlerPDX

  • Global Moderator
  • Sr. Member
  • *****
  • Posts: 327
  • Upstanding Lunatic
    • My AVCS Homepage
Re: OpenAI API Plugin for Voiceattack (ChatGPT)
« Reply #31 on: April 18, 2025, 01:12:54 PM »
Hi Sempler, yes I would like to create simple profile that looks like this in the attachment. Basically it says in the when I say box Ask AI * , then after that I created text value to capture it in a variable named Question.  so far good but when it comes to execute external plugin, of course I select the openai plugin but the rest, I am unsure how to populate.  for example the plugin context, small integer, text variable and etc.

What you are trying to do would not be feasible.  Using wildcards in the manner you are trying will produce extremely poor recognition of the words you say after "Ask AI" because it would be free-form non-dictation recognition.  Even using Dictation directly, such a system would fail to produce intelligible sentences to present to OpenAI (ChatGPT) as a user input.  This is because Dictation in Windows is quite poor, and because generic recognition is not at all sufficient.

The reason VoiceAttack works so well is that we provide pre-defined command phrases for it to listen for.  This makes it far easier for recognition to discern what we said because it has a list of such commands to compare to.  This benefit is not present for Dictation or free-form speech such as when using wildcards.


For these reasons, I built in direct tools to capture user speech in audio format, and send that off to OpenAI Whisper for a proper transcription, and then to use that transcription as the user input which is sent to OpenAI as a ChatGPT input prompt.

You would want to review my Wiki for the plugin context "ChatGPT", including checking out the flow chart which describes how this plugin context logically progresses.  My ChatGPT context can be used as a one-off, or as part of a continuing session where previous input prompts are retained as part of an ongoing conversation.

I would also suggest that you check out my AVCS CHAT Profile for VoiceAttack, which properly utilizes the ChatGPT context to create a voice interactive session where responses are spoken using text-to-speech, and user input is gathered from transcription of captured audio during the "Get Input" phase of this plugin context.  Reviewing how I created this profile and how it operates may help you create your own flavor of a voice ChatGPT interaction system.

You would also benefit from reviewing the included OpenAI Plugin Sample Profile and the ChatGPT plugin context example therein, referring to my Wiki as needed to understand more about the optional variables, what they do, etc.

To be clear:  My AVCS CHAT requires users to first initiate input by saying, "Hey VoiceAttack", which would be followed by a text-to-speech response that prompts the user to speak their input.  Once the user stops speaking, what they said is transcribed and then sent to ChatGPT for a response.  If the user replies after that response, within a reasonable time (a few seconds), it will be treated as new user input where ChatGPT retains the previous input(s) and response(s) for contextual reference.  If the user simply does not speak in the few seconds after ChatGPT responds, AVCS CHAT will go back to "sleep" until the user says, "Hey VoiceAttack", once again - and when doing so (even if hours have passed), ChatGPT will still retain the previous user input and ChatGPT responses that came before until that session is ended with the command, "Stop chatting", or if VA is restarted.

Check out my AVCS CHAT profile powered by my OpenAI Plugin here:
https://forum.voiceattack.com/smf/index.php?topic=4520.0
« Last Edit: April 18, 2025, 02:58:42 PM by SemlerPDX »