AVCS - Voice Chat Artificial Intelligence Profile (ChatGPT)

[SemlerPDX]

Artificial Intelligence Chat for VoiceAttack
by SemlerPDX

I am extremely proud to present my latest and greatest AVCS Profile for VoiceAttack yet, which allows for natural speaking conversations with a true artificial intelligence we can customize!

It works by starting up a "conversation" with ChatGPT, first providing it with a System Prompt that helps refine how we want it to behave, and then it just sits in the background not listening, not sending anything to OpenAI or the ChatGPT API, until we wake it with a question, such as saying, "Hey VoiceAttack". It will respond, and wait for our question, and then (since it take a second to process using AI) a subtle 'thinking' sound plays until it responds in text-to-speech to our input. Then, it returns to listening for our next question or comment on what it just said - it remembers the context of our recent interactions in this manner, and if we simply do not respond, or use an existing VoiceAttack command, it will merely return to it's sleep loop until woke.

But far more than just being another 'Alexa' or 'Siri' clone, AVCS CHAT uses the provided optional response variables that contain any hyperlinks or code blocks which were culled from the text-to-responses of ChatGPT, and automatically opens code examples in notepad or presents a list of any URL's we can open from a response.

The included Chat System Menu allows us to view, edit, create, and delete System Prompts we can use at the beginning of a Chat Session - or set the default System Prompt to use.  By default, the system prompt is one tailored for text-to-speech and brevity, although the "max tokens" are set to 2048 and it may override those instructions if asked for more complete details on the fly.

Say, "Open the Chat System Menu" - much fun will be had!

The beauty of this type of system is that we can tailor a perpetual background chat session we can resume anytime with a simple "Hey VoiceAttack" voice command, perhaps if the knowledge base of ChatGPT contains enough information about a game, we can craft a system prompt that makes it think it is an expert in that game and provides answers to guide players specifically as information relates to that game. This can establish a context at the start so that when you ask something that may also apply to the real world, you'll most likely get the response as it relates to that game. This can make for some rather entertaining and immersive conversations with a true AI we can call our own.

Customize EVERYTHING! It's YOUR Personal AI!

The profile has many "function" commands, which are not called by voice, but by the plugin or other commands.  Their 'When I say' phrases are written such as "((VoiceAttack is...))" so that when these commands show up in the VoiceAttack Event Log, it feels correct like "((VoiceAttack is Speaking))" or "((VoiceAttack is Thinking))" .... each of these can and SHOULD be changed if desired, perhaps you call your profile a name like I do - change each 'When I say' phrase - and then inside the "Hey VoiceAttack" comand, change each of the text variables starting with "OpenAI_Command_" to reflect your altered command 'When I say' phrases:


Even the "Hey VoiceAttack" command phrase could be "Hey Arnold" or however you would like to refer to your personal chat AI. Just make sure you follow comments in the profile, it's all quite simple and ready to use.

Download this profile here (and also install OpenAI Plugin for VoiceAttack below):
AVCS CHAT Profile for VoiceAttack

Code: [Select]

FLOW EXAMPLE:
User says, 'Hey VoiceAttack'  (to awaken chat & global listening, if disabled)
Profile says, "Yes?"
User says anything - a VoiceAttack command, OR a question for ChatGPT
(if not VoiceAttack command)
Profile processes input and sends to ChatGPT and plays a subtle 'thinking' sound until ChatGPT responds
Profile says, "(response from ChatGPT)" using text-to-speech (opening any code blocks in a notepad)
- User says another question/etc. and ChatGPT responds again, OR user says nothing and input times out -
ChatGPT enters 'Sleeping' loop until users says, 'Hey VoiceAttack' again to awaken it

AVCS CHAT is designed to work as a Global Profile always ready in the background, but this is not required.

===============  OPTIONAL WAYS TO USE THIS PROFILE  ===============


OPTION 1:  Switch to the AVCS Voice Chat Artificial Intelligence Profile and say, "Hey VoiceAttack" to chat

OPTION 2:  Open Profile Options for any of your Profiles, select Include Commands from other profile: 
        "AVCS Voice Chat Artificial Intelligence (latest version)"
  -or-
OPTION 2b:  Open VoiceAttack Options and select Global profiles, then Include Profile Commands from the profile: 
        "AVCS Voice Chat Artificial Intelligence (latest version)"

=========================================================

AVCS CHAT is powered by the OpenAI Plugin for VoiceAttack which I also created specifically so we all could make profiles like this. You will need to download and install this plugin in order to use the AVCS CHAT Profile for VoiceAttack:

The OpenAI VoiceAttack Plugin provides a powerful interface between VoiceAttack and the OpenAI API, allowing us to seamlessly incorporate state-of-the-art artificial intelligence capabilities into our VoiceAttack profiles and commands.

Find complete details, download link, and documentation on GitHub:
OpenAI Plugin for VoiceAttack

I'm so excited to bring the power of true artificial intelligence to VoiceAttack through this plugin for all profile and command builders out there interested in working with OpenAI Technologies in VoiceAttack! I know everyone assumes that now that this technology is available, it will be easy to incorporate into existing programs or workflows, but the reality is that this is a brand new technology being made available and until some aspects of it become more accessible, working with the OpenAI API itself is a great way to get our foot in the door and start taking advantage of this awesome power right now.

All of the same known limitations of these AI models apply here, ChatGPT will boldly state incorrect facts with high confidence at times, and we should always double-check or test responses - only difference is now, we can berate it verbally and ask for a correction which it can speak back to us!

If you enjoy this profile, Click this Pic to check out my other AVCS Profiles:

[razzafrag]

Hi. I get this error in the openai_errors.log when i try the "hey voiceattack" and then ask it a question.   System.Exception: OpenAI Plugin Error: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d__19.MoveNext()

Any pointers?
Thanks!

[SemlerPDX]

Hi. I get this error in the openai_errors.log when i try the "hey voiceattack" and then ask it a question.   System.Exception: OpenAI Plugin Error: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d__19.MoveNext()

Any pointers?
Thanks!

Can you please verify that the folder at the following path (or your VoiceAttack programs folder, wherever it is) has the following files inside it?


...

(edit)
Also, please verify you follow these installation steps exactly:
Import the VoiceAttack Plugin Profile Package
1. Set VoiceAttack to Run as Admin and Disable Plugin Support (only required for import)
2. Re-Start VoiceAttack (only if you had to perform either task above, to apply changes)
3. Press the Import Profile button, select this profile package (OpenAI API Plugin for VoiceAttack.vax) and import
    - Before re-enabling Plugin Support, you can import profiles which use this plugin now, such as AVCS CHAT
4. When done importing, Re-enable Plugin Support in VoiceAttack Options
5. Re-Start VoiceAttack
6. The OpenAI Plugin will finish installing files, telling you to Restart one last time (step 7 below)
    - Before restarting to complete installation, you can now turn off Run as Admin (if desired)
7. Re-Start VoiceAttack (final step - OpenAI Plugin pop-up will tell you to)

[razzafrag]

Thanks for the response. Yes  I do have those files in the Assemblies folder.
I tried installing twice, using those instructions.   I didnt get the restart instruction on step 6 the second time. 

[SemlerPDX]

Thanks for the response. Yes  I do have those files in the Assemblies folder.
I tried installing twice, using those instructions.   I didnt get the restart instruction on step 6 the second time.

Can you please open the Plugin folder itself and verify that the following files exist there:


(the message didn't appear on second installation because the files already exist in that folder, was treated like an 'update' overwrite by the plugin, which is okay, too)

[razzafrag]

Yes, see image

[SemlerPDX]

what image?

[razzafrag]

sorry,  link to image : https://imgur.com/a/gsyshq4

[SemlerPDX]

sorry,  link to image : https://imgur.com/a/gsyshq4

I have no idea what to tell you. I cannot replicate your issue. I even deleted the 'Microsoft.Bcl.AsyncInterfaces.dll` file in my plugin folder and restarted VoiceAttack and the AVCS CHAT profile, then said "Hey VoiceAttack", asked a question and got a response from ChatGPT - no errors at all. I then restored the file from the recycle bin.

Maybe try running VoiceAttack as Admin to see if that changes anything.

[SemlerPDX]

sorry,  link to image : https://imgur.com/a/gsyshq4

What Windows Operating system are you using?  What is the version of VoiceAttack you have installed?

[razzafrag]

v1.10.3 of VA, Windows 10
Tried the old Windows reboot ... still same error
OpenAI Plugin Error at 2023-05-08 1:06:16 PM:
System.Exception: OpenAI Plugin Error: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at OpenAI_VoiceAttack_Plugin.OpenAIplugin.<PluginContext>d__40.MoveNext()

How does it locate the files? Am i missing a Path setting maybe?

[razzafrag]

I have a Windows 11 machine as well for testing my own app.  I will install on there and let you know.

[SemlerPDX]

v1.10.3 of VA, Windows 10
Tried the old Windows reboot ... still same error
OpenAI Plugin Error at 2023-05-08 1:06:16 PM:
System.Exception: OpenAI Plugin Error: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at OpenAI_VoiceAttack_Plugin.OpenAIplugin.<PluginContext>d__40.MoveNext()

How does it locate the files? Am i missing a Path setting maybe?

This is nothing on your end, this is in the codebase. I have no idea why this is happening. Could you please try copying the `Microsoft.Bcl.AsyncInterfaces.dll` file to the root VoiceAttack program folder, where the VoiceAttack.exe sits? This is only for a test - we'd undo this whether or not it was successful.

There will be other things we can try

[razzafrag]

Same error on WIndows 11 PC.  Same VA version.
==========================================================================
OpenAI Plugin Error at 2023-05-08 1:29:06 PM:
System.Exception: OpenAI Plugin Error: Could not load file or assembly 'Microsoft.Bcl.AsyncInterfaces, Version=1.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d__19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at OpenAI_VoiceAttack_Plugin.OpenAIplugin.<PluginContext>d__40.MoveNext()

[razzafrag]

Sent you a PM

[SemlerPDX]

For anyone who come in after, we have resolved this issue by updating the OpenAI Plugin for VoiceAttack ... this file needed to be part of the shared assemblies the plugin puts in place:

I have had to release a day-one update. Apparently, the "Microsoft.Bcl.AsyncInterfaces.dll" assembly must also be placed into the VoiceAttack shared assemblies folder along with the other 4 .dll's already put there by this plugin.

 Please update when able - and this will resolve any former issues with this assembly.

Code: [Select]

MINOR UPDATE - v1.1.0.0 Changelog May-8-2023

Fixes/Improvements:
 - Missing "Microsoft.Bcl.AsyncInterfaces.dll" resolved
 - Configuration.CheckSharedAssemblies() added this assembly to VoiceAttack shared assemblies list


[SemlerPDX]

For users who may get an error (or endlessly waiting for a response) once in awhile that results in no response:

An occasional error may occur when contacting the OpenAI API through use of any application, including any VoiceAttack profiles using the OpenAI Plugin such as my own AVCS CHAT profile. The GetResponse phase (and the 'thinking' sound, in AVCS CHAT) may seem like it is looping infinitely, but in fact, it is waiting for a return from the OpenAI API. Eventually, it may result in no response (and any sounds just ending), or users being so confused that they restart VoiceAttack.

I wanted users to know that they can just press the "Stop" button in VoiceAttack, closing and restarting is not required to end any looping sounds or a seemingly endless wait for a response from OpenAI API requests. Users can then immediately try their same input again - though note that any 'continuing session' will have ended due to the use of a "Stop" command, and the contextual memory of recent input/response pairs will be cleared, starting fresh again.

The error which would appear in the "openai_errors.log" may look like this:

Code: [Select]

==========================================================================
OpenAI Plugin Error at 2023-05-10 9:30:17 AM:
System.Exception: OpenAI Plugin Error: Error at chat/completions
(https://api.openai.com/v1/chat/completions) with HTTP status code: 429. Content: {
  "error": {
    "message": "That model is currently overloaded with other requests. You can retry
 your request, or contact us through our help center at help.openai.com if the error
 persists. (Please include the request ID a123456b7cdef890a12b3c456d789e0f in your
 message.)",
    "type": "server_error",
    "param": null,
    "code": null
  }
}

   at OpenAI_VoiceAttack_Plugin.ChatGPT.<Chat>d19.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at OpenAI_VoiceAttack_Plugin.OpenAIplugin.<PluginContext>d40.MoveNext()
==========================================================================


Note on this 'overload' error from OpenAI API:
In the whole of development and testing, and all of my calls to OpenAI API since I got beta access in December, I have never seen these 'overload' messages resulting in an API call failure. There is no way I could have anticipated it beyond the current exception handling which already occurs, however I also read the OpenAI Discord channels often, and we are NOT the only ones surprised and bothered by this seemingly brand new issue with the OpenAI API.  This company has had to scale up faster than any new website in recent history - they went from a hundred thousand users to over ten million in less than two months, and I imagine each month of 2023 that goes by sees more and more tools such this allowing more and more users to access the OpenAI API, so they will need to scale it up accordingly.

We just have to wait and deal with the 'overloads' that happen now and then. Just know it is not a fault of the plugin systems, the libaries I am using to access the OpenAI API, or anything to do with individual user accounts, and there is nothing we as users can change or do better. This is as good as it gets for now, and will only get better in time.

[SemlerPDX]

I've had to release a minor update for AVCS CHAT, an oversight on my part when I was building the custom System Prompts menu command. I had intended to set the value of the SystemPrompt text variable to the contents of the chosen file path, and instead had set it to the literal value of the file path - being nonsense, ChatGPT sessions simply ignore this, and reply to input with standard ChatGPT manners.

Code: [Select]

MINOR UPDATE - v1.1 AVCS CHAT Changelog May-18-2023

*NOTE: small oversight, system prompts not getting read properly, value was path and not file contents

Edited Commands:
 -Open the Chat System Menu (one action, see below)

Fixes/Improvements:
 -Value of 'Set text [OpenAI_SystemPrompt] to [~fileChoice] (save value to profile)' action was literal
 -Changed action to get contents of file at '~fileChoice' rather than the path to that file

Please download the updated version and import the profile into VoiceAttack:
AVCS CHAT Profile for VoiceAttack