Ability for plugin to execute VoiceAttack commands

[cmdrmcdonald]

It would be very handy if a plugin could execute VoiceAttack commands by itself, ideally calling the command by name.

This feature would make it much easier for the plugin to be responsive to changes it detects from external data sources, and make the plugin much more standalone rather than requiring additional setup by the user (that is often carried out incorrectly).


EDIT by Pfeil: Commands can now be executed from plugins/inline functions, variables can be scoped using prefix characters (as detailed in the "Advanced Variable Control (Scope)" section of VoiceAttackHelp.pdf)

[Gary]

I have tested an implementation of this and it works...  The code is in VA, but I have been dragging my feet as it will require an interface.  All the other parameters are generic, but whatever object that gets passed in will not.  The idea was that the object passed to the plugin would be able to control various aspects of VA (including command execution).  This would continue to expand as need and time permit...  Will require additional thought   I guess in other words it's coming, but it will be a bit.

[cmdrmcdonald]

Sounds great.  Given the potential complexity of this it might be a good idea to start with a subset of all available features (just allow calling of other VA commands, for example  and build up when the problems with the first one have been ironed out.

[Elmegaard]

I may have misunderstood this, but I have solution. At least for this problem. 

I guess you can work around this, but it will become a complex solution.

You can create a virtual sound card with different software. Then have your microphone input to this software along with a script you have created, which contain voice commands (pre-recorded). Have this script called from both keybindings and from VoiceAttack (if possible?).

This way only VoiceAttack will hear your command and not other devices using the microphone.

I have created a small model illustrating this (epic paint skills ftw):


This example would of course automatically loop the "Hello" command, but other more complex commands could be chained together like this.

[Pfeil]

It would be very handy if a plugin could execute VoiceAttack commands by itself, ideally calling the command by name.

I guess you can work around this, but it will become a complex solution.

I would assume Gary is working on a more flexible solution, but VoiceAttack can already execute a command by name, if that's all you need.

You have the plugin set a text value, and use a token for the "Execute by name (Advanced)" feature of the "Execute Another Command" action:

Code: [Select]

Execute command, '{TXT:VariableFromPlugin}' (by name) (and wait until it completes)


[cmdrmcdonald]

Thanks for the responses, but the options presented allow me to run a command from a script, not from a plugin.  The point of the request is to avoid the current situation where an event loop or some other permanently running script is required to interface back to the plugin.

[cmdrmcdonald]

At this point the thing I really need is to be able to set variables from inside my plugin.  If I can do this I can remove a chunk of the script-side logic that is catching people out.  Any thoughts on adding an VA_? call to do this?

[Gary]

I've got it down to add variable manipulation from the VA object.  Just a little bit jammed up at the moment o_O

[cmdrmcdonald]

I've got it down to add variable manipulation from the VA object.  Just a little bit jammed up at the moment o_O

I understand.  If you could sneak this in before the next release it would be a massive help.

[Gary]

I will do my best to get this one through

[Gary]

Got the, 'execute command' portion going.  It has an optional, 'wait for return' flag.

If it's off, the command is fire and forget.  If it's on, it execution flow does not resume until the command completes.


I've decided that the best thing to do is to pass in a VA proxy class that is of type, 'dynamic'.  You won't have Intellisense, but you (we) won't have to reference anything and we don't have to worry about interfaces getting out of sync between what I am imagining to be a LOT of builds.  The proxy will be able to tell you what its version is, as well as the version of VA so you can execute accordingly.  Fun stuff!

[cmdrmcdonald]

Sounds exciting.  Can't wait to remove the event loop and associated pain and have VoiceAttack's variables be automatically up-to-date thanks to this.

[Gary]

To continue what I was saying earlier, I really (really) don't want to have to make anybody reference anything, and *I* only want to reference the bare minimum of things I need to reference with VA.  Again, to get around the shared interface references, I would like to pass the VA proxy class as, 'dynamic'.  That means the plugins must be .net framework 4.0+.  I don't think that's much of a hardship here in almost 2017.  It also means late binding... which is really not that much of a concern in this context I'm sure.  What it does buy is not having to update SDKs each time something is added.

So, question for you all.  Right now, Invoke1 looks like this:

Code: [Select]

public static void VA_Invoke1(String context, ref Dictionary<string, object> state, ref Dictionary<string, Int16?> shortIntValues, ref Dictionary<string, string> textValues, ref Dictionary<string, int?> intValues, ref Dictionary<string, decimal?> decimalValues, ref Dictionary<string, Boolean?> booleanValues, ref Dictionary<string, DateTime?> dateValues, ref Dictionary<string, object> extendedValues)

Init1 is almost exactly the same (just missing the context).

This monstrosity is the product of iterations of something that was never intended to talk directly to VA, and attempt to keep things somewhat isolated and transactional (mostly so variable access was controlled and user-visible).  What I *could* do is stuff the vaProxy in there as another parameter.  No big deal, as it makes it consistent with every other iteration of the interface.  It would be redundant, as every parameter would be contained in the proxy class.

=OR=

I can go this route:

Code: [Select]

public static void VA_Invoke1(dynamic vaProxy)
To get the context:  vaProxy.Context
To get an integer:  vaProxy.GetInteger("someVarName")
To set an integer:  vaProxy.SetInteger("someVarName", 55)

and so on

I like the second option, as it cuts down on the ever-growing parameter list and is certainly cleaner and easier to read.  I don't like it because everything is now hidden behind one mysterious class, and it will make it where the plugin devs will have to rewrite their code somewhat.

The third option is to have both...  where there would be an Invoke2(dynamic vaProxy).  If VA sees that Invoke2 is being used on init, it will use it instead of Invoke1.

[Gary]

I am going to be out of pocket until later on Tuesday (late afternoon or evening).  It's checkup time again (yikes).

I going to upload what I've got so far into the, 'unofficial' bin:  http://www.voiceattack.com/unofficial
I think it's mostly (if not all) the elements that would be in a first release.

I went with the single-parameter approach, as I'm ready to ditch the endless parameter list

Attached is a stripped-out plugin class (like you would find in the samples folders).  It shows a simple Invoke1 with commented out code using dynamic 'vaProxy'.  Basically it's all the same stuff you had access to before, just moved around (and there's only one dictionary (session state)).

If you decide to run this, the log will complain, 'Taskbar files not found'.  If you want to try out the taskbar feature, just download the attached dll and make sure it's in the same directory as VA:  http://www.voiceattack.com/filesend.aspx?id=mstaskbar.dll   Windows will complain that it's not from a known source and try to block it... just a heads-up.
The taskbar icon now has buttons to control listening, plus a right-click jump list with some stuff that may be handy and needs that dll (still works without it, just naggy).

Also attached is the set of commands that are currently available.

[cmdrmcdonald]

Thanks for this, I'll have a play and let you know how it goes.

Good luck with the checkup.

[Antaniserse]

Interesting.

Given that commands are executed by name, and so they will be either hard coded in the plugin or inferred by means of context/variables set in the profile and carried over, would it be possible for 'ExecuteCommand' to return a boolean, TRUE if command name actually found in the profile, FALSE otherwise, or maybe add a 'bool CommandExist(name)' function to perform a simple name check beforehand?

[cmdrmcdonald]

I suspect that I'm misunderstanding something, but I've tweaked your example as follows:

Code: [Select]

        public static void VA_Init1(dynamic vaProxy)
        {
            vaProxy.SetTextValue("vastr", "foo");
        }

The plugin shows as loaded, but if I try to write {TXT:vastr} to the log it comes back as Not Set.

Also, will I be able to retain the vaProxy provided here and use it to update values ad-hoc?  I assume that I will, otherwise we don't have the push fucntionality, but wanted to confirm and find out if there are any caveats.


Separately, regarding the argument selection I think that going with the single vaProxy is fine.  I would, however, suggest that you do move these to VA_*2 versions, and keep the VA_*1 versions around for a release or two to allow people to migrate to the new style (obviously without the additional functionality).

[Gary]

As long as Invoke1 does not return, the proxy should stay in scope.  This brings up a good point, which is what to do if you do a STOP command and a plugin is setup to never return.  Since I would think it would be detrimental to a plugin that's acting like a service to stop whenever, 'stop all commands' is given, I will be adding another function to the interface that can be optionally monitored... the plugin developer can decide what contexts to handle.

I just tried the, 'SetTextValue' and I'm having no problem with it.  I *do* consistently make the mistake of not compiling before testing.  Note:  I changed it from, 'SetTextValue' to 'SetText' to fall in line with everything else.  That will be in the next release.

I realized that I failed to put the plugin functionality in its own app domain to try to keep them from bringing VA down.

In regards to the interfaces and migration, VA is fully backward-compatible all the way to the first version, so if anybody is not ready to make the jump they're not dead in the water.

bool CommandExist(name)

Anything for you, Antaniserse   I'll make sure to get that in next as well.

Yeah... the commands WILL execute by guid, *but* the guids change on export/import.

[cmdrmcdonald]

As long as Invoke1 does not return, the proxy should stay in scope.  This brings up a good point, which is what to do if you do a STOP command and a plugin is setup to never return.  Since I would think it would be detrimental to a plugin that's acting like a service to stop whenever, 'stop all commands' is given, I will be adding another function to the interface that can be optionally monitored... the plugin developer can decide what contexts to handle.

Hmm... I want to avoid a continuously-running command at all if possible, so that I can push updates to variables as they occur without the problems that we have at the moment with getting users to run a command on startup (two reasons for this: one is some users struggle to configure it, two is that other profiles often have their own command on startup which causes problems).  So ideally in your code you'll run VA_Init1() and then a new function like VA_Monitor1() that doesn't need to return.  In my code I'll loop within VA_Monitor1() to watch the log and push updated variables through the proxy.  I'll still use VA_Invoke1() when a use wants the plugin to do something actively.

Alternatively, if the vaProxy passed to VA_Init1() remains valid even after VA_Init1() returns I can spin off a thread from within my VA_Init() and monitor from there.

Is this going the be possible with the monitor that you talk about above?

I just tried the, 'SetTextValue' and I'm having no problem with it.  I *do* consistently make the mistake of not compiling before testing.  Note:  I changed it from, 'SetTextValue' to 'SetText' to fall in line with everything else.  That will be in the next release.

I realized that I failed to put the plugin functionality in its own app domain to try to keep them from bringing VA down.

Okay; I'll give it another go.

In regards to the interfaces and migration, VA is fully backward-compatible all the way to the first version, so if anybody is not ready to make the jump they're not dead in the water.

That's great to hear.

[Gary]

Could you spin up the new thread on init and just use the passed proxy to check variables/state?
The snippet I pasted below just loops and calls the, 'alive' command (which just writes, 'alive' to the log).  I have a command that sets the, 'killprocess' variable if I want the loop to stop.
I then call another test command that calls the invoke1 and it increments the static integer and writes it to the log on return.

This way you can have the monitoring process kick off and you can even have a separate context to start it and stop it (no user initialization needed).

Code: [Select]

public static void VA_Init1(dynamic vaProxy)
{
    MyPollingFunction(vaProxy);
}

public static void MyPollingFunction(dynamic vaProxy)
{
    System.Threading.Thread thread = new System.Threading.Thread(delegate()
    {
         while (true)
         {

              vaProxy.ExecuteCommand("alive", true);
              if (vaProxy.GetBoolean("killProcess") ?? false)
                  break;
              System.Threading.Thread.Sleep(1000);
          }
      });
      thread.IsBackground = true;
      thread.Start();
  }

  private static int _count = 0;
  public static void VA_Invoke1(dynamic vaProxy)
  {
       _count++;
       vaProxy.SetInt("abc", _count);
   }


Hoping this is what you're looking for!

[cmdrmcdonald]

Yep I can do that, but when the main thread returns from VA_Init1() won't it invalidate the proxy as per your previous comment?

If not then I'm fine to do this (and in fact this was my original plan).

[cmdrmcdonald]

I'm also still failing to use the vaProxy.  When I try a SetInt on it I have an exception

'object' does not contain a definition for 'SetInt'

This is copying your example from above.  Is it possible that the unofficial exe you provided isn't up-to-date?  I'm seeing 1.5.12.18 (prerelease) as the version number in the options window.

[Gary]

We're probably out of sync.  I haven't done a code check-in yet with this, so I can't compare versions   I'll try to get something out tonight that will work better!

[cmdrmcdonald]

Any luck with this? I'd love to have this up and running.

[Gary]

Working to make this a full beta.  I should have something out later today.  Just getting everything tied up.

[Gary]

I've removed the, 'listening' functions as well as the SetProfile function.  These can be done easily in a command that's a little more visible to the user.  I've also constrained vaProxy for each of the functions.  So, Context is only available in Invoke1, and nothing but SessionState is available in VA_Exit1.

Added VA_StopCommand to the required interface.

New functions:  CommandExists, WriteToLog, ParseTokens

[Gary]

New beta is out there.  Updated the help document a bit for the new stuff 

[cmdrmcdonald]

I must be an idiot and doing something wrong.  I tried to build a very simple plugin that just wrote to the log on Init() and I receive the following message in the log:



I've double-checked and this is 1.5.12.19

The code is:

Code: [Select]

    public class VoiceAttackPlugin
    {
        public static string VA_DisplayName()
        {
            return "My New Plugin";  //this is what you will want displayed in dropdowns as well as in the log file to indicate the name of your plugin
        }

        public static string VA_DisplayInfo()
        {
            return "My VoiceAttack Plugin\r\n\r\nThis is just a sample.\r\n\r\n2016 VoiceAttack";  //this is just extended info that you might want to give to the user.  note that you should format this up properly.
        }

        public static Guid VA_Id()
        {
            return new Guid("{D31D2BB5-5E09-4e75-9BBE-FE2A02E3D04E}");  //this id must be generated by YOU... it must be unique so VoiceAttack can identify and use the plugin
        }

        public static void VA_Init1(dynamic vaProxy)
        {
            vaProxy.WriteToLog("Hello world", "red");
        }

        public static void VA_Exit1(dynamic vaProxy)
        {
        }

        public static void VA_StopCommand()
        {
        }

        public static void VA_Invoke1(dynamic vaProxy)
        {
        }
    }

If the code looks right the only thing I can think of is some setup in the project that isn't working.  The full project is available at https://dl.dropboxusercontent.com/u/14331465/VATest.zip  Please could you take a look and let me know what I've done wrong?

[Gary]

I popped this in, compiled and it worked straight away (shows, 'Hello world in the log with a red icon).

Does it initialize when you take out that line?

Are you sure you are building to the right directory?

Are you using .net framework 4 (not 4.5... probably doesn't matter but you never know)?

Attached is what I built just now.

[Gary]

Oooooooooooooooooooooooooooooooooooooooooooooooooooh.  Ok... hang on... don't do anything yet.