Pausing/Resuming listening based on Mouse location?

[TheDocBennett]

I work between a PC and a Laptop. The PC has three monitors and is where I do most of the "work" while the Laptop is used mostly for emails/chat/etc. Now, to get my PC and the laptop controlled by one pair of keyboard/mouse, I use the software/wireless KVM https://www.stardock.com/products/multiplicity/ with really good results.

For a while now, when I've had something on the Laptop, like Slack, that I wanted to respond to a VA command going into the PC I had to use the "write to file" then an Eventghost directory watcher on the Laptop to see that changed file and take the actions I want.

I've got VA installed and working on the Laptop and I even have my mic line into the PC piped into the Laptop as well, so VA can hear me just fine.

When I drag my mouse from my PC's main screen down through the bottom edge it shows up on the Laptop's screen and my keystrokes are passed there as well. Voiceattack events are not passing through this "link" however.

My goal is to get the "listening" of VA on my PC to halt when my mouse cursor gets "parked" at the bottom of the screen and for VA on my Laptop to listen whenever the mouse is NOT parked at the top of the screen.

[Pfeil]

Something like this could work:

Code: [Select]

Begin Text Compare : [{STATE_LISTENING}] Equals '1'
    Set Boolean [~lastListeningState] to True
Else
    Set Boolean [~lastListeningState] to False
End Condition
Start Loop While : [1] Equals [1]
    Begin Device State Check :  Mouse Y (Screen) Is Greater Than 1070
        Begin Date Compare : [~timeAtOffPosition] Has Been Set
            Begin Date Compare : [~timeAtOffPosition] Is Less Than Current Date/Time
                Begin Boolean Compare : [~lastListeningState] Equals True
                    Stop VoiceAttack listening
                    Set Boolean [~lastListeningState] to False
                End Condition
            End Condition
        Else
            Set date [~timeAtOffPosition] to [~timeAtOffPosition] plus [500] milliseconds
            Set date [~timeAtOnPosition] value to [Not Set]
        End Condition
    Else
        Begin Date Compare : [~timeAtOnPosition] Has Been Set
            Begin Date Compare : [~timeAtOnPosition] Is Less Than Current Date/Time
                Begin Boolean Compare : [~lastListeningState] Equals False
                    Start VoiceAttack listening
                    Set Boolean [~lastListeningState] to True
                End Condition
            End Condition
        Else
            Set date [~timeAtOnPosition] to [~timeAtOnPosition] plus [500] milliseconds
            Set date [~timeAtOffPosition] value to [Not Set]
        End Condition
    End Condition
End Loop

First, the current listening state is checked, and stored into a boolean variable. This should be more efficient to run in a loop, as the only other alternative is to use a token, which would need to be re-parsed every iteration.
This state is intentionally not re-checked, enabling you to toggle listening on either machine by other means, and having it stay that way until you move the cursor into or out of the "parking" zone.

The loop checks a boolean variable named "1" against itself, thus producing an infinite loop (make sure to check the "Evaluate 'Not Set' as false" option).

The cursor position is checked only for vertical position (if the cursor can only transition while on a specific screen of your PC, you'll want to add a check for the the horizontal as well); Adjust this value depending on the size of either screen (if the cursor is consistently "parked" at the very bottom, you don't need the 10 pixel margin), and invert it for the one that'll be below the other
E.G.

Code: [Select]

Begin Device State Check :  Mouse Y (Screen) Is Less Than 10
for the laptop

I also added a time delay. The example is configured so that the cursor must be at the bottom edge of the screen for at least 500ms before listening is toggled, to avoid toggling each time the cursor just happens to touch the bottom edge.


As with any VoiceAttack command, this will stop if you switch profiles, or all commands are stopped.

[TheDocBennett]

This is bloody brilliant!

Can I buy you a beer?

One question - when I was trying to achieve this with AHK I was able to compare with the machine's name... so the same script could run concurrently and not require different code/profiles.

Is there a VA token option to identifying a machine, then it could use that for determining the top/button buffer zone for turning off listening?

[Pfeil]

There is no native method for it, but this can be done with a very simple inline function.

Replace everything up to and including

Code: [Select]

Begin Device State Check :  Mouse Y (Screen) Is Greater Than 1070with

Code: [Select]

Begin Text Compare : [{STATE_LISTENING}] Equals '1'
    Set Boolean [~lastListeningState] to True
Else
    Set Boolean [~lastListeningState] to False
End Condition
Inline C# Function: Get machine name
Start Loop While : [1] Equals [1]
    Begin Condition : ([~machineName] Equals 'PC' AND Mouse Y (Screen) Is Greater Than 1070) OR ([~MachineName] Equals 'Laptop' AND Mouse Y (Screen) Is Less Than 10)

Where "Get machine name" contains:

Code: [Select]

using System;

public class VAInline
{
public void main()
{
VA.SetText("~machineName", Environment.MachineName);
}
}
Obviously you'll want to replace "PC" and "Laptop" with the actual machine names.
If you're not sure of the exact name, you could write it to the log by inserting

Code: [Select]

Write [Blue] '{TXT:~machineName}' to log
after the inline function.


The last condition is created using the condition builder (either select that option when initially creating it from the "Other >" context menu, or right-click the existing condition and choose "Edit With Condition Builder" from the context menu).
Pay close attention to how the conditions are ordered and grouped.

I didn't benchmark it, but chances are a string compare is more efficient than getting the current mouse cursor position, so I'd go with this order to do a maximum of two string compares each cycle, but only one cursor position check (realistically it probably doesn't matter on a modern machine, though).

[TheDocBennett]

You continue to amaze me.

This worked like a charm, man. Seriously, how can I buy you coffee/tea/beer?