Using cloud dictation for text chat

[Rhaedas]

Disclaimer: I'm still playing with this, so it's not a polished product, plus the profile commands here come out of my larger one, so there's a few included things that are specific to my housekeeping of variables and of Elite Dangerous' comms chat window. But it's not very complex what's here, so could easily be edited and used for similar things. Also, while the site I'm utilizing is open access, if it goes away (like other APIs have in the past) obviously this won't work anymore.

So the basic concept: I found a site that will listen to your microphone when triggered and through Google's cloud processing (it only works in Chrome right now, using HTML5 stuff that no one else supports yet) dictate what you say into its window, then let you do things with that text result. VA has the ability to focus on different windows as well as manipulate the mouse and keys, so I had an inspiration, given past lackluster results with using Windows built in dictation results.

Here's how it works. When you give a command to start a message in the chat window, VA first goes to the microphone on this site in Chrome and turns it on. Then it goes to Elite Dangerous' window, opens the comms panel, and inserts the first few keys to either do a local broadcast, direct, or a reply, depending on what command you spoke. When you're done speaking the text, you use a key to signal that you're finished. That triggers VA to go to Chrome, cut off the mic, put the text into the clipboard, do some housekeeping stuff on the site (may not be needed but I wanted ever time to start clean) and then paste the clipboard into the comms panel and hits Enter, delivering the message. I also have a command to cancel, if the dictation was bad or if you changed your mind.

If you decide to try or use this, the main thing you'll have to adjust in the VA code is where the mouse goes to click each thing. I have the window sized small, so if you got it the same dimension it should work as is, given it's using coordinates from the left corner. You'll also have to have both Chrome open at the site plus the game or app, as VA will get an error if it doesn't find it. Also, the commands for the chat panel may need editing, as they're mine in the game, but may differ for others' ED and certainly would need to be changed for a different purpose.

The profile attached is the several commands I am using for this particular function. The website is https://speechnotes.co/. If you use Chrome for other stuff you should be able to open it in a new window or maybe even in just a tab will work. I don't, so I just installed it and made this the start page.

Seems like a nice wall of text there. Let me know of any questions, suggestions, improvements, etc.

[axonMagnus]

this is absolutely brilliant thank you very much , i dont have elite dangerous but i would like to use it for normal operation can you please for the love of Gabe give me one that will let me dictate to any application ?

[Rhaedas]

Sure thing. These are the four commands needed, you can add to the begin and send dictation commands as needed or rename them.

In the begin dictation if you need to add anything before the dictation this is where you'd do it. For Elite I had moved to the comms window and sent "/l " to be in front of the text. If you do add anything that needs to be sent to an app, also check the box "send this to" and find that app in the pull down box. The app will need to be running for it to show.

In the send dictation you'll also need to add whatever needs to come after the dictation text to send it, such as an enter key press, a mouseover to where a submit button is, etc. Same thing with checking the "send this to" box and finding your app in the pull down as in begin. You can change the trigger key to anything you want, the numeric zero was convenient and not used in my game.

And lastly, you'll probably have to alter the coordinates for the mouse clicks in the begin and end chrome. Another option is to resize that window until they match up to what I have set. I have it as small as possible before icons started overlapping.

Two other modifications I did after the first example profile. One that is in this attached on was to cut off VA listening until the dictation was finished. Mainly because I happened to run across a phrase that I used while dictating a message, and of course it got both triggered in VA and dictated. the end dictation starts VA up again. The other probably isn't necessary, but I changed Chrome to start up in incognito mode. I noticed that if cookies aren't saved, any startup will have the session number begin at 1 again. Just felt neater to do that, but I doubt it matters if you add them up instead.

Let me know if I missed anything. If you know how to basic edit VA commands, this should be straight forward.

[axonMagnus]

thanks you very  much my good sir , i managed to make it work , thoughi cut down the commands just to two and its pretty impressive. After i finished i sent an email to the dev of that tool asking him to impliment shortcuts to the tool for listening and stoping and copying the text to the clipboard ti would make our lives so much easir and we would be able to run it with out bring chrome to focus.

right now am looking for chrom mouse macro extensions that way we can make it run on the background some how with out making it switch window to focus .  i will report back with results

once again thank you for  your help

"Doubt kills more dreams than failure ever will"

[axonMagnus]

hey buddy i have some great news turns out google docs was recently implimented with the google speech to text and the best part is its got shortcuts no more mouse macros . here is my solution

http://prntscr.com/b9brrb

http://prntscr.com/b9bsep

the the control and delete , delets my commands " stop dictating" from the dictation very simple and effective just like i like it , have fun buddy. i khow  you will love this .i believe we wont have to worry about chrome coming into focus any more

take care buddy 

"Doubts kill more dreams than failure ever will"