Actually checking the recording state would require ShadowPlay to have an interface for that.
This project may have components that could be adapted for such a purpose, by someone with the appropriate knowledge.
Otherwise, you can keep track of what the last known recording state should be. Essentially, assuming ShadowPlay is not recording already when VoiceAttack starts, it could be assumed that recording will start the first time the command is executed, and that it will stop the second time the command is executed, and so on.
E.G.
Press A key and hold for 0,02 seconds and release
Set Boolean [recordingState] to Toggle
Begin Boolean Compare : [recordingState] Equals True
Say, 'ShadowPlay should now be recording'
Else
Say, 'ShadowPlay should now stop recording'
End Condition
Obviously, if ShadowPlay was already recording when VoiceAttack was started, or it stops recording after you have executed the command, for any reason, that assumed state will no longer match the actual state.
Press F1 while VoiceAttack has focus to open VoiceAttackHelp.pdf in your default PDF viewer, which contains information on VoiceAttack's features, including those used in the above example.
These topics may also be of use:
Control flow (If, Else, ElseIf, Loop, Jump) basicsVariables and tokens summed up