Paul Eremeev

Oct 22 2012

Visual Studio commands

Oct 22 2012

Author: Paul Eremeev

Introduction
Using IDE commands
Creating and handling commands in VSPackage. Vsct files
Handling commands through EnvDTE.DTE interfaces
References
Other articles in this series

This article deals with creation, utilization and handling of Visual Studio commands in its extension modules through automation object model APIs and IDE services. The relations between IDE commands and environment UI elements, such as user menus and toolbars, will also be examined.

This article is obsolete. You can read the new version of this article here.

Introduction

Visual Studio commands provide a way for direct interaction with development environment through the keyboard input. Almost all capabilities of different dialog and tool windows, toolbars and user menus are represented by the environment's commands. In fact, main menu items and toolbar buttons are practically commands themselves. Although it is possible for a command not to possess a direct representation in the development environment's UI, as commands are not the UI elements per se, they can be represented by such UI elements as menu items and toolbar buttons.

PVS-Studio IDE extension package integrates several subgroups of its commands into Visual Studio main menu, and these commands serve as one of the plug-in's main UI components (with another one being its MDI toolwindow), allowing a user to control all of the aspects of static code analysis either from the environment's UI or by invoking the commands directly through command line.

Using IDE commands

Any IDE command, regardless of its UI representation in the IDE (or of the lack of it), could be executed directly through the Command or Immediate windows, as well as by starting devenv.exe with the '/command' argument.

The full name of a command is formed according to its affiliation with a functional group, as for example the commands of the 'File' main menu item. Command's full name could be examined in the 'Keyboard, Environment' Options page. Also, the 'Tools -> Customize -> Commands' dialog allows inspecting all of the commands which are currently registered within the environment. This dialog sorts the commands by their respective functional groups and UI presentation types (menus, toolbars), also allowing to modify, add or delete them.

Commands can receive additional arguments which should be separated from the command's name by a space. Let's examine a call to a standard system command of the main menu, 'File -> New -> File' for example, with a passing of additional parameters to it through the Command Window:

>File.NewFile Mytext /t:"General\Text File" 
  /e:"Source Code (text) Editor"

A command's syntax generally complies with the following rules:

command's name and arguments are separated by a space
arguments containing spaces are wrapped by double quotes
The caret (^) is used as an escape character
One-character abridgments for command names can be combined, as for example, /case(/c) and /word(/w) could be presented as /cw

When using the 'command' command-line switch, name of a command with all of its arguments should be wrapped by double quotes:

devenv.exe /command "MyGroup.MyCommandName arg1 arg2"

For the sake of convenience, a command could be associated with an alias:

>alias MyAlias File.NewFile MyFile

Commands integrated into IDE by PVS-Studio extension can be utilized through the /command switch as well. For example, this mode could be used for the integration of our static analysis into the automated build process. Our analyzer itself (PVS-Studio.exe) is a native command-line application, which operates quite similar to the compiler, i.e. it takes a path to the file containing source code and its compilation arguments and then it outputs analysis results to stdout/stderr streams. It's quite obvious that the analyzer could easily be integrated directly into the build system (for instance, into a system which is based on MSBuild, NMake or even GNU Make) at the same level where C/C++ compiler is being called. Of course, such integration already provides us, by its own definition, with complete enumeration of all of the source files being built, with all of their compilation parameters. In turn, this allows for a substitution (or supplementation) of a compiler call by call to the analyzer. Although the described scenario is fully supported by PVS-Studio.exe analyzer, it still requires a complete understanding of build system's internals as well as an opportunity to modify a system in the first place, which could be problematic or even impossible at times.

Therefore, the integration of the analyzer into the build process can be performed in a more convenient way, on a higher level (i.e. at the level of Continuous Integration Server), by utilizing Visual Studio extension commands through the /command switch, for example, by using the PVS-Studio.CheckSolution command to perform analysis on MSVS solution. Of course, such use case is only possible when building Visual C++ native project types (vcproj/vcxproj).

In case Visual Studio is started form a command line, the /command switch will be executed immediately after the environment is fully loaded. In this case, the IDE will be started as a regular GUI application, without redirecting its standard I/O streams to the console that was used to launch the environment. It should be noted that, in general, Visual Studio is a UI based development environment and so it is not intended for command line operations. It is recommended to employ Microsoft MSBuild utility for building inside build automation systems, as this tool supports all of native Visual Studio project types.

Caution should be applied when using Visual Studio /command switch together with non-interactive desktop mode (for example when calling IDE from a Windows service). We've encountered several interesting issues ourselves when we were evaluating the possibility of integrating PVS-Studio static analysis into Microsoft Team Foundation build process, as Team Foundation operates as a Windows service by default. At that moment, our plug-in had not been tested for non-interactive desktop sessions and was incorrectly handling its child windows and dialogs, which in turn lead to exceptions and crashes. But Visual Studio itself experienced none of such issues, almost none to be more precise. The case is, Visual Studio displays a particular dialog for every user when it is started for a first time after an installation, and this dialog offers the user to select a default UI configuration. And it was this dialog that Visual Studio displayed for a LocalSystem account, the account which actually owns the Team Foundation service. It turns out that the same dialog is 'displayed' even in the non-interactive desktop mode, and it subsequently blocks the execution of the /command switch. As this user doesn't have an interactive desktop, he is also unable to close this dialog normally by manually starting the IDE himself. But, in the end, we were able to close the dialog manually by launching Visual Studio for LocalSystem account in the interactive mode through psexec tool from PSTools utilities.

Creating and handling commands in VSPackage. Vsct files

VSPackage extension utilizes Visual Studio command table (*.vsct) file for creating and managing commands that it integrates into the IDE. Command tables are text files in XML format which can be compiled by VSCT compiler into binary CTO files (command table output). CTO files are then included as a resources into final builds of IDE extension packages. With the help of VSCT, commands can be associated with menus or toolbar buttons. Support for VSCT is available starting from Visual Studio 2005. Earlier IDE versions utilized CTC (command table compiler) files handling their commands, but they will not be covered in this article.

In a VSCT file each command is assigned a unique ID — CommandID, a name, a group and a quick access hotkey combination, while its representation in the interface (if any) is specified by special flags.

Let's examine a basic structure of VSCT file. The root element of file is 'CommandTable' node that contains the 'Commands' sub-node, which defines all of the user's commands, groups, menu items, toolbars etc. Value of the "Package" attribute of the "Commands" node must correspond with the ID of your extension. The "Symbols" sub-node should contain definitions for all identifiers used throughout this VSCT file. The 'KeyBindings' sub-node contains default quick access hotkey combinations for the commands.

<CommandTable"http://schemas.microsoft.com/VisualStudio/2005-10-
18/CommandTable">

    <Extern href="stdidcmd.h"/>
    <Extern href="vsshlids.h"/>
  <Commands>
    <Groups>
    ...
    </Groups>
    <Bitmaps>
    ...
    </Bitmaps>
  </Commands>
  <Commands package="guidMyPackage">
    <Menus>
    ...
    </Menus>
    <Buttons>
    ...
    </Buttons>
  </Commands>

  <KeyBindings>
    <KeyBinding guid="guidMyPackage" id="cmdidMyCommand1"
 editor="guidVSStd97" key1="221" mod1="Alt" />
  </KeyBindings>
  <Symbols>
    <GuidSymbol name="guidMyPackage" value="{B837A59E-5BF0-4190-B8FC-
FDC35BE5C342}" />
    <GuidSymbol name="guidMyPackageCmdSet" value="{CC8B1E36-FE6B-48C1-
B9A9-2CC0EAB4E71F}">
      <IDSymbol name="cmdidMyCommand1" value="0x0101" />
    </GuidSymbol>
  </Symbols>
</CommandTable>

The 'Buttons' node defines the commands themselves by specifying their UI representation style and binding them to various command groups.

<Button guid="guidMyPackageCmdSet" id="cmdidMyCommand1"
priority="0x0102" type="Button">
  <Parent guid="guidMyPackageCmdSet" id="MyTopLevelMenuGroup" />
  <Icon guid="guidMyPackageCmdSet" id="bmpMyCommand1" />
  <CommandFlag>Pict</CommandFlag>
  <CommandFlag>TextOnly</CommandFlag>
  <CommandFlag>IconAndText</CommandFlag>
  <CommandFlag>DefaultDisabled</CommandFlag>
  <Strings>
    <ButtonText>My &Command 1</ButtonText>
  </Strings>
</Button>

The 'Menus' node defines the structure of UI elements (such as menus and toolbars), also binding them to command groups in the 'Groups' node. A group of commands bound with a 'Menu' element will be displayed by the UI as a menu or a toolbar.

<Menu guid=" guidMyPackageCmdSet" id="SubMenu1" priority="0x0000"
type="Menu">
  <Parent guid="guidMyPackageCmdSet" id="MyTopLevelMenuGroup"/>
  <Strings>
    <ButtonText>Sub Menu 1</ButtonText>
  </Strings>
</Menu>
<Menu guid=" guidMyPackageCmdSet" id="MyToolBar1" priority="0x0010"
type="Toolbar">
</Menu>

And finally, the 'Groups' element organizes user's IDE command groups.

<Group guid="guidMyPackageCmdSet" id="MySubGroup1" priority="0x0020">
  <Parent guid="guidMyPackageCmdSet" id="MyGroup1" />
</Group>

To include vsct file into MSBuild-based VSPackage project, it is necessary to insert the following node used for calling VSCT compiler into your csproj project file (note, that in the auto-generated project created from an SDK template, a vsct file will be already included in a project):

<ItemGroup>
  <VSCTCompile Include="TopLevelMenu.vsct">
    <ResourceName>Menus.ctmenu</ResourceName>
  </VSCTCompile>
</ItemGroup>

Next, the ProvideMenuResource attribute of your Package-derived class should point to this node that you've inserted into your project earlier:

 [ProvideMenuResource("Menus.ctmenu", 1)]
...
public sealed class MyPackage : Package

Assigning handlers to the commands defined in a VSCT file is possible through a service that is available through the IMenuCommandService. A reference for it can be obtained by the GetService method of your Package subclass:

OleMenuCommandService MCS = GetService(typeof(IMenuCommandService)) as
  OleMenuCommandService;

Let's examine an example in which we assign a handler to a menu command (this command should be declared in a vsct file beforehand):

EventHandler eh = new EventHandler(CMDHandler);
CommandID menuCommandID = new CommandID(guidCommand1CmdSet, id); 
//ID and GUID should be the same as in the VCST file
OleMenuCommand menuItem = new OleMenuCommand(eh, menuCommandID);
menuItem.ParametersDescription = "$";
MCS.AddCommand(menuItem);

To obtain command's arguments while handling its invocation, the EventArgs object should be casted into OleMenuCmdEventArgs:

void CMDHandler(object sender, EventArgs e)
{                 
  OleMenuCmdEventArgs eventArgs = (OleMenuCmdEventArgs)e;
                   if (eventArgs.InValue != null)
                       param = eventArgs.InValue.ToString();
  ...
}

Handling commands through EnvDTE.DTE interfaces

The EnvDTE.DTE automation object allows for a direct manipulation (creation, modification and execution) of commands through the dte.Commands interface and dte.ExecuteCommand method. Utilizing the Automation Object Model for invoking, modifying and creating IDE commands, as opposed to using VSCT mechanism available only for VSPackage, allows the interaction with IDE commands from within Add-In extension packages as well.

The DTE automation object allows a direct creation, modification and invocation of commands through the DTE.Commands interface. A command can be directly added to the IDE by Commands.AddNamedCommand method (but only for an Add-In extension):

dte.Commands.AddNamedCommand(add_in, "MyCommand", "My Command", 
  "My Tooltip", true);

The command added in this way will be preserved by the IDE — it will reappear in the menu after IDE restart, even if the extension which created the command is not loaded itself. That's why this method should only be utilized during the first initialization of an Add-In module, after its installation (this is described in the article dedicated to Visual Studio Automation Object Model). The OnConnection method of an Add-In contains a special initialization mode which is invoked only for a single time in the module's entire lifetime. This method can be used to integrate UI elements into the IDE:

public void OnConnection(object application, 
                         ext_ConnectMode connectMode, 
                         object addInInst, ref Array custom)
{
  switch(connectMode)
  {
      case ext_ConnectMode.ext_cm_UISetup:
          ...
          break;

      ...
  }

}

The EnvDTE.Command interface represents a single IDE command. This interface can be used to modify a command which it references. It permits managing IDE commands from either a VSPackage, or an Add-In module. Let's obtain a reference to the EnvDTE.Command object for our custom command 'MyCommand1' and utilize this interface to assign a 'hot-key' to it for a quick access:

EnvDTE.Command MyCommand1 = 
MyPackage.DTE.Commands.Item("MyGroup.MyCommand1", -1);
MyCommand1.Bindings = new object[1] { "Global::Alt+1" };

The quick-access combination assigned to MyGroup.MyCommand1 will now be available through 'Keyboard, Environment' environment settings dialog.

As was mentioned before, Visual Studio command is not a UI element by itself. The Commands.AddCommandBar method allows the creation of such UI elements, as main menu items, toolbars, context menus and the association of these elements with user-created commands.

CommandBar MyToolbar = dte.Commands.AddCommandBar("MyToolbar1", 
  vsCommandBarType.vsCommandBarTypeToolbar) as CommandBar;
CommandBar MyMenu = dte.Commands.AddCommandBar("MyMenu1", 
  vsCommandBarType.vsCommandBarTypeMenu) as CommandBar;
CommandBarButton MyButton1 = MyCommand1.AddControl(MyToolbar) as 
  CommandBarButton;
MyButton1.Caption = "My Command 1";

The 'Delete' method of Command/ CommandBar objects could be utilized to remove a command or toolbar from IDE.

MyCommand1.Delete();

In general, it is not recommended creating commands each time an Add-In plug-in is loaded and removing them each time it is un-loaded, as such behavior could slow-down the initialization of IDE itself. Even more, in case the OnDisconnect method is somehow interrupted in the process, it is possible that the user commands will not be completely deleted from the IDE. That is why it is advised that the integration, and subsequent removal, of IDE commands should be handled at the times of module's installation/uninstallation, as for example, by obtaining DTE interface reference from a stand-alone installer application. The initialization of Add-In modules and acquisition of DTE references is thoroughly described in the article devoted to EnvDTE Automation Object Model.

Any IDE command (either custom or default one) could be called by the ExecuteComand method. Here is the example of invoking our custom MyCommand1 command:

MyPackage.DTE.ExecuteCommand("MyGroup.MyCommand1", args);

To handle command execution, an Add-In extension should be derived from the IDTCommandTarget interface and it should also implement the Exec method:

public void Exec(string commandName, 
  vsCommandExecOption executeOption, ref object varIn, 
  ref object varOut, ref bool handled)
{
  handled = false;
  if(executeOption == 
    vsCommandExecOption.vsCommandExecOptionDoDefault)
  {
    if(commandName == "MyAddin1.Connect.MyCommand1")
    {
      ...
      handled = true;
      return;
    }
  }
}