Tuesday, April 14, 2020

Comprehensive Shell (CreateProcess & ShellExecuteEx Together)

Module mShellW – Creating New Processes
Download from VBForums
Download from ME

Overview This module greatly extends what you can do with the Shell function in VBA or Visual Basic 5 or 6 (VB6):
  • Everything is Unicode, including the Shelled-to environment.
  • The code works “as-is” in 32 or 64-bit MS Office or in Visual Basic 5 or 6.
  • This module can be run on Windows XP and later.
  • You can start a program by specifying a document with an extension. For example, if you specify D:\MyFiles\abc.docx and MS Word is registered to open .docx files then Word will start with this file.
  • There is a very easy function to use to shell to CMD.EXE that takes care of all of the intricacies in setting up the commandline to CMD.
  • You can specify how long, if at all, the code waits for the shelled-to program to complete before returning. Specify anything from 0 to effectively forever. You can also specify a time for a pop-up window to the user if he wants the program to quit waiting any further. This is a nice escape from a program that you specify “wait for forever to return” and it gets hung up and never quits.
  • You can execute programs from anywhere on your PC. For example, if you are running 32-bit VBA or VB6 but are running on a 64-bit version of Windows, you stil can run any 64-bit program. You can even tell it that you want to run the 64-bit version of calc.exe and many other programs that reside in the Windows\System32 folder. Normally a 32-bit program gets redirected away from this folder “behind the scenes” but we work around that problem.
  • You can run programs that require elevated permissions (the UAC prompt). Normally this is not possible because the Windows CreateProcess function won’t do this but you have the option to 1) keep the privilege level the same as it currenty is, 2) run the new process elevated, 2) run the new process elevated only if it is required (avoids the UAC prompt unless it is necessary) or 3) do not run elevated (fails if elevation is required).
  • If your program is running elevated, you can specify that the shelled-to routine is non-elevated. Usually a shelled-to routine will be elevated if the host program is elevated.
  • You can run batch files (.bat and .cmd), windows scripts and PowerShell all without knowing anything other than the extension of the file to “run.”
  • You can execute a program if you don’t know where it is. For example, you installed Notepad++.exe in “C:\Program Files (x86)\” but as the programmer you don’t know if your users installed it there or if they even installed it at all. And to make matters worse, if they did install it, did they install the 32 or 64-bit version? This code can find the executable file.
  • Normally, shelling is done via the CreateProcess Windows API call, the significance of which is that the new process is independent from the calling program which makes it multi-threaded. You can have the calling program wait until the shell-to routine is complete or you can return control to the calling program and have it continue while the shelled-to program runs in the background. The other common method of shelling to an external program is to use ShellExecuteEx but that routine has limitations which we avoid with CreateProcess. ShellExecuteEx’s main advantage is being able to specify a document instead of an executable and we provide that functionality without using ShellExecuteEx. However, we do include ShellExecuteEx so you can specify it if you really want. Also, ShellExecuteEx is the routine we use to raise permissions when desired or required since CreateProcess lacks that capability.
  • You can use the Windows’ CreateProcess function or ShellExecuteEx functions for your external call. There are pros and cons for each approach; either will work in most instances. The differences are discussed later in this document. CreateProcess is not limited to 255 characters in the commandline. ShellExecuteEx can run programs with elevated privileges.
  • Most programs start a new instance of themselves when we open with CreateProcess even if an existing instance is running. However, this is not true by default for MS Excel, MS Word and Notepad++. If an instance is already running when you specify opening another one, the new document(s) will eb opened in the instance that is already running. This may be okay but know that the new documents share the same memory (if you are running 32-bit versions of these programs this could be limit) and if for some reason something causes the instance to crash then everything in it including the new documents will crash as well. If they are in a different instance they have their own memory space and do not normally crash when another instance crashes. When you use DocShellW in this module you can specify that the shell is to a new instance and if the program is MS Excel, MS Word or Notepad++ then we apply some special code to force them to start a new instance. You can keep the other behavior of putting them al into one instance by specifying NewInstance as False in the call.
  • This module requires the use of my core library mUCCore. It provides my error handling ystem, Unicode support, the basis for support for all flavors of VBA including 64-bit as well as VB6 (compiled or in the IDE) and it contains many routines I use all of the time. It is accompanied by a separate user guide. You don’t need to know what it does in order to use this shelling module but you may find many parts of it useful for your programs.
  • When I test my shell functions I can hard-code various files I want to open or programs to run. I can’t do that with the sample program I have provided because your files are different than mine and are in different locations than mine. So I have included another of my library modules, this one named FileStuffLite. It includes many things not needed for mShellW but I am using a small part of it just for the demo/test. It is not needed for mShellW to function.
  • The code is LARGEADDRESSAWARE. For VB6 runing in 32-bit Windows you can access up to 3 GB of memory and if run in 64-bit Windows (yes, VB6 itself is 32-bit) you can access up to 4 GB. If you are running Excel 2013 or 2016 you can acces up to 4 GB of memory. This shell by itself does not set that up but it is coded such that you won’t get memory crashes from this module when data or code is above the 2 GB mark.

Main Routines ShellW – The main routine for running another program. DocShellW – Open a file with whatever program is registered in Windows for opening that type of file. For example, you could specify “Test.doc” and the file would be opened with Microsoft Word on most PC’s but if .doc files are set to be opened with some other program then that one is used. CMDRun – Run a batch file, a PowerShell file, a program, a document file or just a command prompt via CMD (very similar to the old DOS prompt). You control whether it goes away after executing whatever you want it to do. Support Routines QuoteIfNec – If the specified string (normally a path to a program or a document) has a space character in it Windows generally wants it enclosed in quotes. This routine checks if the string has a space character and if so it encloses the whole string in quotes. ShellHandlesClose – Closes process and thread handles and resets internal variables for them to 0. HasProcessClosed – Determines if a specified process has ended. IsExeRunning – Returns True if the specified EXE file is running. FindEXEPath – Returns the full path of the specified .EXE file. Can be called within ShellW. FindEXEFromDoc – Returns the program in the current PC’s registry that is set to open the specified document file. Can be called from within DocShellW. NOTE - This same package with VBA samples is being uploaded to the Office Development forum here.