In JMS, we suggest not to remove the directory but rename it with another name.
When you rename it instead of removen this allows you to login with standard Joomla and also you keep the copy of your installation directory.
The installation directory is required to allow JMS install fresh slave website using the standard Jommla installation.
Therefore the installation directory must be restored once JMS has installed the patches.

JMS contain a ZIP with the latest official Joomla installation directory.
I suggest you keep yours in case where you use a specific language distribution or packaging.

Therefore, once you have installed the JMS patches, if you want to re-use your installation directory instead of the JMS installation directory, you could rename it again to restore its original "installation" directory.

The objective is to keep you Joomla as close as possible than it was when you have installed it and reduce as much as possible the patches and modifications.