After finishing the game Seal Guardian and taking some rest, I was recently back to refactoring the engine code of the game Seal Guardian. In this game, the engine has the ability to hot-reload all asset type from texture, shader, game level to Lua script. But it lacks the ability to hot-reload C/C++ files. So I decided to spend some time on finding resources about hot reload C/C++. It turns out hot-reload C/C++ is not that trivial on Windows as the PDB file is locked. And I found this approach of patching the PDB path inside DLL looks interesting. So I gave it a try and the sample program is uploaded to here (only tested with Visual Studio Community 2017).
First try
Because the PDB file path is hard coded inside the DLL file, the approach used by cr.h is to correctly parse the DLL file to find the PDB file path and replace it with another new file path according to the Portable Executable format.
So I tried something similar, but different from cr.h, instead of generate a new DLL/PDB file name every time the DLL get re-compiled, I use a fixed temporary name (I don't want to have many random files inside the binary directory after several hot reload...) For example, when Visual Studio generate files:
- abc.dll
- abc.pdb
- ab_.dll
- ab_.pdb
The reason I don't choose a more meaningful name like abc_tmp.dll is because I worry that having a longer file name length than the original name may mess up the offset values stored inside the DLL. So I just replace the last character with an underscore character.
This approach works and every time I start debug without debugger by pressing Ctrl+F5 in Visual Studio, and then edit some code and re-build solution by pressing F7, the DLL get hot-reloaded. When the sample program exit, the ab_.dll and ab_.pdb files get deleted.
However, when the program quit with a debugger attached, the program can't delete the ab_.pdb file...
Second try
We know that the Visual Studio debugger is locking the PDB file, what if when we detect a debugger is attached, can we detach the debugger programmatically before the program exit? Luckily the EnvDTE COM library can help with this task and someone has written sample code to do this (Although that sample code said we need to modify the "VisualStudio.DTE" string to your installed version like "VisualStudio.DTE.14.0", but I have tested with Visual Studio Community 2017 and it works without modification). So, by detaching the debugger programmatically, we can delete the temporary PDB file when program exit.
Third try
Now we can detach debugger programmatically, Why not try re-attach the debugger after every hot re-load? With the re-attach debugger code written, I tried running the program by pressing F5(Start Debugging) and then pressing F7 to re-compile the solution. A dialog pop up:
And I happily press 'Yes' and hope the hot-reload works, do you know what happened? The debugger stopped, but the application also quit... Looks like this approach can only work when using Ctrl-F5(Start without debugger)... I searched for the web for how to disable killing the app when debugger stop, but I can only find people suggest to detach the debugger instead. So I work around this problem by detach the debugger and re-attach it during the program start to avoid the debugger to kill the app when it stop.
So, the hot-reload function is almost working now, just press F5 to start and F7+Enter to re-compile. But sometimes the debugger fail to re-attach to the reloaded app. After spending sometime to investigate the issue, it is due to EnvDTE::Process::Item() function may fail to find the reloaded app process, returning error code RPC_E_CALL_REJECTED. I don't know why this happens, may be the process is busy at reloading the new DLL, so the final work around is to wait a bit and let the process finish their work and re-try it several times.
Fourth try
We know that detaching the debugger will unlock the PDB, what if we just detach the debugger to unlock the PDB, and only copy the newly complied DLL without patching a new PDB path? Unfortunately, it fails and saying that .vcxproj file is locked...
So I can only revert back to use the "Third try" approach...
Last try
We finally have a workable approach to reload the DLL, how about the executable itself? So I tried the "edit and continue" function in Visual Studio. And it works! But only for once... It is because after edit and continue, stopping the debugger will make Visual Studio kill the app... When manually detach the debugger from Visual Studio, it fails with:
So, "edit and continue" function does not compatible with my hot-reload method which relies on detaching the debugger...
Conclusion
In this post, I have described the methods I tried when writing hot-reloadable DLL code on windows. The steps are as follow:
When the program loads a DLL:
1. Copy its associated PDB file.After editing some code:
2. Copy the target DLL file and modify the hard coded PDB path to newly copied PDB path done in step 1.
3. Load the copied DLL in step 2 instead.
4. Detach the debugger to compile the DLL from Visual Studio.
5. Unload the copied DLL.
6. Repeat the above step 1 to 3.
7. Re-attach the debugger.
1. In Visual Studio, press F5 to compile and run the program with debugger.You can try the above work flow by downloading the sample code. I have only tested it with Visual Studio Community 2017 and may not work with other version of Visual Studio. This method is far from perfect, and if anyone knows a better method and don't require work around, please let me know. Thank you very much!
2. Edit some code, then press F7 to re-build the solution.
3. Press enter to confirm the "Do you want to stop debugging?" dialog.
4. The program will reload the new DLL and re-attach the debugger automatically after compilation.
Reference
[1] https://github.com/RuntimeCompiledCPlusPlus/RuntimeCompiledCPlusPlus/wiki/Alternatives
[2] https://ourmachinery.com/post/dll-hot-reloading-in-theory-and-practice/
[3] https://ourmachinery.com/post/little-machines-working-together-part-2/
[4] https://blog.molecular-matters.com/2017/05/09/deleting-pdb-files-locked-by-visual-studio/
[5] https://github.com/fungos/cr
[6] http://www.debuginfo.com/articles/debuginfomatch.html
[7] https://msdn.microsoft.com/en-us/library/ms809762.aspx
[8] https://handmade.network/forums/wip/t/1479-sample_code_to_programmatically_attach_visual_studio_to_a_process
