One of the perks of working on MySQL is that the code can compile (and has been compiled) on pretty much all of the modern (and no so modern) C/C++ compilers and all the modern OSes. This enables each individual developer to choose their “weapon”: pick the OS and the compiler they feel best about. We of course have standards when it comes to producing the binaries for our customers. But individual server developers can be very diverse in their choice of platforms. And we are. It’s even encouraged since this adds additional test coverage to the codebase and engages the developers as advocates of certain OS/compiler combinations. This way developers feel “ownership” of these combinations and strive to provide fixes for the problems they see.

I of course am no exception to all this. I too did my share of odd tool choices (ahem, gdb/vi/ssh, ahem). I’ve recently found myself using Microsoft Visual Studio on my laptop as my “daily driver”. I have history with VS. But I also really like how the product is developing as of late. The pace of innovation is great and the team behind it extremely responsive.  Thus individual users like me are feeling increasingly “in control” and that drives loyalty up.

I’ve tried the Visual Studio.Code too, but, being used to Visual Studio, I found it lacking in many ways for my needs.

HOWEVER !

As the team keeps working on MySQL, the codebase grows larger and larger with each release. Gone are the days of 5.0 when the whole thing could be compiled in under 5 mins. Now a full rebuild on my i5 based SSD laptop takes about an hour. And, having to do this 10s of times a day on different branches, I’m on a constant lookout for clever methods to trim that time down.

I could of course get a more powerful desktop. But then I’d run into the whole issue of having to maintain and work on two different machines. So not an option for me.

Many things have been tried to decrease that time on the local PC. But, thanks to the newest developments in Visual Studio, there’s a brand new way to do this that I want to guide you through today.

It is using visual studio on windows as a front-end. And outsource all of the “heavy lifting” (compilation, linking, running) to a remote machine. A Linux machine. I have access to a really powerful remote Linux server that I can use to compile and test MySQL on. But I really would like to avoid going back to my gdb+vi-over-ssh combo. Thus I was pretty excited when I’ve seen a feature added to Visual Studio to compile and debug remotely on another computer. In this way I can still stay with my local rich UI (and not have to ferry it over an rdesktop type of connection over the Net) and still make good use of that powerful Linux server.

Let me guide through the steps I’ve taken to make this work:

• Install Visual Studio 2019 Preview. And fully update it. I’m using VS 2019 Pro 16.3 Preview 2. It might work with the release version too. But this code is under very active development. So I’ve found the Preview more useful atm. To quote William Gibson: The future is already here. It’s just not evenly distributed yet ! 🙂
• Make sure you have the following components installed in the Visual Studio Installer (not in the minimal default install, so please make sure you select them)

• Add your remote host’s login details in VS: Tools/Options/Cross Platform/Connection Manager.

• Check out the MySQL git tree. I’m really serious about my gitting so I use the command line:

• Fire up Visual Studio UI and open the mysql git tree folder (git-mysql in my case) as a project folder.

• Wait for Visual Studio to grok the fact that this is not just any old folder but a cmake git folder. Monitor the “background activity” (indicator on the bottom left of the status bar) if you’re impatient. Eventually all activity dies down, a “Build” top level menu appears and VS tries to run a local cmake on the local project (since this is the default). Note that, since MySQL requires some 3d party libraries, this will fail the first time VS attempts it if you don’t have these installed in the location cmake expects them in. That’s OK (we don’t want to compile locally anyway) and expected. Here’s what the failure looks like for me:

• Now it is time to add the remote configuration option. Click on the “configuration” drop down in your toolbox (one that currently says something like “x64-debug”) and select “Manage Configurations”.  You get an editor opened on CMakeSettings.json.
• Hit the green plus button and add a second configuration based on “linux-Debug”.
• The edit window is really nice but I like editing JSON configs better so I click on the “Edit JSON”  “link” and do my edits, namely:
  • Adjust the output paths on the remote
  • Adjust the cmake settings on both the local and the remote to have the locations of my boost library (-DWITH_BOOST) and the openSSL library headers to use (-DWITH_SSL).
  • Since my remote Linux server has many cores I’m also adding a -j20 to my build command options to have GNU make run on 20 threads.

• If you’ve got everything in the above file right when you save it VS will add a second choice in the “Configuration” combo: “Linux-debug” (after having run another cmake locally of course). Make sure Linux-debug is selected.
• VS will rsync all of the mysql sources to the remote. That might take a while, depending on the connection you have. Note that if the cmake binary on the remote is not the most recent one VS might ask you if you want to add one for you. Say “yes” !

• When done rsyncing it will run a cmake on the remote box. And then try to extract the target info. This can take a while too. So be patient. When it’s done your output window will look like this:

• Now the VS IDE has updated itself with all the “target” data: what binaries and libraries are there for your project to build. If you go to “Solution Explorer” you can now switch views from the file based view it’s displaying by default to “CMake Targets View”. And you’d get all of the MySQL build targets in a nice list. Just like you do for the local builds. Unfortunately, unlike local builds, these do not expand to show you the source files involved etc. But it’s pretty cool even as it is:

• Now you can hit “Ctrl-Shift-B” (Build/Build All), lean back, and enjoy the fruits of your labor. You get the fast remote Linux host to compile MySQL as fast as it can while you’re watching it into your laptop’s VS IDE.
• That’s it ! You’ve compiled MySQL on the remote Linux box via Visual Studio.

The next thing I’ve tried (since according to a person much wiser than myself software development is the cycle of making mistakes and then fixing them) is to open one random source file and make a syntax error. I’ve opened components/test/test_udf_registration.cc and renamed the constructor method of the udf_list class to “udflist12”.

The amazing thing was that IntelliSense immediately picked it up (as I was typing it) and highlighted the error. Just like it would for a local project.

But things got a bit uglier when I’ve hit “Build All”: I’ve got the whooping 15 errors in my error list:

That’s a lot. Admittedly most of these are Intellisense errors that you can filter out. And when you do you get the single compile error (the first one in the list).

But yay ! Remote compilation errors displayed in nice local rich UI list !

I was now ready to debug stuff 🙂

Of course, being the server developer that I am I wanted to debug the server.

But to run the server I need a data directory !

Fortunately, since MySQL 5.7 this is easy: I just run the server with –initialize and it will create the data directory for me. I admit I also run it with –initialize-insecure when debugging so I can avoid having to read the random password generated by root from the server log. This is safe for me since I don’t usually have my servers listening on the Internet and there’s nothing valuable in the data directories I create. But I still feel bad doing it.

So the process of debugging a server from within Visual Studio for me consists of:

• Set “mysqld(runtime_output_directory\mysqld)” as a “Startup Project”: go to Solution Explorer, switch it to cmake targets view, right click on mysqld and select “Set as Startup Project”. Make sure to avoid the “install” variant !
• Set command line settings to mysqld so that it creates a data dir. From the same right click menu choose “Debug and Launch Settings” and then the non-install variant. This will open another JSON configuration file: launch.vs.json. Then add the “args” array to it containing all of the mysqld’s arguments, including the –initialize option. Note the relative path for the data dir (it puts it into the build dir this way without me having to type the full path), the ports/sockets (I’m not running as root on the remote box so I need to alter the defaults) and of course the –console –gdb combo to make the server not daemonize iteself and reply to Ctrl-C signals. Here’s how mine is looking like:

• Using the above config I run the server once so it can create its data dir.
• I then go back to the file and remove the –initialize option. From then on I can start the server and it will find its data dir. So I only re-do the initialize step above if I need a clean data dir. And make sure to delete the old dir before I do too.

Note that, at least on my VS version, every time I hit “Debug and Launch Settings” as described above I get a new configuration !  So if you need to edit launch.vs.json either do that by explicitly opening it as a file or make sure to delete the new section added !

Now I’m ready to put a breakpoint and see if the server would hit it starting up. A fun place to do that is on init_server_components() in sql/mysqld.cc. This is called relatively early at every startup. I add the red dot and I hit F5.

At this point VS needs to process the symbols in the remote executable. So I get some of this:

And I need to wait a bit. But it’s all worth it as I’m then greeted with the following screen:

That is indeed a thing of beauty ! Just consider what it actually is:

• a working graphical debug and step UI
• debugging a process running on a remote box
• showing me all the symbols and values in a nice UI way
• showing me threads and calls stack in a nice UI way
• embedding the Linux console output of my process into a separate window
• allowing me to be putting breakpoints and stepping through my code

Does it have pitfalls ? Of course. It does drive like a product under active development. There are delays here and there and there are rough edges too. But, being a developer myself, I can tell that most of the important work is done. Now what remains is polishing it and make it it more robust. And this is also a normal software development cycle. So please make sure to stay up to date !

I know a whole lot of you are using UNIX based systems for development. But Windows offers some really good tools too ! I even dare say better 🙂 So please give it a try. And make up your own mind !

And of course don’t forget to contribute that code you develop back to the MySQL community 🙂

BTW, If you are looking for a good development documentation, we’re putting a lot of effort in the doxygen docs for MySQL: https://dev.mysql.com/doc/dev/mysql-server/latest/. This should give you ideas on where to put these breakpoints and what to expect doing it 🙂