About the Author

Daniel Fischer leads the MySQL Build Team at Sun Microsystems. He lives in Karlsruhe, Germany, and joined MySQL AB in 2006. Daniel has 10+ years of UNIX development experience and likes to pretend that he can write documentation as well.

We are always looking for interesting articles about MySQL! Have you written something and would like to it published here? Please contact us via feedback form »

Getting started with Bazaar for MySQL code

Since you're reading this, you probably know that Sun is switching to the Bazaar version control system for all development work on the MySQL server. Unlike the version control system that we've been using previously, Bazaar is an open source project and freely available to anyone. This means that it is now much easier to follow ongoing development, or even to participate in it! On the downside, just like our previous tool, Bazaar is not quite as straight-forward as traditional version control systems such as CVS or subversion. The aim of this article is to give an overview and a general idea of how to set up Bazaar, how to access the MySQL server source code repositories, and the basic commands for working with the source code.

Setup

Bazaar is written in python and requires the python 2.4 interpreter, or any newer version. If you're on Solaris, Linux, Mac OS X or a similar UNIX or UNIX-like operating system, chances are that you already have python. Go to the Bazaar download page and download the latest stable source package. You can run Bazaar directly from the extracted archive, or you can install it system-wide. For details, and a list of "soft" dependencies that can improve your Bazaar experience, see the installation FAQ list. If you're on Windows, get the Windows installer from the download page and see the Windows installation page for details. Note that you will have to install several libraries that Bazaar depends on.

After this, you have a basic setup of Bazaar, including a command-line interface. (By the way: If you somehow translated "go to the download page and download the latest stable release" to an apt-get invocation or similar, please make sure that you have at least Bazaar version 1.5.) There are many plugins for Bazaar that are outside the scope of this article. One that might be of interest is bzr-gtk, a plugin that implements a graphical user interface for several common tasks. You can find it on the Bazaar site.

Accessing the MySQL Server repositories

The MySQL Server on Launchpad. The grey box that says "current development focus" has the link to the 5.1 branch.

The MySQL source repositories are mirrored on Launchpad. Visit the MySQL Server project page on launchpad to see available branches. About in the middle of that page, you'll see a "Timeline" section with several release series listed. Click on the one that says 5.1 to get to an overview of branches in the MySQL 5.1 series. Then, click on the mysql 5.1 link in the "Code for this series" section to get to the actual MySQL 5.1 branch. You'll see a bunch of lines of general information about the branch, and the most recently committed changesets. More important, this page also tells you how to get the source code. At the top of the page, there is a line that says "Get this branch", followed by a command, in this case bzr branch lp:mysql-server. This command is all you have to enter to create a local branch of MySQL 5.1 on your computer.

Sounds easy, right? There has to be a downside, right? Right! Creating a branch means that bzr has to download the complete history of MySQL up to the most recent revision in the branch you're branching off. In the case of MySQL, this means downloading close to 60.000 changesets. On top of that, I'm sorry to have to admit that Bazaar is not exactly the fastest version control system there is. Creating the branch could take more than half an hour plus several cups of coffee!

The bad news is that you're not going to avoid this. The good news, however, is that you only have to wait this long exactly once. Now is a good time to decide what you intend to do with the MySQL source code: If you're planning to create multiple local branches, for example your own branch with your own changes, or just several of the different MySQL release series, there is something that you should know about: Shared repositories.

When you create a checkout with CVS, you get a bunch of files - but only that. The history remains in the CVS repository. When you commit a change, you commit it to the repository. Bazaar, as I mentioned previously, is not as straight-forward. When you create a branch, your copy contains the complete history. (There are ways to avoid this, but unfortunately, Bazaar is still slow, and it doesn't matter too much in the end.) If you create two local branches, both contain the entire history. If you create a third branch, you get the entire history again. And so on.

This is a bit redundant, and also a bit redundant. To avoid the redundancy, you can tell Bazaar to store the revision history of several branches in one common repository. This is called a shared repository. Bazaar will only put any part of the history in the shared repository exactly once. Consequently, it will also only download any part of the history exactly once. Since the various MySQL server branches share a good deal of their history, this will save you most of the download time when creating your second branch, and also a lot of disk space. Here's how you do it:

First, before creating any branches, create an empty directory in which you will keep all your branches. Then, create a shared repository by running bzr init-repo. in it. Now you're set! Create your first branch in this directory, as outlined above. It will still take about half an hour to one houre to complete. Now, create another branch, for example, MySQL 5.0, in the same shared repository: bzr branch lp:mysql-server/5.0. It should be much faster this time! Here are all of the commands you're going to type again, for your convenience:


$ mkdir bzr
$ cd bzr
$ bzr init-repo .
$ bzr branch lp:mysql-server
$ bzr branch lp:mysql-server/5.0

Now that you have one or two MySQL branches, and know how to get more, let's explore what you can do with them.

Building the MySQL Server from Bazaar

Assuming you're not reading through all of this without an idea of what to do with the MySQL source code, one of the first things you might want to do is building the MySQL server, so you can run it.

It's pleasantly easy to build the server right in the branch that you created. The MySQL manual has instructions on building from a source distribution. The steps for building from a Bazaar branch are almost the same. However, depending on your operating system, you need to prepare the source before you can follow the instructions.

Optional: Export the Source

There is one thing that you might want to do if you want to keep your branch clean: Export the branch and build the exported code. There is no strict requirement for this, but it makes some tasks easier. This is fairly straight-forward. From within the branch you created earlier, run the following command:


$ bzr export ~/mysql-5.1-export

This will copy the source code to the directory you specify, without the versioning information. Change to that directory before continuing with these instructions.

Building on Linux and UNIXoid operating systems

On Linux or on other UNIXoid operating systems, you'll need to have the GNU autotools installed. The MySQL source includes a script that runs these tools. We'll start by invoking it:


$ BUILD/autorun.sh

In all likelihood, you'll see tons of warnings about GNU extensions in makefiles. Ignore those. We're going to use GNU make anyway.

If everything went well, you'll now find a configure script in the top directory of the branch. The next step is running that script. Invoking it with no arguments will configure a sensible default that will yield you a usable MySQL server that still doesn't include all features. For the purpose of this article, this will do. If you need specific functionality compiled in, please consult the MySQL reference manual at the above mentioned address; it has a list of commonly used configure options.


$ ./configure

The configure script should eventually finish and print out the following message:


MySQL has a Web site at http://www.mysql.com/ which carries details on the
latest release, upcoming features, and other information to make your
work or play with MySQL more productive. There you can also find
information about mailing lists for MySQL discussion.


Remember to check the platform specific part of the reference manual for
hints about installing MySQL on your platform. Also have a look at the
files in the Docs directory.


Thank you for choosing MySQL!

The next step is doing the actual build, which is not any more difficult than the previous two steps. We're just going to invoke the make utility that should be installed on your system. If you're on Linux or any other GNU system, where GNU make is the default, you can just type make:


$ make

If you're on a non-GNU system where the default make implementation is not GNU make, you'll need to substitute gmake for make:


$ gmake

Now, this could potentially take a long time, from few minutes to an hour, depending on how fast your computer is. On my laptop, it takes about five minutes. You'll see a whole bunch of messages scroll by that inform you of what make is doing. Eventually, it will finish. If it doesn't display an error message, we're good.

The easiest way to run the MySQL server that we just built is through the included test suite. You'll need a perl interpreter for this.


$ cd mysql-test
$ perl mysql-test-run.pl

You can also use the test suite to set up a running instance of the MySQL server that you can then connect to through the command line:


$ cd mysql-test
$ perl mysql-test-run.pl --start-and-exit
$ ../client/mysql -S ./var/tmp/master.sock -h localhost -u root
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.1.26-rc-log Source distribution


Type 'help;' or '\h' for help. Type '\c' to clear the buffer.


mysql>

This is most useful if you actually want to work on the source code. If you only want to build the server so you can use it, there is a script included that will create a binary distribution very similar to those that we publish:


$ scripts/make_binary_distribution

This should yield a file that looks similar to the one I got:


$ ls -l mysql-5.1.26-rc-linux-x86_64.tar.gz
-rw-r--r-- 1 df df 11923974 Jun 12 11:26 mysql-5.1.26-rc-linux-x86_64.tar.gz

You can use it just like you would use a similar package published by us. Congratulations, you're done!

Building on Windows

Building on Windows is slightly different. I'm going to show the basic steps. There is a README file in the win/ subdirectory of the source code that you should read even if you don't need more detailed information (it is good practice to always read README files).

First, since typical Windows distributions don't include a compiler, you'll have to install one. If you don't want to buy Visual Studio, you can use the express edition that you can download at http://msdn.microsoft.com/vstudio/express/. Make sure to pick Visual C++ Express.

Then, there are some more tools that you need to have installed. The first one is called CMake and you can download it at http://www.cmake.org. The second one is called Bison, and can be downloaded from http://gnuwin32.sourceforge.net/packages/bison.htm. Make sure that both are in your PATH, as you will need to run them.

After you've installed these prerequisites, please change to the directory that you want to build (either the branch you created earlier, or an export). Then, configure the build:


$ win\configure

Please refer to the win\README file for options that you can specify. Without options, you'll get a basic, runnable MySQL Server binary, but you might want to enable one of the storage engines that are not included by default, for example, InnoDB or Archive.

Now you need to run CMake to create project files that you can use with Visual Studio. For example, if you have Visual Studio 2005 or Visual C++ Express Edition, you'd run:


$ cmake -G "Visual Studio 8 2005"

Run cmake without arguments to get a list of generators that it supports. If you have installed cmake via cygwin, it will be rather short. In this case, you should install Windows cmake again as outlined above, and make sure that the Windows cmake comes first in the path.

That's it - the source is configured. If you have the commerical Visual Studio, you can build the solution that cmake created for you from the command line:


$ devenv.com mysql.sln /build "RelWithDebInfo"

In any other case, or if you just don't like the command line, you can of course just open the mysql.sln file with your IDE and build from there.

More on Bazaar

After all of the above, you know how to access the MySQL Server code. Still, if you're used to using SVN or CVS for revision control, Bazaar is quite different. Explaining Bazaar in detail is outside the scope of this article, but I'm still going to show you some of the differences to get you started.

First of all, Bazaar is a distributed revision control system, while SVN and CVS are centralized. With Bazaar, you don't have one central repository that everybody works on and commits in. Instead, every developer can work in a branch of his own, and all branches are equal to Bazaar. Internally, at the Sun Database Group, every developer usually has at least one branch of his own that he works in. Teams of developers have team branches where they aggregate the work of the team's members. And finally, the release team for a complete MySQL Server release aggregates all of the team's work into the "main tree", which is what we call the reference branch that we build releases from.

With SVN and CVS, development happens in increments, from one revision to the next revision to another. The entire history of a repository is linear. Every single revision is based on one specific parent revision.

With Bazaar, all branches are independent of each other. Sure, they share most of the code stored in them. But if one developer commits a change, that doesn't affect any other branch. In order to propagate changes, one pushes one branches' changes into another, or merges changes from another branch into it. Whenever two branches are merged that have concurrent work in them, the revision history becomes non-linear. One revision can be derived from multiple parents. The revision history becomes a revision graph.

bzr visualize

If you want to get an idea of what the revision graph of a large repository such as the MySQL Server's looks like, you can install the bzr-gtk plugin for bzr from http://bazaar-vcs.org/bzr-gtk. It contains a command "visualize" that visualizes a branches' history. From within your branch, run this command to get a first impression:


$ bzr visualize --limit 100

The --limit clause limits the number of displayed revisions to one hundred. Displaying the full graph takes quite a bit longer, but if you're interested, run the command without the "--limit 100" arguments and go make some coffee while waiting for it.

Basic Bazaar usage

You've already seen how to use Bazaar to create a local branch from an existing one. I don't intend to go into the depths of Bazaar usage in development, but I'd like to present you with a small cheat sheet that contains the commands for several common actions with several different revision control systems.


SVN               CVS            BK                   Bazaar
svn help          cvs --help     bk help              bzr help
svn add           cvs add        bk add               bzr add
svn annotate      cvs annotate   bk annotate          bzr annotate
svn co            cvs co         bk clone             bzr branch
svn commit        cvs commit     bk ci + bk commit    bzr commit
svnadmin create   cvs init       bk setup             bzr init
svn diff          cvs diff       bk diffs             bzr diff
svn export        cvs export     bk export            bzr export
svn log           cvs log        bk changes           bzr log
svn update        cvs update     bk pull              bzr pull, bzr merge