Lenz Grimmer is a member of the community relations team at MySQL AB. He lives in Hamburg, Germany and has worked for MySQL since April, 2002. Before joining the community team in December 2005, he was a member of the release engineering team that is in charge of creating the official release builds of the MySQL server.
David Axmark co-founded MySQL AB together with Michael "Monty" Widenius and Allan Larsson in 1995. Today, he is a member of the MySQL Community Relations team and has an advisory role in the MySQL Management. He travels all across the globe to advocate for MySQL and Open Source Software in general and has just recently moved from Uppsala, Sweden to Ascot in the United Kingdom, where he lives with his wife and two children.
Hi David! Thanks for volunteering to answer my questions. You have probably answered these a million times already, but can you tell our readers how MySQL got started and what motivated Monty and you to write your own database?
MySQL got started in Sweden/Finland over 10 years ago. I had been working with the other founders doing work for lots of different customers. Our biggest customers analyzed data of larger companies and we helped them with that by using a text-based database tool that Monty had written – "Unireg". Even before that (since the early eighties) we had used Free Software like (X)Emacs, gcc, bash, gdb, perl, and lots of other applications. I even wrote a 2700 line shell script to automatically install packages doing everything from extracting of source tarballs, applying home grown patches, compiling and installing in our directory tree. We always installed in our private directory so that we only touched one place on a customer machine. And that private directory was called /my well before MySQL. We wanted a short name and at that time the Unix root directory did not have any other entries starting with "m" so it allowed quick and easy tab completion in the Unix shell.
We thought about releasing some of our own software as Free Software during this time. But our software was not general purpose enough and getting people to actually use it and documenting it (some of it was in Swedish/Finnish) would have been very hard work.
While doing the analytics work mentioned above we had a huge problem delivering data to the customers. So when the Internet became popular we immediately bought a fixed Internet line (providing a whopping 64kbit/s bandwidth) and asked our customers to also get an Internet connection. And then we started to make static HTML pages generated from our non-SQL database. Dynamic pages done with horrible unireg macros in the HTML code followed. We wanted to solve that and being able to use embedded SQL statements would have been really nice. But we could not find any Free-Software SQL database that worked for us (we looked at everything available at that time).
Another issue we had was how hard it was to really finish something and make it beautiful. We had worked for many years on customer projects mixed with our own projects. But our customers never wanted to pay for a nice and elegant solution – something that did the job however stupid, fragile and inelegant was usually OK for them. So we always ended up doing more on our own time than the customer had paid us for, which was not a very profitable model – but we wanted to be proud of our code.
We also had problems with big customers asking us to use "standard" tools instead of that strange homegrown stuff. We tried and it took more time to get decent performance out of the proprietary database we used than it took us to develop and maintain our own code. Which was much more fun, too! But one problem here was the name – "Unireg" did not ring any bells for these customers. We were often thinking it would be much better to be able tell them "we are using SQL".
So what visions did the MySQL founders have in the 1990s?
We did not really have any grand visions – spending a lot of time on visions would have distracted us from getting real work done! I spent a lot of time on mailing lists and Usenet groups like gnu.misc.discuss and free-software-as-business. There I learned about the dual licensing that was used (and is still used) for Aladdin Ghostscript. I thought the same system would work even better for a database. With dual licensing we could have a successful software company and still give away high quality code for free to lots of people all over the world. This way, developing countries, individuals or small companies would not be forced to illegally copy our application, but would be able to use real high quality stuff legally. And without any strange legal or technical limitations.
All of the reasons above made us start working on MySQL back in 1995. Monty did the initial coding of an SQL layer on top of the already existing ISAM data storage parts (the first parts of what would eventually become MySQL were actually written around 1979). In May 1996, MySQL version 1.0 was released internally to a limited tester group of 4 people, and in October 1996, MySQL 3.11.1 was released to the public as a binary distribution for Sun Solaris. About a month later, after I had spent some time to learn the GNU autotools and implement them, a source distribution was released and we also added binaries for Linux.
All the porting and installing of free/libre open source software (FLOSS) over the years had given us a lot of experience with good software that was too hard to compile, install and get started with. And in order for us to succeed commercially, we needed lots of users. So we worked hard to make the installation and first use as easy as possible. We came up with the 15 minutes rule: we wanted a user to be able to have MySQL up and running 15 minutes after he finished downloading it. That may sound like a lot of time in today's world of automatic installation from RPM or DEB packages. But back in 1996, you normally installed from source code that you usually had to modify slightly. Often it was only a few characters you had to change, but for someone who didn't know the code that could have been too difficult already. You might ended up spending hours installing a package. And we wanted people to spend those hours getting an application working with MySQL instead (since someone who fails to get things working early is very likely to try something else and never look back). To make sure that things were going smoothly for as many installers as possible, we borrowed access to remote machines from other users (an SSH login and some disk space) to test compiling every release of MySQL on 8-10 different operating systems/machines. We transferred the source code, compiled it and ran a test suite on each of those boxes before every release (even when we were only 2 persons doing this on part time). That was the hard work behind the 15 minute rule. Nowadays 15 seconds is not an unusual time to install MySQL, if you are using tools like apt-get or yum.
In 1997 we got our first customers who used our extremely simple online shop. We had a single page with a choice between a license or support and some text fields to fill in the credit card details that were processed manually later.
"Choice of license" sounds like you offered MySQL under different licenses from the very beginning. Can you elaborate on that?
We did dual licensing from the start (one free license and one commercial license) – but our free license was not FLOSS compatible. We followed 8 of the 10 criteria for an Open Source license as defined by the Open Source Initiative (OSI), which implies that we also failed one of the 4 freedoms for being Free Software. The licensing was set up to allow us to give MySQL away for free to the majority of our users, and get money from people who wanted to distribute MySQL as part of a closed source product.
With dual licensing, users of both licenses win, since the free users get a high quality free product where the development is paid for by the customers. And they use that product in many ways from very basic use to the extreme cutting edge. And while doing so they generate bug reports and ideas for new features that the customers then will benefit from. In addition to that, the free users also contributed tools, connectors, documentation, good answers on the mailing lists and news groups as well as many other good things.
This dual licensing gave us a steady income, allowing us to hire and pay more programmers (we actually had no other staff the first years, not something to recommend). Merely providing services would have been a bad idea for growth, since service delivery would have required more of our own management time, so we would have had to slow down on product development.
This model worked well for us and we grew fast. But any mention of MySQL on a discussion board or mailing list would always lead to a licensing discussion. We would see that regardless of how free we were – to continue to grow we needed to change to a full Free Software/Opensource.org (FSF/OSI) compatible license. And for us the GPL was the obvious/only choice. Both because it would allow us to continue the dual licensing model and it was my FLOSS license of choice.
So in 2000 we changed the free license from our own homegrown variant (which was based on the Ghostscript public license) to the GPL. Business-wise that had a drastic effect – our sales fell 80% over night. In a normal company that would have caused extreme panic. But we had saved some money, kept expenses low and Monty and I had a rock solid belief in that we needed to stay the most used Open Source database in order to succeed in the long run. Some of our early employees (and one founder) thought that it was a too drastic decision and we should have waited with it. I don't agree – I think we timed it well. And after a year or so we were back at the same level of sales!
Yes, looking at where MySQL stands today, it really seems to have paid off! How did this "Community QA" process work in practice?
We did binary releases every 4-6 weeks of our development (called Alpha) version to get feedback on new code. And some of our brave users used our Alpha releases in production. Those users gave us very valuable feedback, based on real life usage scenarios. And in a few cases they proposed very specific changes. We then sent them what is called a patch, that just contained the changed lines in the source code. These users then applied the patch, recompiled, installed and tested the new binary in production and gave us feedback on how the fix worked for their use case. A normal company would have hired some internal QA guy who tried to test the fix according to some spec, and that is not even close to as good. These people used MySQL to solve real life problems instead of doing lab tests (apart from that you have to pay for the internal QA guy).
So you utilized your community to provide you with free QA, and in return they were able to freely use your product. What other areas did the community help you with these days?
We always tried to make MySQL easy to use for as many people as possible. An effect of that is that we tried to support as many programming languages and environments as possible. So we made it as easy as we could to add new drivers for MySQL to other languages. And we tried not to impose any judgement on languages that got new drivers. An example of that was an odd little language called PHP/FI that was not very impressive in 1996 for guys used to C, Perl, Common Lisp, etc. But the guy who worked on it (Rasmus Lerdorf) made a good impression on us so we spent a little time on it. And then a little more. And a few years later PHP was a much more impressive language and the default database for it was MySQL. In fact for most of these languages we actually added very few drivers by ourselves – it was the community that added PHP, Perl, Python, Ruby, Java, Tcl, Snobol etc. We usually learned about all these some time later, when people were referring to them in some email discussion.
Another nice example of users who helped us was one guy who started sending patches to the manual. At first he corrected simple mistakes in Monty's and my very special English. Then he started rewriting the same English to be easier to read. After the first 5-6 patches, I automated the application of them, and that was good since we got something like 150 patches! In one case this guy tried to document the exact working of the date functions in MySQL. He sent question after question to Monty who answered all of them diligently. And then he got another question back, since there was still an unexplained corner case. So after a few rounds of discussing, Monty rewrote the code to be easier to explain in all cases. By the way, this outstanding documenter (Paul DuBois) now works for us not as a volunteer but as a paid Docs Team member since a number of years!
Windows was another issue, we had talked at least every week for the first years about doing a Windows version. We had a lot of people asking about one. And while we were still discussing it, we received an email from Gianmassimo Vigazzola from Italy. He had done a port of MySQL to Windows and sent us a patch (all hail the power of FLOSS)! Even though the code was working, he did not know the MySQL internals that well, so it was impossible for him to do all the changes in the best possible way. However, Monty used his patch as a reference and did the Windows port an order of magnitude faster than he would have done without Gianmassimo's initial work.
Now, every time Monty needed to do a Windows build, he had to reboot his machine and use an operating system and compiler we had to pay for. And running Windows was a bit of a pain. Hence, we decided to charge money for Windows binaries that people used for production. Not very formally, we basically said "You have 30 days to try it, if you use it in production after that time you should pay for a commercial license". We had no artificial limitations, nagging popups or something similar in the binary. We got a lot of people who complained about this, but it worked and we got lots of people buying licenses on our website for the Windows version. So that paid for hiring more programmers to grow faster. But after we had switched to the GPL, this special exception was also removed from the license terms, of course.
What other aspects (apart from the license model) made MySQL unique in your opinion?
Another special thing about MySQL was (and still is) hiring new employees. We started out as a multinational company (Sweden/Finland) with everyone working from home. So it was natural that our first hires were made over email and it did not matter at all where they were located physically. We were pretty sure about those candidates anyway since we usually hired people who had been active on the mailing lists. In one case we even hired an ex-customer who submitted really good bug reports (that is not so easy).
We had another philosophy: we did give free help, but only publicly. So if you sent us a private email, or said that you could not provide the details publicly, we answered the email, but told you to ask on the mailing lists or buy support if you needed a private answer. And on the mailing list we answered questions we thought would help many, not just the guy who asked. And if we saw repeating similar questions we updated the manual (or code) and sent a pointer to the updated manual. This process drastically reduced the number of times we had to answer the same questions. And other people could start answering based on what they learned from the manual and our answers.
We also updated the manual as soon as new code was written. We are programmers, so writing documentation was not that fun, and if you push it towards the future it tends to never happen. The new manual was published with a "make web" command so the online manual documented the next unreleased version most of the time. So at least no one could say it was out of date!
We convinced ourselves to think that if we added a cool feature to the code, unless we explained it in the manual so that people know how to use it, we might as well have left it out of the code. That thinking provided some more energy towards the (boring) documentation writing.
Looking back, what would you have done differently? What are the lessons learned? How do the ideas from 1995 stack up today?
Believe it or not, I wouldn't have done much differently. Our values and priorities put the emphasis on improvements crucial for user base growth and community building. By coincidence or perhaps sheer luck when it comes to timing, MySQL grew more than we had ever imagined. However, we should have probably spent a bit more time on getting at least minimal legal assistance early on. Some basic legal groundwork in the beginning would have helped us to save lots of time and money later on several occasions.
The priorities were not the case for all technically promising companies who had got the timing right. Around the same time we started we knew people at another Swedish company that worked on a great web server. And it was licensed under the GPL! Since this was before the Apache web server dominated the field, you would have thought that they had a glorious future. Well, that future did not work out, since they failed in some basic principles. For example, they did not release a new version of their product for over a year, in spite of their last public release having lots of annoying bugs. The reason for that was that their management had decided to focus on paying customers and thought that those GPL users could take care of themselves somehow.
The end result of that thinking was that the community around the product shrank to a few very happy users/customers. The at that time technically inferior Apache web server got all the users and developers. So nowadays hardly anybody knows about the Swedish web server "Spinner/Roxen", but everyone has heard about the Apache web server.
I have also heard many people stating things like: "if we had the support you had, I would have also started an open source company". Well, we had no support when starting MySQL. And in some ways it might have been easier to start in a developing country instead of Scandinavia:
Aspects that would probably have made it easier:
But starting in a developing country would also made it harder:
As for the large Swedish customers who did not like our software because it was to odd or unknown we got success there, too. One large customer choose MySQL as an alternative to their expensive proprietary database as a company wide policy. Without knowing, they had used some of the same code years before with another name. And that they had indirectly paid for MySQL development by giving us consulting work.
Does the MySQL AB of today still resemble the company you envisioned and built up back then?
The business model is basically the same, but we have extended it in many directions. The philosophy of giving free public advice is still with MySQL today and we have many people who answer publicly in our forums and mailing lists when they can. And if you want a guaranteed and timely answer, you can go ahead and buy service from MySQL just like when we started. The biggest advance so far was the introduction of our subscription offering "MySQL Network" in 2005, which has now been expanded and rebranded to "MySQL Enterprise" and now includes a wide range of additional paid-for services.
As for our vision of having MySQL used by people who could not afford to buy a normal database, that worked out well, too. People are using MySQL all over the world, as I have found out while traveling to many countries, including very poor ones.
MySQL is still easy to install (even if we can still make it better, especially on Windows). The 15 minute to get going goal has stayed around and is quoted regularly in the company. But now people have the ambitious goals to make it possible to set up a High Availability MySQL (multiple machines) in 15 minutes!
The spirit of having good documentation is still living in the company. We have a superb group of documentation writers now and our manual has grown enormously. And we also have docs for all our supporting tools and connectors.
Probably the largest change since the early days is the number of people working for MySQL. Once upon a time, you could make a cross department decision by having a discussion with 5-10 people. With so many more employees, it now requires a lot more people to involve and discuss a decision with. As someone who has never worked for even a small company before (I only worked for very small companies), it can sometimes be frustrating with all the red tape (even if you yourself understand why all those rules are needed in a larger company).
Also, the role of an individual is much more limited in a larger company. When you are small, it's possible to do multiple jobs. Really early it's not just possible, it's required! When you grow as an organization, you need to have more specialized people. It is very different to work in a 10-people company (and we thought we were big when we were past 10 employees!) compared to a 300-400 people company.
We still hire people all over the world. We strive for having employees in compatible timezones as weekly phone calls at 06:00am or 02:00am are not that fun. And today's recruitment practice involves a large number of phone interviews and at least one in person interview.
It has been a long journey now with lots of positive experiences but also lots of bumps. It's nice to be able to travel to any corner of the world and have people thank you for MySQL!
David, I too would like to thank you for your time and all your dedication to making MySQL the most popular Open Source Database!
This interview was performed in July 2007
Read and post comments on this article in the MySQL Forums. There are currently 0 comments.