Interview with Monty

My Dinner with Monty

May 20, 2006

This interview took place with Monty Widenius, co-founder of MySQL, at his house outside of Helsinki, Finland. In Monty’s usual understated Finnish style, we talked about MySQL 5.0, coding styles and cooking.

Q. The last interview we did here was when MySQL 4.1 was being finalized. Now you’ve shipped MySQL 5.0. What’s the current status?

We released a new version of 5.0 that fixed a few issues, and it’s quite decent now. Our support people are very pragmatic. They don’t recommend things they don’t believe in. They can now recommend 5.0. They say that now it seems to be an ok release and with the improvements and bug fixes, every release gets better. MySQL 5.0 is probably as good as any production release from any company. We have high ambitions though and we are fixing all issues reported.

Q. And how about MySQL 5.1? You showed that at the MySQL User’s Conference recently.

MySQL 5.1 is in beta now but we still have a lot to do. The current state of 5.1 has some new capabilities that are important, like disk-based cluster and partitioning. For 5.1 we are also improving our development processes. We have had some small glitches internally but we are now on track on fixing these. We are not always doing things as efficient as I would like. So we are adding more tools and processes so we can improve our productivity and have less duplication of effort.

Q. What kind of tools?

In the last six months we have built some new tools to improve things. For example we have a pushbuild tool that checks things pushed into our main development tree. It lets you know in advance before doing a full build of the tree if there are any problems. We also have a tool called autopush that will check for things like memory leaks, before we put code into the main tree. No one should be able to make a push that breaks the tree. If the build breaks then it’s a big waste of time.

Q. Let’s jump back to the users conference. Did you enjoy that?

I always love to meet our users and customers, that gives me feedback to see that we are on track. Currently it seems that our users are very happy with what we are doing, which makes me feel pretty good. People like our storage engine strategy.

Q. Were you nervous on stage?

I’m always nervous on stage. It’s a big crowd and I’m using to sitting down in my cellar with very few people. Seeing hundreds of people in front of me makes me nervous.

Q. You had a demonstration of Falcon in your keynote. What’s it been like to work with Jim Starkey?

I’ve known Jim for many years and we have always respected Jim’s accomplishments with InterBase and Firebird. But Falcon is not Firebird. The code in Falcon is completely new and Jim has taken all the new ideas he’s gotten over the years and never had a chance to implement and create something that in his mind is completely fantastic and the only right way to do it. And the future will tell us how right he is.

Q. You’ve had a few people staying over at your house this week.

Yes, we had Sanja Byelkin from Ukraine, Sergei Golubchik from Germany originally from Russia, Mikael Ronström from our cluster team in Sweden, and Guilhem Bichot from France. And also this guy named Zack, I think he’s from Canada. (laughs).

Q. What have you been working on?

We are working on storage engines. We’re making an extension of MyISAM to make it ACID compliant and also add new features that make it even more suitable for data warehousing. In other words, make it an engine that addresses a lot of the major issues people have asked about over the years.

Q. You’ve spent a lot of time working in data warehousing and MySQL is pretty widely used in that area. What are some of the newer capabilities you’re adding?

Computers have much more power and memory than they’ve had before so you need to be able to use that. But disk speed has not gone up that much compared to other things, so you need to do things a little differently. In data warehousing there’s still a lot of things to do with table scans, at least for some part of the table, and very clever indexing and new methods to skip some blocks of data, make it more efficient. We are taking that into account and coming up with new index types, bit mapped indexes and some new revolutionary index types to make it possible for us to do table scans in some queries in a fraction of the time that you can do otherwise.

We have a really good team to pull this off. We have really high ambitions, and I have full confidence that we will reach them.

Q. What’s a typical day like when you have a team at your house?

As developers we like basically to work late in the evening and then sleep a little bit longer to get some of the energy back. We don’t start too early, around 9:00 or 10:00 and then we have interesting discussions until 5:00 or 6:00 in the evening. Then we eat something good, I hope, and then we continue with discussions late into the night, 2:00 or 3:00 am, 4:00 am is not uncommon.

Q. I think it was after 1:00 am I went to bed the other night and you and Guillem were still working on something.

We were working on logging issues and recovery.

Q. How far along are you in this new project?

We have completed all major parts of the planning stage. A couple of small issues have crept up since our last meeting that we are just ensuring we got right. Now we are going into the coding phase.

Q. What’s your approach with specifications?

What we have tried to do is have complete specs that have all the attributes for the structure we need, APIs, function calls, flow charts for most of the parts. We spent a lot of time doing around 100 page of specs in detail for all the functionality. I believe we have covered all functionality except for cursors which is a new thing we are still discussing in detail. So we don’t expect any surprises in the coding phase.

Q. It seems like a lot of work up front.

The difference with MySQL and other companies is that we are virtual. When we start the coding, we don’t have the flexibility to go next door and ask someone “how should we do that?” By making detailed specs from the start we have a couple of advantages.

We can work on our individual tasks without having to worry about what other people are doing because we already know how the connections of the modules will work. We also have documentation we can go back to, so if we have questions or disagreements we can go back see how it was resolved.

Later if we add new people on the project we don’t have to go over everything in detail with them because we have complete specifications. I also believe that in the future this will help QA find bugs and other people to take over the project when some of us may move on to other things.

Q. Where are the specs stored?

We put the specs into our internal worklog tool. We put all ideas and specs in there. It’s open so anyone from MySQL can check them out, review them, give feedback.

Q. Some people have said that your style of programming favors performance over modularity. Is that true?

I would disagree with that. I believe most things should be black boxes. I believe you need to code the black boxes to make them as fast as possible. Performance doesn’t imply lack of modularity. It’s more a question of the style you use. Just because some code is called very seldom doesn’t mean that it should be done in an inferior manner. You need to develop a discipline of writing code properly so that it’s part of your backbone and that every single line is optimal. Even if a module is seldom used it should still be fast. You need every single line to be optimal. Sooner or later even if that module is not often accessed, someone will use that as a basis for another module that will be accessed a lot. And then you’ll have to modify it if it wasn’t done correctly.

One of the biggest mistakes people make is just copying code from one place to another and in some cases the code is not optimal or suitable, but it’s still in use. Because people have not cleaned it up you get inferior code.

Q. What sort of tools do you use to test your code?

We use some tools to automatically check the correctness of code, like valgrind and Purify. We also force developers to write test cases for all new code. And we have some tools for checking performance with benchmarks. We are putting in place more processes for that. Most importantly, we use code reviews to ensure that the code is well written and designed.

Q. What in your career enabled you to develop a focus on performance?

I started with very small memory machines. Early on I wrote games on the ABC 80, in Basic and Assembler. The ABC had just 16K of memory and a Z-80 processor. So the first database application I wrote had to fit into a very small amount of memory.

Q. When did you start programming in C?

I got Unix in 1982. It was Denix, a Swedish version of AT&T Unix. That’s when I wrote the ancestor to MySQL called Unireg. In 1994 then it became MySQL, which was released in 1995. The first version was released on Solaris with Linux and others soon to follow.

Q. What’s your development system today?

Today I use a normal Intel-based Linux machine with a couple of gigs of RAM. I have a 64 bit Pentium machine for testing. From time to time I use Solaris to run Purify and Pure Coverage. I have a 24 inch screen at the moment. I work by having 3 windows mainly. A debugger on the left, Xemacs in the middle and a shell to the right. And a virtual desktop for handling email, IRC. I should be able to run commands, edit and debug on the same screen. That makes me very productive. For debugging I used DDD which is a wrapper around GDB. That’s one of the best debuggers on the market and even better it’s free. It’s actually what’s known as postcard-ware. If you like it, you send them a postcard.

Q. And did you send one?

I sent a virtual postcard. And I added my comments on the web site.

Q. You had a developer meeting recently in Sorrento. What was that like?

We have a developers meeting once a year where we go over what needs to be done in the future, also what’s coming up for the next release. This year I was focused on MyISAM so I spent most of my time on specs with the rest of the team. It was great seeing everyone, being able to discuss things in a group.

Q. I know that you like to cook. Did you enjoy the food in Italy?

I like Italian food, as long as you can avoid the cheese.

Q. So what are you making for dinner tonight?

Ah…well. (pauses) Actually, I will not tell you. I like to serve things that people don’t usually expect. I cook things from any part of the world. I like to experiment. I make food that I like and a lot of people agree that what I like is not a total disaster. They can at least eat part of it.