Tino Rachui: Using MySQL Cluster in Sun's Virtual Desktop Infrastructure

Sun Virtual Desktop Infrastructure (VDI3) is a desktop virtualization technology that provides virtual desktop access, brokering, and hosting, enabling Windows, OpenSolaris, or Linux virtual desktops. These desktops can be accessed by using existing PCs, Macs or thin clients such as Sun Ray.

Sun VDI is made up of three main component layers:

• A virtualization platform (utilizing VirtualBox or VMware Infrastructure)
• The VDI Desktop broker
• The desktop access client

The VDI Core configuration data and runtime information are stored in a MySQL database which may be shared across multiple VDI Core instances on the network, thereby ensuring access to the VDI Core even in a failover scenario.

VDI3 - Architecture overview

At this year's Java One Conference in San Francisco, Sun VDI technology was used to provision 21,000 virtual desktops which could be accessed via hundreds of Sun Ray Thin Clients spread all over the conference center. You can read more about the details of this setup in this blog post: Sun VirtualBox and Sun VDI Power JavaOne

Tino Rachui is a member of a development team that works on Desktop Virtualization and Sun Ray development at Sun Microsystems. A part of this group is located in Hamburg, where Tino is based.

I was curious to learn more about how they use MySQL Cluster within the VDI product and recently managed to sit down with Tino to get my questions answered.

Hi Tino, nice to meet you! Can you give our readers a short background about yourself? How long have you been at Sun and what are you currently working on?

Tino Rachui

Tino: Nice to meet you too! I've been working at Sun for more than 9 years. I started in StarOffice/OpenOffice.org development and moved on to the Sun Ray team roughly 3 1/2 years ago. I was initially working on the core parts of Sun Ray, especially the Sun Ray Data Store. However, the focus of my work radically shifted towards desktop virtualization ~2 years ago. Within this effort, I'm mainly responsible for the design and implementation of the VDI configuration database and the database access layer.

Our team is currently in the finishing stages of VDI 3.1, so the daily business at the moment is mainly affected by bug fixing, extensive testing, documentation and all the little tasks required to get a good product out the door.

Great, all the best and good luck for this upcoming release! I was quite excited to learn that MySQL Cluster is actually a vital part of the VDI Core. Can you explain what role it plays in this setup?

Tino: Sure! All static and runtime configuration data that VDI manages (e.g. information about virtual desktops, desktop pools and the association between desktops, pools and users) is stored in a MySQL database.

At configuration time, the user has the choice to use the MySQL Cluster that comes bundled with VDI 3.x or to use an external MySQL database, which can be an ordinary MySQL server. When choosing to use the bundled MySQL Cluster, the user is almost completely unaware that a MySQL Cluster will be configured under the hood. In operation, there is little the user has to take care of regarding the MySQL Cluster.

Using the bundled MySQL Cluster is the suggested configuration as it offers fail-safety: if the VDI database is down, users wouldn't have access to their desktops anymore — so having a solution without a single point of failure (SPOF) was one of the major requirements right from the beginning. This is where MySQL Cluster came to play an essential role in our design.

In essence the MySQL Cluster is an embedded database for the VDI administrator, delivering a Highly Available (HA) service in a fully transparent manner. This is pretty unique in the market, especially no VDI competitor can offer such a level of HA built-in.

I agree, embedding a clustered database right into the application is a unique approach. How do you access this data? Do you use a MySQL Server to communicate with the data nodes or do you use the NDB API directly?

Tino: No, we are not using the NDB API directly, we start an SQL node on every VDI host. The VDI core on this host talks to the local SQL node to get access to the VDI configuration data. There are several reasons for this choice; one of them is the fact that the VDI core is written in Java and we are using the Java Persistence API (JPA) to implement our data access layer. Another one is flexibility. It appeared to be more flexible to access the data via SQL nodes, as we are also supporting the aforementioned external database case.

I see. Yes, going via the MySQL API gives you more flexibility and allows the user to utilize an existing MySQL installation, if required. What are your experiences with this setup? How is MySQL handling this task? Anything we could improve?

Tino: MySQL Cluster lives up to its promise to provide a SPOF-less solution. We've really not been disappointed in this regard. The trade-off is increased complexity in terms of setting-up and managing the database and increased hardware requirements for a high-availability VDI setup.

Ease of use is another big requirement for VDI 3 - this implies that we require little or no MySQL knowledge from our targeted customers. This seems to somehow contradict the decision to use MySQL Cluster. However we've put a lot of effort (quite successfully based on the feedback we've got so far) into hiding this complexity. During the VDI configuration when you choose to use the bundled MySQL Cluster, the user is almost completely unaware that a MySQL Cluster will be configured under the hood. In operation, there is little the user has to take care of regarding the MySQL Cluster.

Achieving this level of complexity-hiding was sometimes challenging, as the management capabilities of MySQL Cluster have some shortcomings. It is, for instance, difficult to completely manage MySQL Cluster from a central point e.g. the management node. For example, it's not easy to control the SQL nodes remotely or start up data nodes from the management node unless you create some additional glue code by yourself (which is what we did). It would be great if there would be a standard way to perform these tasks.

Another ambitious area we've come across was the online upgrade of MySQL Cluster. With the upcoming VDI 3.1 release we are also upgrading to a newer MySQL Cluster version. That means we have to have an upgrade story for our existing VDI 3.0 customers. MySQL Cluster has a rather strict online upgrade procedure. Due to the way we deploy MySQL Cluster this was causing quite some headache for us. For the future we would wish to have a bit more flexibility in this regard, e.g. by relaxing the requirement to restart the SQL nodes/API clients at the very end when all management and data nodes have been upgraded already.

Thanks for these suggestions! I'm pretty sure our developers are aware of these pain points and are working on getting these fixed. What are your future plans with regards to MySQL within VDI?

Tino: Based on the positive experiences we've had up to now I think in the future MySQL should be the first choice for us, not only because of its technical capabilities but also because of the excellent working relationship we've have established with the MySQL Cluster team in the meantime. I've rarely seen a team as competent and responsive as the MySQL Cluster team in my career at Sun and we look forward to further improve the manageability and availability of VDI with Cluster.

Thank you Tino, I appreciate that you spared some of your time to answer my questions!

This interview was performed in November, 2009.