Developing Libdrizzle

This weekend I am supposed to be giving a talk at FOSDEM on Libdrizzle.  Unfortunately my kids and I all fell ill on Thursday (my wife appears to be immune) so I had to cancel my plans (infecting 5000 people didn't seem wise :)

Instead I am writing this blog post about Libdrizzle and my part in it which covers some of what I was going to talk about.

History of Libdrizzle

Libdrizzle started out as a from-scratch C connector for Drizzle and MySQL originally created by Eric Day.  It was designed to be high performance and use common standards to make it easy to work on.  In the summer of 2010 it was merged into the main Drizzle tree where development has been focused.  There were several attempts to split it out again but until now none were truly successful.

For a few years Brian Aker and I have been randomly discussing things we could have done to make the API easier for developers and users.  As well as features we could possibly add in the future.  Fast forward to Summer 2012 and I decided to do something about this.

The Dawn of Libdrizzle Redux

Around July 2012 I took the Libdrizzle 1.0 code from Drizzle trunk and got it to compile separately using CMake.  I then had the groundwork to work on some of the things we had talked about.  I called this the Libdrizzle Redux project since we were bringing new life to Libdrizzle, primarily as a MySQL Connector.

Initially this was going to be version 3.0 but it was pointed out by Henrik that 3.0 and 4.0 were used as library versions in certain packages so we went straight to 5.0.

After getting to the "It Compiles!" state the first thing I did was strip out the server side API.  I had spoken to many people over the years about the connector and each one only used the client side API, not the server side.  The other thing that was made plainly clear was the API was too complex.  There were many functions that were confusing as to whether the library or the application was supposed to allocate and free objects.  So this was massively simplified very quickly.

Libdrizzle Today

Between Brian and I there have been massive changes to the Libdrizzle code and we now have the basis for a regression suite which has already found several problems with the original code.  Just off the top of my head:

• There is a new Prepared Statement API
• There is a new Binary Log Retrieval API (in 5.1.3 this will become a callback API) which is already MySQL 5.6 checksum compatible
• Massively simplified API, the number of things needed to setup a connection and execute a query have been reduced
• Non-blocking API has been made more stable
• Windows support (via. MinGW)
• Buffer and state system massively improved
• CMake has been replace with DDM4
• We compile in C++ with many warnings switch on (C++ compiler tends to find more problems than the normal C compiler)
• 106 revisions/merges in the BZR repo.  Most of this since November 2012.

Last Sunday Brian and I had a hacking day in Seattle, we worked non-stop on Libdrizzle for nearly 12 hours, I doubt there were many sections of code that were not touched that day.

Why I Develop Libdrizzle

My day job is the Project Manager on Load Balancer as a Service at HP Cloud (you should hear a lot more about that in coming months).  Whilst this is great I don't get to touch and debug a whole lot of code any more.  So Libdrizzle is something I do in my spare time to keep my brain fully active.

I also like to think that the code will be useful to someone.  I'm actually a fan of the BSD license in this context.  It means commercial code can easily statically compile with Libdrizzle which in-turn means more MySQL Server/Percona Server/MariaDB Server adoption and more support contracts for Oracle/Percona/SkySQL/etc...

Even if no one uses the code I still find it a fun challenge and something I will continue for as long as I can.

The Future of Libdrizzle

In the next few days we will be releasing Libdrizzle 5.1.3, this will contain all the work that was done in Seattle (and my flights to/from Seattle) including a few nice API improvements.  Along with this I will create the first release of Drizzle Tools.

Drizzle Tools is a project which contains utilities for use with a MySQL server, at the moment the only one included is drizzle_binlogs which connects to a remote MySQL server as a slave and downloads the binary logs to local files as a back.  It also has a continuous mode which effectively gives an up-to-the-second backup of the data.  I have the code half-written for the next tool which will not make the first release but should be useful to many people.

Of course anyone is welcome to come hack on Librizzle and Drizzle Tools, file bugs, ask questions, etc...  The more knowledge share the better :)

There will be more posts soon showing how to use Libdrizzle and drizzle_tools as well as announcing the new features, so watch this space!

Questions about the MariaDB C Connector

I should first start this blog post with the acronym IANAL (I am not a lawyer).  Also the views in this post are mine and not my employer's (a lawyer did ask me to say that part).

Questions on the MariaDB JDBC Driver have already been raised so I wanted to do something similar for the MariaDB C Connector.  The JDBC's issues are mostly ethical whereas my C Connector questions are mostly legal.

But first a little history...  MySQL 3.23.58 was the last version to include an LGPL licensed connector for MySQL, the connector went GPL (and of course commercial) after this version.  Nowadays the connector is GPL with a FLOSS exception, that exception allows you to compile with certain other licensed software.  The main reason for this is PHP version 4 switched from a dual-licensed under the GPL to PHP licensed only.  The PHP license is incompatible with GPL and LGPL licensed in both directions, this is because clauses 3&4 of the PHP license conflicts with clause 6 of the GPL license (which is also clause 10 of the LGPLv2 license).  The FLOSS exception isn't in 3.23.58, it wasn't needed back then.

Why is this relevant?  The MariaDB C Connector is based on the LGPL 3.23 connector and is coded-up to have the current features of the MySQL connector.  Some of this is based on the PHP licensed MySQLnd connector.

Being LGPL licensed is one of the main selling points of the connector.  It means you can dynamically link the library with a commercial application.  Something you would have to buy a license for to do with the MySQL connector.

So, my questions are:

1. How can MariaDB's C Connector legally include LGPL and PHP licensed code in the same source under the LGPL umbrella?

2. There are questions around the licensing and availability of the native LGPL SDBC (OpenOffice) driver that other parts of the connector are based on.  In the comments of another blog post it was claimed it that the SDBC code was "... developed by MySQL in 2006 and 2007 and never reached GA – discontinued when Sun Microsystems acquired MySQL AB" but I haven't seen any evidence of this.  The only Sun code I can find for SDBC is 2008/2009 and LGPLv3 which can't be re-licensed as LGPLv2.  Now there is a newer MySQL SDBC driver which is Apache2 licensed.  Where can we find the code MaraDB based their work on?

3. Although not distributed in the packages there are GPL licensed test cases in the bzr source.  Can you globally call it LGPL with these in there?

I do hope answers to these can be put to rest so that customers can continue to be happy using the alternative connector.