A couple of years ago Domas Mituzas created a tool which could basically be thought of as basically a lightweight multi-threaded mysqldump. By this I mean it can retrieve data from multiple tables simultaneously and can even break a table down into parts for simultaneous retrieval. Sometime around 2010 I started hacking on mydumper too but stopped whilst working at Rackspace (Drizzle was way more than full-time for me).
Back when Domas first blogged about it he was managing to dump his sample data over 10x faster than mysqldump!
Since the 0.1 series Domas has fixed a lot of problems and since starting at SkySQL I have finished a lot of work that I started over a year ago.
Today sees the first release of the 0.2 series of mydumper, called 0.2.0. It has many changes over the last 0.1 release:
- Better error handling
- Many bug fixes
- Documentation
- Binary log dumping
- Compression protocol support
- Schema dumps
My personal favourite fetaure I added here is the binary log dumping. Mydumper can connect to a MySQL server and retrieve all the binary logs in parallel whilst also retrieving the table data.
Many more features are already in development including additional features on the binary log dumping but for those who want to try it now you can obtain the source from Launchpad here.
What do you use to load the data dumped by this?
ReplyDeleteHi Mark,
ReplyDeleteCurrently there is 'myimporter' https://github.com/frimik/myimporter
I want to work on something more C orientated though which can be bundled in with mydumper and support mysqldump (with multi-threaded restore) as well as binlogs.
Why not use mk-parrallel-dump/mk-parrallel-restore from Maatkit?
ReplyDeleteHi Daniël,
ReplyDeleteThree reasons off the top of my head:
1. mydumper is faster (according to Domas' benchmarks)
2. mydumper supports binlog dumping
3. mydumper isn't deprecated
[...] few days ago Mark Callaghan asked about restoring mydumper backups. There is of course the great ‘myimporter‘ tool written by Mikael Fridh, but [...]
ReplyDelete[...] be working with tools such as LinuxJedi’s modifications to Domas’ oh-so-tasty mydumper. Basically, a little bit of everything ; ) One of the first things I’ll be doing is [...]
ReplyDelete