First a little background information. I wanted to import an 8 Gig Wikipedia table dump for a project that I was working on. The table was too big to import all of the records on my Linode server. I only needed certain rows and columns, so I tried parsing the dump file in PHP. This worked, but it was much too slow. So then I went looking for text processor programs. This turned up awk, grep, lex and sed. awk, grep and sed are all line based processors. These would not work because the dump file has multiple database records per line. That left lex and its successor flex.
Continue reading Mysqldump to TSV Conversion Using Flex →