Data Crunching

Every day, programmers all around the world have to recycle legacy data, translate from one proprietary format to another, check that configuration files are internally consistent, and search through web logs to see how many people have downloaded the latest release of their product.

This kind of data crunching may not be glamorous, but knowing how to do it effectively is essential to being a good programmer. This book describes the most useful data crunching techniques, explains when you should use them, and shows how they will make your life easier. Along the way, it will introduce you to some handy, but underused, features of Java, Python, and other languages. It will also show you how to test data crunching programs and how data crunching fits into the larger software development picture.

After reading this book, you’ll be able to

• Tackle real-world legacy data conversion problems;
• Apply advanced tools such as regular expressions and XSLT;
• Transform XML and other structured documents;
• Extract ad hoc information from relational databases using SQL;
• Design, build, and test programs for searching log files, converting data sources, configuring other programs, and more!

Whether you are a programmer, a system administrator, or a QA team member working late to make sure the latest beta is ready for release, if you feel like you’re spending too much time on the “other stuff” that every project seems to contain, then this book is for you. About the AuthorGreg Wilson holds a Ph.D. in Computer Science from the University of Edinburgh, and has worked on high-performance scientific computing, data visualization, and computer security. He is the author of Practical Parallel Programming (MIT Press, 1995), is a contributing editor at Doctor Dobb’s Journal, and an adjunct professor in Computer Science at the University of Toronto.