iLike frequently receives large XML files from our partners that we need to analyze.

For ad-hoc analysis of XML from command line, xml_grep is a really valuable tool. Instead of extracting lines that match regular expressions, xml_grep extracts xml nodes that match xpath expressions.

However, it took me a bit of trial and error to figure out how install and use it successfully. Here's a quick cookbook of how I did it; I'll update this post with additional useful examples as I encounter them.

Installation

In most *nix distributions, you can install by simple typing

sudo cpan -i "XML::Twig"

Answer 'y' when Cpan asks you if you want to install other packages that xml_grep depends on.

(xml_grep is included as a part of the XML::Twig package in CPAN. XML::Twig also provides a rich Perl API; however the command line examples here do not require any Perl knowledge.)

Usage examples

Suppose you have the following xml:

xml_grep allows you to specify both --root (the node to match and print out) and --cond (an xpath expression relative to root that filters the results.) If more than one root is provided, the results are combined using OR; likewise with cond.

1. Here's the simplest possible example. --root is an xpath expression specifying which nodes will be printed. -p causes the results to be pretty printed:

Print all events : xml_grep -p --root="Event" foo.xml

2. The next example shows how we can use --cond to filter the results based on an attribute of a subnode of the root.

Print all events for a given artist: xml_grep -p --root="Event" --cond='ArtistID[@ID='806533']' foo.xml

3. Here's how we can match against the text contents of a subelements:

Print all events on a given date: xml_grep -p --root="Event" --cond='EventDate[string()="2009-07-25"]' foo.xml

Multiple cond flags are combined using OR. To print nodes matching one of several artists:

4.Print all events for a given artist: xml_grep -p --root="Event" --cond='ArtistID[@ID='806533']' --cond='ArtistID[@ID='806532']' foo.xml

5. The result of xml_grep is itself an XML document and can be piped into another xml_grep to do additional extraction.

Extract the time and performance name of events on a given date: xml_grep -p --root="Event" --cond='EventDate[string()="2009-07-25"]' foo.xml | xml_grep --root='EventTime' --root='PerformanceName'

5. The result of xml_grep is itself an XML document and can be piped into another xml_grep to do additional extraction.

6. Match a regular expression: xml_grep --root="EventName[string() =~ /Bil.* El.*/]" foo.xml