| Python |
| Perl |
| PHP |
| Java |
Foo
). The module ll.xist.ns.html does this automatically: >>> from ll.xist.ns import html >>> node = html.p(html.p(u"foo")) >>> print node.asBytes() /home/walter/pythonroot/ll/xist/sims.py:238: WrongElementWarning: elementfoo
For your own elements you can specify the content model too. This is done by setting the class attribute model inside the element class. model must be an object that provides a checkvalid method. This method will be called during parsing or publishing with the element as an argument. When a validation violation is detected, the Python warning framework should be used to issue a warning. The module ll.xist.sims contains several classes that provide simple validation methods: Empty can be used to ensure that the element doesn't have any content (like br and img in HTML). Any does allow any content. NoElements will warn about elements from the same namespace (elements from other namespaces will be OK). NoElementsOrText will warn about elements from the same namespace and non-whitespace text content. Elements will only allow the elements specified in the constructor. ElementsOrText will only allow the elements specified in the constructor and text. None of these classes will check the number of child elements or their order. For more info see the sims module. Entities -------- In the same way as defining new element types, you can define new entities. But to be able to use the new entities in an XML file you have to use a parser that supports reporting undefined entities to the application via skippedEntity (SGMLOPParser and ExpatParser in the module ll.xist.parsers do that). The following example is from the module ll.xist.ns.abbr: from ll.xist import xsc from ll.xist.ns import html class html(xsc.Entity): def convert(self, converter): return html.abbr( u"HTML", title=u"Hypertext Markup Language", lang=u"en" ) You can use this entity in your XML files like this:AäΩ語>
>>> print node.asBytes(encoding="iso-8859-1")
AäΩ語>
>>> print xsc.Comment(s).asBytes()
Traceback (most recent call last):
File "", line 1, in ?
File "/home/walter/pythonroot/ll/xist/xsc.py", line 600, in asBytes
publisher.publish(stream, self, base)
File "/home/walter/pythonroot/ll/xist/publishers.py", line 205, in publish
self.node.publish(self)
File "/home/walter/pythonroot/ll/xist/xsc.py", line 1305, in publish
publisher.write(self.content)
File "/usr/local/lib/python2.3/codecs.py", line 178, in write
data, consumed = self.encode(object, self.errors)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-4: ordinal not in range(128)
When you include an XML header or an HTML meta header, XIST will
automatically insert the correct encoding when publishing:
>>> from ll.xist import xsc
>>> from ll.xist.ns import xml, meta
>>> e = xsc.Frag(xml.XML10(), u"\n", meta.contenttype())
>>> print e.conv().asBytes(encoding="iso-8859-15")
Another useful parameter is xhtml, it specifies whether you want
pure HTML or XHTML as output:
xhtml==0
This will give you pure HTML, i.e. no final / for elements
with an empty content model, so you'll get e.g.
in the output. Elements that don't have an empty content model, but are empty will be published with a start and end tag (i.e. ). xhtml==1 This gives HTML compatible XHTML. Elements with an empty content model will be published like this:
(This is the default). xhtml==2 This gives full XML output. Every empty element will be published with an empty tag (without an additional space):
or . Writing a node to a file can be done with the method write: >>> from ll.xist.ns import html >>> node = html.div(u"äöü", html.br(), u"ÄÖÜ") >>> node.write(open("foo.html", "wb"), encoding="ascii") All these methods use the method publish internally. publish gets passed an instance of ll.xist.publisher.Publisher. Searching trees =============== There are two methods available for iterating through an XML tree and finding nodes in the tree: The walk method and XFind expressions. The walk method --------------- The method walk is a generator. You pass a callable object to walk which is used for determining which part of the tree should be searched and which nodes should be returned. ll.xist.xsc provides several useful predefined classes for specifying what should be returned from walk: FindType will search only the first level of the tree and will return any node that is an instance of one of the classes passed to the constructor. So if you have an instance of ll.xist.ns.html.ul named node you could search for all ll.xist.ns.html.li elements inside with the following code: for cursor in node.content.walk(xsc.FindType(html.li)): print unicode(cursor.node) FindTypeAll can be used when you want to search the complete tree. The following example extracts all the links on the Python home page: from ll.xist import xsc, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) for cursor in node.walk(xsc.FindTypeAll(html.a)): print cursor.node[u"href"] This gives the output: http://www.python.org/ http://www.python.org/search/ http://www.python.org/download/ http://www.python.org/doc/ http://www.python.org/Help.html http://www.python.org/dev/ ... The following example will find all external links on the Python home page: from ll.xist import xsc, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) def isextlink(cursor): if isinstance(cursor.node, html.a) and not unicode(cursor.node[u"href"]).startswith(u"http://www.python.org"): return (True, xsc.entercontent) return (xsc.entercontent,) for cursor in node.walk(isextlink): print cursor.node[u"href"] This gives the output: http://www.jython.org/ http://sourceforge.net/tracker/?atid=105470&group%5fid=5470 http://sourceforge.net/tracker/?atid=305470&group%5fid=5470 http://sourceforge.net/cvs/?group%5fid=5470 http://www.python-in-business.org/ http://www.europython.org/ mailto:webmaster@python.org ... The callable (isextlink in the example) will be called for each node visited. The cursor argument has an attribute node that is the node in question. For the other attributes see the Cursor class. The callable must return a sequence with the following entries: ll.xist.xsc.entercontent enter the content of this element and continue searching; ll.xist.xsc.enterattrs enter the attributes of this element and continue searching; boolean value If true, the node will be part of the result. The sequence will be "executed" in the order you specify. To change the top down traversal from our example to a bottom up traversal we could change isextlink to the following (note the swapped tuple entries): def isextlink(node): if isinstance(node, html.a) and not unicode(node[u"href"]).startswith(u"http://www.python.org"): return (xsc.entercontent, True) return (xsc.entercontent,) Note that the cursor yielded from walk will be reused by subsequent next calls, so you should not modify the cursor and you can't rely on attributes of the cursor after reentry to walk. XFind expressions ----------------- A second method exists for iterating through a tree: XFind expressions. An XFind expression looks somewhat like an XPath expression, but is implemented as a pure Python expression (overloading the division operators). Our example from above that searched for lis inside uls can be rewritten as follows: for child in node/html.li: print unicode(child) A XFind expression returns an iterator for certain parts of the XML tree. In an XFind expression a/b, a must be either a node or an iterator producing nodes (note that an XFind expression itself is such an iterator, so a itself might be a XFind expression). b must be an XFind operator. Every subclass of ll.xist.xsc.Node is a XFind operator. If b is such a subclass, a/b will produce any child nodes of the nodes from a that is an instance of b. If b is an attribute class, you will get attribute nodes instead of child nodes. Other XFind operators can be found in the module ll.xist.xfind. The all operator will produce every node in the tree (except for attributes): from ll.xist import xfind from ll.xist.ns import html node = html.div( html.div( html.div(id=3), html.div(id=4), id=2, ), html.div( html.div(id=6), html.div(id=7), id=5, ), id=1 ) for child in node/xfind.all: print child["id"] The output of this is: 1 2 3 4 5 6 7 The following example demonstrates how to find all links on the Python homepage via an XFind expression: from ll.xist import xfind, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) for link in node/xfind.all/html.a: print link["href"] An all operator in the middle of an XFind expression can be abbreviated. The XFind expression from the last example (node/xfind.all/html.a) can be rewritten like this: node//html.a. Another XFind operator is contains. It acts as a filter, i.e. the nodes produced by a/xfind.contains(b) are a subset of the nodes produced by a, those that contain child nodes of type b. Searching for all links on the Python home page that contain images can be done like this: >>> from ll.xist import xfind, parsers >>> from ll.xist.ns import html >>> node = parsers.parseURL("http://www.python.org/", tidy=True) >>> for link in node//html.a/xfind.contains(html.img): ... print link["href"] ... http://www.python.org/ http://www.python.org/psf/donations.html http://www.opensource.org/ Note that using the all operator twice in an XFind expression currently won't give you the expected result, as nodes might be produced twice. Calling __getitem__ on an XFind operator gives you an item operator. Such an item operator only returns a specific item (or slice) of those nodes returned by the base iterator. An example: >>> from ll.xist.ns import html >>> e = html.table(html.tr(html.td(j) for j in xrange(i, i+3)) for i in xrange(1, 10, 3)) >>> print e.pretty().asBytes()
>>> # Every cell
>>> for td in e/html.tr/html.td:
... print td
...
1
2
3
4
5
6
7
8
9
>>> # Every first cell in each row
>>> for td in e/html.tr/html.td[0]:
... print td
1
4
7
>>> # Every cell in the first row
>>> for td in e/html.tr[0]/html.td:
... print td
1
2
3
>>> # The first of all cells
>>> for td in e/(html.tr/html.td)[0]:
... print td
1
Manipulating trees
==================
XIST provides many methods for manipulating an XML tree.
The method withsep can be used to put a seperator node between the
child nodes of an Element or a Frag:
>>> from ll.xist import xsc
>>> from ll.xist.ns import html
>>> node = html.div(*xrange(10))
>>> print node.withsep(", ").asBytes()
', base="root:spam/index.html")
>>> print node.asBytes()
For publishing a base URL can be specified too. URLs will be
published relative to this base URL with the exception of relative
URLs in the tree. This means:
* When you have a relative URL (e.g. #top) generated by a
convert call, this URL will stay the same when publishing.
* Base URLs for parsing should never be relative: Relative base
URLs will be prepended to all relative URLs in the file, but
this will not be reverted for publishing. In most cases the
base URL should be a root URL when you parse local files.
* When you parse remote web pages you can either omit the base
argument, so it will default to the URL being parsing, so that
links, images, etc. on the page will still point back to their
original location, or you might want to use the empty URL
URL() as the base, so you'll get all URLs in the page as they
are.
* When XIST is used as a compiler for static pages, you're going
to read source XML files, do a conversion and write the result
to a new target file. In this case you should probably use the
URL of the target file for both parsing and publishing. Let's
assume we have an URL #top in the source file. When we use the
"real" file names for parsing and publishing like this:
node = parsers.parseFile("spam.htmlxsc", base="root:spam.htmlxsc")
node = node.conv()
node.write(open("spam.html", "wb"), base="root:spam.html")
the following will happen: The URL #top will be parsed as
root:spam.htmlxsc#top. After conversion this will be written
to spam.html relative to the URL root:spam.html, which results
in spam.html#top, which works, but is not what you want.
When you use root:spam.html both for parsing and publishing,
#top will be written to the target file as expected.
Pretty printing XML
===================
The method pretty can be used for pretty printing XML. It returns
a new version of the node, with additional white space between the
elements:
from ll.xist.ns import html
node = html.html(
html.head(
html.title(u"foo"),
),
html.body(
html.div(
html.h1(u"The ", html.em(u"foo"), u" page!"),
html.p(u"Welcome to the ", html.em(u"foo"), u" page."),
),
),
)
print node.pretty().asBytes()
This will print:
foo
Element content will only be modified if it doesn't contain Text
nodes, so mixed content will not be touched.
Automatic generation of image size attributes
=============================================
The module ll.xist.ns.htmlspecials contains an element autoimg
that extends ll.xist.ns.html.img. When converted to HTML via the
convert method the size of the image will be determined and the
height and width attributes will be set accordingly (if those
attributes are not set already).
Embedding Python code
=====================
It's possible to embed Python code into XIST XML files. For this
XIST supports two new processing instructions: pyexec and pyeval
(in the module ll.xist.ns.code). The content of pyexec will be
executed when the processing instruction node is converted.
The result of a call to convert for a pyeval processing
instruction is whatever the Python code in the content returns.
The processing instruction content is treated as the body of a
function, so you can put multiple return statements there. The
converter is available as the parameter converter inside the
processing instruction. For example, consider the following XML
file:
Parsing this file and calling convert results in the following:
5050
in the output. Elements that don't have an empty content model, but are empty will be published with a start and end tag (i.e. ). xhtml==1 This gives HTML compatible XHTML. Elements with an empty content model will be published like this:
(This is the default). xhtml==2 This gives full XML output. Every empty element will be published with an empty tag (without an additional space):
or . Writing a node to a file can be done with the method write: >>> from ll.xist.ns import html >>> node = html.div(u"äöü", html.br(), u"ÄÖÜ") >>> node.write(open("foo.html", "wb"), encoding="ascii") All these methods use the method publish internally. publish gets passed an instance of ll.xist.publisher.Publisher. Searching trees =============== There are two methods available for iterating through an XML tree and finding nodes in the tree: The walk method and XFind expressions. The walk method --------------- The method walk is a generator. You pass a callable object to walk which is used for determining which part of the tree should be searched and which nodes should be returned. ll.xist.xsc provides several useful predefined classes for specifying what should be returned from walk: FindType will search only the first level of the tree and will return any node that is an instance of one of the classes passed to the constructor. So if you have an instance of ll.xist.ns.html.ul named node you could search for all ll.xist.ns.html.li elements inside with the following code: for cursor in node.content.walk(xsc.FindType(html.li)): print unicode(cursor.node) FindTypeAll can be used when you want to search the complete tree. The following example extracts all the links on the Python home page: from ll.xist import xsc, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) for cursor in node.walk(xsc.FindTypeAll(html.a)): print cursor.node[u"href"] This gives the output: http://www.python.org/ http://www.python.org/search/ http://www.python.org/download/ http://www.python.org/doc/ http://www.python.org/Help.html http://www.python.org/dev/ ... The following example will find all external links on the Python home page: from ll.xist import xsc, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) def isextlink(cursor): if isinstance(cursor.node, html.a) and not unicode(cursor.node[u"href"]).startswith(u"http://www.python.org"): return (True, xsc.entercontent) return (xsc.entercontent,) for cursor in node.walk(isextlink): print cursor.node[u"href"] This gives the output: http://www.jython.org/ http://sourceforge.net/tracker/?atid=105470&group%5fid=5470 http://sourceforge.net/tracker/?atid=305470&group%5fid=5470 http://sourceforge.net/cvs/?group%5fid=5470 http://www.python-in-business.org/ http://www.europython.org/ mailto:webmaster@python.org ... The callable (isextlink in the example) will be called for each node visited. The cursor argument has an attribute node that is the node in question. For the other attributes see the Cursor class. The callable must return a sequence with the following entries: ll.xist.xsc.entercontent enter the content of this element and continue searching; ll.xist.xsc.enterattrs enter the attributes of this element and continue searching; boolean value If true, the node will be part of the result. The sequence will be "executed" in the order you specify. To change the top down traversal from our example to a bottom up traversal we could change isextlink to the following (note the swapped tuple entries): def isextlink(node): if isinstance(node, html.a) and not unicode(node[u"href"]).startswith(u"http://www.python.org"): return (xsc.entercontent, True) return (xsc.entercontent,) Note that the cursor yielded from walk will be reused by subsequent next calls, so you should not modify the cursor and you can't rely on attributes of the cursor after reentry to walk. XFind expressions ----------------- A second method exists for iterating through a tree: XFind expressions. An XFind expression looks somewhat like an XPath expression, but is implemented as a pure Python expression (overloading the division operators). Our example from above that searched for lis inside uls can be rewritten as follows: for child in node/html.li: print unicode(child) A XFind expression returns an iterator for certain parts of the XML tree. In an XFind expression a/b, a must be either a node or an iterator producing nodes (note that an XFind expression itself is such an iterator, so a itself might be a XFind expression). b must be an XFind operator. Every subclass of ll.xist.xsc.Node is a XFind operator. If b is such a subclass, a/b will produce any child nodes of the nodes from a that is an instance of b. If b is an attribute class, you will get attribute nodes instead of child nodes. Other XFind operators can be found in the module ll.xist.xfind. The all operator will produce every node in the tree (except for attributes): from ll.xist import xfind from ll.xist.ns import html node = html.div( html.div( html.div(id=3), html.div(id=4), id=2, ), html.div( html.div(id=6), html.div(id=7), id=5, ), id=1 ) for child in node/xfind.all: print child["id"] The output of this is: 1 2 3 4 5 6 7 The following example demonstrates how to find all links on the Python homepage via an XFind expression: from ll.xist import xfind, parsers from ll.xist.ns import html node = parsers.parseURL("http://www.python.org/", tidy=True) for link in node/xfind.all/html.a: print link["href"] An all operator in the middle of an XFind expression can be abbreviated. The XFind expression from the last example (node/xfind.all/html.a) can be rewritten like this: node//html.a. Another XFind operator is contains. It acts as a filter, i.e. the nodes produced by a/xfind.contains(b) are a subset of the nodes produced by a, those that contain child nodes of type b. Searching for all links on the Python home page that contain images can be done like this: >>> from ll.xist import xfind, parsers >>> from ll.xist.ns import html >>> node = parsers.parseURL("http://www.python.org/", tidy=True) >>> for link in node//html.a/xfind.contains(html.img): ... print link["href"] ... http://www.python.org/ http://www.python.org/psf/donations.html http://www.opensource.org/ Note that using the all operator twice in an XFind expression currently won't give you the expected result, as nodes might be produced twice. Calling __getitem__ on an XFind operator gives you an item operator. Such an item operator only returns a specific item (or slice) of those nodes returned by the base iterator. An example: >>> from ll.xist.ns import html >>> e = html.table(html.tr(html.td(j) for j in xrange(i, i+3)) for i in xrange(1, 10, 3)) >>> print e.pretty().asBytes()
| 1 | 2 | 3 |
| 4 | 5 | 6 |
| 7 | 8 | 9 |
0, 1, 2, 3, 4, 5, 6, 7, 8, 9
The method shuffled returns a shuffled version of the Element or
Frag:
>>> from ll.xist import xsc
>>> from ll.xist.ns import html
>>> node = html.div(*xrange(10))
>>> print node.shuffled().withsep(", ").asBytes()
8, 1, 3, 6, 7, 5, 2, 9, 4, 0
There are methods named reversed and sorted that return a reversed
or sorted version of an element or fragment:
>>> from ll.xist import xsc
>>> from ll.xist.ns import html
>>> def key(n):
... return unicode(n)
>>> node = html.div(8,4,2,1,9,6,3,0,7,5)
>>> print node.sorted(key=key).reversed().withsep(",").asBytes()
9,8,7,6,5,4,3,2,1,0
The method mapped recursively walks the tree and generates a new
tree, where all the nodes are mapped through a function. An
example: To replace Python with Parrot in every text node on the
Python page, do the following:
from ll.xist import xsc, parsers
def p2p(node, converter):
if isinstance(node, xsc.Text):
node = node.replace(u"Python", u"Parrot")
node = node.replace(u"python", u"parrot")
return node
node = parsers.parseURL("http://www.python.org/", tidy=True)
node = node.mapped(p2p)
node.write(open("parrot_index.html", "wb"))
The function must either return a new node, in which case this new
node will be used instead of the old one, or return the old node
to tell mapped that it should recursively continue with the
content of the node.
URLs
====
For URL handling XIST uses the module ll.url. Refer to its
documentation for the basic functionality (especially regarding
the methods __div__ and relative).
When XIST parses an XML resource it uses a so called "base" URL.
This base URL can be passed to all parsing functions. If it isn't
specified it defaults to the URL of the resource being parsed.
This base URL will be prepended to all URLs that are read during
parsing:
>>> from ll.xist import parsers
>>> from ll.xist.ns import html
>>> node = parsers.parseString('', base="root:spam/index.html")
>>> print node.asBytes()
For publishing a base URL can be specified too. URLs will be
published relative to this base URL with the exception of relative
URLs in the tree. This means:
* When you have a relative URL (e.g. #top) generated by a
convert call, this URL will stay the same when publishing.
* Base URLs for parsing should never be relative: Relative base
URLs will be prepended to all relative URLs in the file, but
this will not be reverted for publishing. In most cases the
base URL should be a root URL when you parse local files.
* When you parse remote web pages you can either omit the base
argument, so it will default to the URL being parsing, so that
links, images, etc. on the page will still point back to their
original location, or you might want to use the empty URL
URL() as the base, so you'll get all URLs in the page as they
are.
* When XIST is used as a compiler for static pages, you're going
to read source XML files, do a conversion and write the result
to a new target file. In this case you should probably use the
URL of the target file for both parsing and publishing. Let's
assume we have an URL #top in the source file. When we use the
"real" file names for parsing and publishing like this:
node = parsers.parseFile("spam.htmlxsc", base="root:spam.htmlxsc")
node = node.conv()
node.write(open("spam.html", "wb"), base="root:spam.html")
the following will happen: The URL #top will be parsed as
root:spam.htmlxsc#top. After conversion this will be written
to spam.html relative to the URL root:spam.html, which results
in spam.html#top, which works, but is not what you want.
When you use root:spam.html both for parsing and publishing,
#top will be written to the target file as expected.
Pretty printing XML
===================
The method pretty can be used for pretty printing XML. It returns
a new version of the node, with additional white space between the
elements:
from ll.xist.ns import html
node = html.html(
html.head(
html.title(u"foo"),
),
html.body(
html.div(
html.h1(u"The ", html.em(u"foo"), u" page!"),
html.p(u"Welcome to the ", html.em(u"foo"), u" page."),
),
),
)
print node.pretty().asBytes()
This will print:
The foo page!
Welcome to the foo page.