xpython - experimenting with xml/language integration holger krekel @ europython 2004 We will perhaps eventually be writing only small modules which are identified by name as they are used to build larger ones, so that devices like indentation, rather than delimiters, might become feasible for expressing local structure in the source language. Donald E. Knuth, "Structured Programming with goto Statements", Computing Surveys, Vol 6 No 4, Dec. 1974 background & motivation for doing xpython ----------------------------------------- - in python visual indentation is significant and is used instead of bracing characters. Very good. - but python also has non-visual start/finish like constructs, e.g. try-finally or try-except statements: for error-handling and finalization. - i was (and still am to a certain degree) scared by the existing xml/html-templating techniques. They are limited, hard to learn and often hard to debug --- compared to python itself. - i wanted to learn more about the C-Python interpreter and stayed away from family activities during X-mas the xpython experimental approach --------------------------------- - introduce a new block statement which allows an object ("execution handler") to interact with the execution of its indented code block. - basic example: class ExecutionHandler: def __enter__(self): print "enter" def __leave__(self): print "leave" : print "hello" - observation: here hooks are ortoghonal to to the definition of the code block. ----> How to do scope/namespace interactions without messing up? real life example: resource finalization (problem) -------------------------------------------------- task: finalizing a file appropriately today's solution: f1 = open(fn1) try: #... do_something_with(f1) #... finally: f1.close() problem: the structure doesn't provide visual clarity especially for longer try-bodies, this gets worse if you have multiple resources (say multithread-locks andfile-accesses) xpython solution for resource finalization ------------------------------------------ class autoclose: def __leave__(self): for value in self.__dict__.values(): if hasattr(value, 'close'): value.close() : do_something_with(f) 1. autoclose is instantiated because it's a class 2. scope interaction: a) open(fn1) is assigned to the local variable 'f' *and* b) the file object is set as an attribute on the autoclose-instance 3. when execution leaves the indented code block, __leave__ is called performing the close-method on any of its instance values. encapsulating exception handling into an object ----------------------------------------------- - writing exception handling code is often bothersome because you can not factor out try-except handling into an object "up-front". - example: uniformly reacting to IO-errors appropriately : data = self.receive (...) the errorhandler code (e.g. for underlying sockets) can be very "precise", because you don't have to repeat it all the time. i.e. it can handle KeyboardInterrupts and SystemExit cleanly, e.g. class ErrorHandler: def __except__(self, cls, value, traceback): self.finalize_connection() if issubclass(cls, (KeyboardInterrupt, SystemExit)): raise raise MainloopContinue Next Step: interacting execution handlers ----------------------------------------- - so far, we know how a single execution handler can interact with its indented block - but handlers can get to know each other by calling a helper function which iterates over parent handlers, e.g. import iexec for x in iexec.iterhandler(): if i_want_this(x): x.notify(...) - this iteration happens along the frame and block-stack chain (cleaner than managing thread states) ----> execution handlers can form a tree ... which reminds us of? why bother with xml templating at all? -------------------------------------- - there are already numerous xml/html templating languages, but - they come up with limited "pseudo-mini-languages/expressions" connecting to python via specific constructs - XSLT is nice for tree-transformations but not so for dynamic generation of web pages - they are often bothersome to debug and hard to learn compared to python itself. - they usually make testing harder than it needs to be compared to debugging python itself. ----> please let's get away from generating strings and requiring "closing" tags! xpython xml-example: fast and easy ---------------------------------- - we are ready to understand how to generate xml/tree structures: from xpy.xml import Namespace my = Namespace('my') : : render_content(...) - in practice xml structures are often generated after dispatching to some function calls further down the chain. - in xpython Tag *objects* can easily connect across function call boundaries. and you don't have to pass "special" parameters like "requests" because they can be safely discovered if needed [html example, pypy web site] current implementation of xpython / 2.3.4 ----------------------------------------- - a ~ 300 line patch to C-Python: - modified grammar to introduce a new blockstatement with <> syntax. - modified builtin compiler to compile this block-statement with the help of two new opcodes (SETUP_IEXEC and END_IEXEC) - modified interpreter loop to implement these two opcodes - a c-coded 'iexec' module to allow iteration of execution handlers - a small 'xpy' support package to provide easy xml/html templating and finalization (locks etc.pp.) features future directions ----------------- - consider using stackless/greenlets to allow for custom control-flow - simplify/optimize the implementation some more - improve the basic html/xml module that allows fast and easy creation of structured documents (maybe extract information out of XIST) - help people using it in real life situations ...