From ianb at codespeak.net Wed Oct 1 08:57:20 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 08:57:20 +0200 (CEST) Subject: [z3-checkins] r58512 - z3/deliverance/sandbox/ianb/deliverance/trunk Message-ID: <20081001065720.E5A091684D6@codespeak.net> Author: ianb Date: Wed Oct 1 08:57:18 2008 New Revision: 58512 Added: z3/deliverance/sandbox/ianb/deliverance/trunk/build-bundle (contents, props changed) Modified: z3/deliverance/sandbox/ianb/deliverance/trunk/ (props changed) z3/deliverance/sandbox/ianb/deliverance/trunk/regen-docs Log: Added a script to create snapshot bundles Added: z3/deliverance/sandbox/ianb/deliverance/trunk/build-bundle ============================================================================== --- (empty file) +++ z3/deliverance/sandbox/ianb/deliverance/trunk/build-bundle Wed Oct 1 08:57:18 2008 @@ -0,0 +1,34 @@ +#!/bin/sh + +fail () { + echo "$@" + exit 1 +} + +REV=$(python -c 'import pyinstall; print pyinstall.get_svn_revision(".")') +FILE="Deliverance-snapshot-r${REV}.pybundle" + +if [ -e "$FILE" ] ; then + echo "Bundle file $FILE already exists" + exit 0 +fi + +echo "CREATING BUNDLE: $FILE" +echo + +pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ + -e svn+http://codespeak.net/svn/z3/deliverance/sandbox/ianb/deliverance/trunk#egg=Deliverance \ + || fail 'Bundle failed' + +echo 'INSTALLING' +echo + +rm -rf TEST +pyinstall.py -E TEST Deliverance.pybundle \ + || fail 'Install failed' + +cd TEST/src/deliverance +chmod +x test +./test \ + || fail 'Tests failed' + Modified: z3/deliverance/sandbox/ianb/deliverance/trunk/regen-docs ============================================================================== --- z3/deliverance/sandbox/ianb/deliverance/trunk/regen-docs (original) +++ z3/deliverance/sandbox/ianb/deliverance/trunk/regen-docs Wed Oct 1 08:57:18 2008 @@ -3,8 +3,7 @@ mkdir -p docs/_static docs/_build sphinx-build -E -b html docs/ docs/_build || exit 1 if [ "$1" = "publish" ] ; then - cd docs/ + cd docs/_build echo "Uploading files..." - #scp -r _build/* - echo "(no upload location set)" + tar czvf - . | ssh flow.openplans.org 'ssh acura.openplans.org "cd /www/deliverance.openplans.org/; tar xzvf -"' fi From ianb at codespeak.net Wed Oct 1 08:59:37 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 08:59:37 +0200 (CEST) Subject: [z3-checkins] r58513 - z3/deliverance/sandbox/ianb/deliverance/trunk Message-ID: <20081001065937.178B616A009@codespeak.net> Author: ianb Date: Wed Oct 1 08:59:36 2008 New Revision: 58513 Modified: z3/deliverance/sandbox/ianb/deliverance/trunk/ (props changed) Log: ignore the TEST directory From ianb at codespeak.net Wed Oct 1 19:52:42 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 19:52:42 +0200 (CEST) Subject: [z3-checkins] r58536 - z3/deliverance/sandbox/ianb/deliverance/trunk Message-ID: <20081001175242.B65EA169F71@codespeak.net> Author: ianb Date: Wed Oct 1 19:52:39 2008 New Revision: 58536 Modified: z3/deliverance/sandbox/ianb/deliverance/trunk/setup.py Log: update URL Modified: z3/deliverance/sandbox/ianb/deliverance/trunk/setup.py ============================================================================== --- z3/deliverance/sandbox/ianb/deliverance/trunk/setup.py (original) +++ z3/deliverance/sandbox/ianb/deliverance/trunk/setup.py Wed Oct 1 19:52:39 2008 @@ -21,7 +21,7 @@ keywords='wsgi theming html', author='Ian Bicking, The Open Planning Project', author_email='deliverance-discuss at lists.openplans.org', - url='http://openplans.org/projects/deliverance/', + url='http://deliverance.openplans.org/', license='MIT', packages=find_packages(exclude=['ez_setup', 'examples', 'tests']), include_package_data=True, From ianb at codespeak.net Wed Oct 1 19:53:06 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 19:53:06 +0200 (CEST) Subject: [z3-checkins] r58537 - z3/deliverance/trunk Message-ID: <20081001175306.DD3D8169F28@codespeak.net> Author: ianb Date: Wed Oct 1 19:53:06 2008 New Revision: 58537 Removed: z3/deliverance/trunk/ Log: Moving old trunk aside for new experimental trunk to take its place From ianb at codespeak.net Wed Oct 1 19:53:31 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 19:53:31 +0200 (CEST) Subject: [z3-checkins] r58538 - in z3/deliverance: sandbox/ianb/deliverance/trunk trunk Message-ID: <20081001175331.719E9169F28@codespeak.net> Author: ianb Date: Wed Oct 1 19:53:30 2008 New Revision: 58538 Added: z3/deliverance/trunk/ - copied from r58537, z3/deliverance/sandbox/ianb/deliverance/trunk/ Removed: z3/deliverance/sandbox/ianb/deliverance/trunk/ Log: Moving experimental code into the main trunk From ianb at codespeak.net Wed Oct 1 19:54:01 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 19:54:01 +0200 (CEST) Subject: [z3-checkins] r58539 - z3/deliverance/sandbox/ianb/deliverance Message-ID: <20081001175401.1AC16169F28@codespeak.net> Author: ianb Date: Wed Oct 1 19:54:00 2008 New Revision: 58539 Removed: z3/deliverance/sandbox/ianb/deliverance/ Log: removing defunct structure From ianb at codespeak.net Wed Oct 1 22:05:25 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 22:05:25 +0200 (CEST) Subject: [z3-checkins] r58541 - z3/deliverance/trunk Message-ID: <20081001200525.B95C3169E29@codespeak.net> Author: ianb Date: Wed Oct 1 22:05:24 2008 New Revision: 58541 Modified: z3/deliverance/trunk/build-bundle Log: Fix up some problems with the bundle builder Modified: z3/deliverance/trunk/build-bundle ============================================================================== --- z3/deliverance/trunk/build-bundle (original) +++ z3/deliverance/trunk/build-bundle Wed Oct 1 22:05:24 2008 @@ -16,15 +16,17 @@ echo "CREATING BUNDLE: $FILE" echo +## httplib2 location is a temporary problem with a bad pypi entry pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ - -e svn+http://codespeak.net/svn/z3/deliverance/sandbox/ianb/deliverance/trunk#egg=Deliverance \ + http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz \ + -e svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=Deliverance \ || fail 'Bundle failed' echo 'INSTALLING' echo rm -rf TEST -pyinstall.py -E TEST Deliverance.pybundle \ +pyinstall.py -E TEST $FILE \ || fail 'Install failed' cd TEST/src/deliverance From ianb at codespeak.net Wed Oct 1 22:06:10 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Wed, 1 Oct 2008 22:06:10 +0200 (CEST) Subject: [z3-checkins] r58542 - in z3/deliverance/trunk/deliverance: . util Message-ID: <20081001200610.53D40169E29@codespeak.net> Author: ianb Date: Wed Oct 1 22:06:09 2008 New Revision: 58542 Added: z3/deliverance/trunk/deliverance/util/urlnormalize.py (contents, props changed) Modified: z3/deliverance/trunk/deliverance/proxy.py Log: Normalize URLs when proxying. Specifically works around problems where a destination URL has : in the path, which in a normalized form shows up as %3A, but the user may have entered it in without quoting the : in the config Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Wed Oct 1 22:06:09 2008 @@ -27,6 +27,7 @@ from deliverance.security import execute_pyref from deliverance.pyref import PyReference from deliverance.util.filetourl import filename_to_url, url_to_filename +from deliverance.util.urlnormalize import url_normalize class ProxySet(object): """ @@ -229,9 +230,11 @@ def proxy_to_dest(self, request, dest): """Do the actual proxying, without applying any transformations""" # Not using request.copy because I don't want to copy wsgi.input: - orig_base = request.application_url + orig_base = url_normalize(request.application_url) + dest = url_normalize(dest) proxy_req = Request(request.environ.copy()) scheme, netloc, path, query, fragment = urlparse.urlsplit(dest) + path = urllib.unquote(path) assert not fragment, ( "Unexpected fragment: %r" % fragment) if scheme == 'file': @@ -262,7 +265,7 @@ if self.strip_script_name: proxy_req.headers['X-Forwarded-Path'] = proxy_req.script_name proxy_req.script_name = '' - proxied_url = '%s://%s%s' % (scheme, netloc, proxy_req.path_qs) + proxied_url = url_normalize('%s://%s%s' % (scheme, netloc, proxy_req.path_qs)) try: resp = proxy_req.get_response(proxy_exact_request) except socket.error, e: Added: z3/deliverance/trunk/deliverance/util/urlnormalize.py ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/util/urlnormalize.py Wed Oct 1 22:06:09 2008 @@ -0,0 +1,40 @@ +"""Normalize URLs""" +import urlparse +import urllib +import re + +def url_normalize(url): + """Normalizes the quoting of URLs, quoting any characters that should + be quoted (but not double-quoting already quoted characters)""" + scheme, netloc, path, query, fragment = urlparse.urlsplit(url) + scheme = scheme.lower() + if ':' in netloc: + host, port = netloc.split(':', 1) + if scheme == 'http' and port == '80': + netloc = host + elif scheme == 'https' and port == '443': + netloc = host + netloc = netloc.lower() + path = _quote_special(path) + if query: + path += '?' + query + if fragment: + path += '#' + fragment + result = '%s://%s%s' % (scheme, netloc, path) + return result + +_slash_re = re.compile(r'%2f', re.I) + +def _quote_special(path): + """Quotes any characters in the path that should be quoted, unquotes + characters that don't need to be quoted. Also % quoting is + upper-cased""" + parts = [_quote_special_part(part) + for part in _slash_re.split(path)] + return '%2F'.join(parts) + +_percent_re = re.compile(r'%[0-9a-f][0-9a-f]', re.I) + +def _quote_special_part(part): + return urllib.quote(urllib.unquote(part)) + From ianb at codespeak.net Sat Oct 4 23:53:12 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Sat, 4 Oct 2008 23:53:12 +0200 (CEST) Subject: [z3-checkins] r58597 - in z3/deliverance/trunk: . docs docs/_static docs/_templates Message-ID: <20081004215312.538D116A120@codespeak.net> Author: ianb Date: Sat Oct 4 23:53:10 2008 New Revision: 58597 Added: z3/deliverance/trunk/docs/_static/default.css z3/deliverance/trunk/docs/_templates/ z3/deliverance/trunk/docs/_templates/layout.html (contents, props changed) Modified: z3/deliverance/trunk/docs/license.txt z3/deliverance/trunk/docs/proxy.txt z3/deliverance/trunk/regen-docs Log: some updates to the doc look and feel Added: z3/deliverance/trunk/docs/_static/default.css ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/_static/default.css Sat Oct 4 23:53:10 2008 @@ -0,0 +1,853 @@ +/** + * Sphinx Doc Design + */ + +body { + font-family: sans-serif; + font-size: 100%; + background-color: #11303d; + color: #000; + margin: 0; + padding: 0; +} + +/* :::: LAYOUT :::: */ + +div.document { + background-color: #fff; +} + +div.documentwrapper { + float: left; + width: 100%; +} + +div.bodywrapper { + margin: 0 0 0 230px; +} + +div.body { + background-color: white; + padding: 0 20px 30px 20px; +} + +div.sphinxsidebarwrapper { + padding: 10px 5px 0 10px; +} + +div.sphinxsidebar { + float: left; + width: 230px; + margin-left: -100%; + font-size: 90%; + background-color: #D96F1D; + border-right: 1px solid #000; + border-bottom: 1px solid #000; + padding-bottom: 2em; +} + +div.clearer { + clear: both; +} + +div.footer { + color: #fff; + width: 100%; + padding: 9px 0 9px 0; + text-align: center; + font-size: 75%; +} + +div.footer a { + color: #fff; + text-decoration: underline; +} + +div.related { + background-color: #D96F1D; + color: #fff; + width: 100%; + height: 30px; + line-height: 30px; + font-size: 90%; +} + +div.related h3 { + display: none; +} + +div.related ul { + margin: 0; + padding: 0 0 0 10px; + list-style: none; +} + +div.related li { + display: inline; +} + +div.related li.right { + float: right; + margin-right: 5px; +} + +div.related a { + color: white; +} + +div.sphinxsidebar a { + color: #FFDABD; +} + +hr { + color: #944C14; + background-color: #944C14; + height: 2px; + border: 0; +} + +/* ::: TOC :::: */ +div.sphinxsidebar h3 { +/* FIXME: not sure about font */ + font-family: 'Trebuchet MS', sans-serif; + color: white; + font-size: 1.4em; + font-weight: normal; + margin: 0; + padding: 0; +} + +div.sphinxsidebar h4 { + font-family: 'Trebuchet MS', sans-serif; + color: white; + font-size: 1.3em; + font-weight: normal; + margin: 5px 0 0 0; + padding: 0; +} + +div.sphinxsidebar p { + color: white; +} + +div.sphinxsidebar p.topless { + margin: 5px 10px 10px 10px; +} + +div.sphinxsidebar ul { + margin: 10px; + padding: 0; + list-style: none; + color: white; +} + +div.sphinxsidebar ul ul, +div.sphinxsidebar ul.want-points { + margin-left: 20px; + list-style: square; +} + +div.sphinxsidebar ul ul { + margin-top: 0; + margin-bottom: 0; +} + +div.sphinxsidebar a { +/* FIXME: a color? */ +} + +div.sphinxsidebar form { + margin-top: 10px; +} + +div.sphinxsidebar input { + border: 1px solid #98dbcc; + font-family: sans-serif; + font-size: 1em; +} + +/* :::: MODULE CLOUD :::: */ +div.modulecloud { + margin: -5px 10px 5px 10px; + padding: 10px; + line-height: 160%; +/* FIXME: Not sure */ + border: 1px solid #cbe7e5; + background-color: #f2fbfd; +} + +div.modulecloud a { + padding: 0 5px 0 5px; +} + +/* :::: SEARCH :::: */ +ul.search { + margin: 10px 0 0 20px; + padding: 0; +} + +ul.search li { + padding: 5px 0 5px 20px; + background-image: url(file.png); + background-repeat: no-repeat; + background-position: 0 7px; +} + +ul.search li a { + font-weight: bold; +} + +ul.search li div.context { + color: #888; + margin: 2px 0 0 30px; + text-align: left; +} + +ul.keywordmatches li.goodmatch a { + font-weight: bold; +} + +/* :::: COMMON FORM STYLES :::: */ + +div.actions { + padding: 5px 10px 5px 10px; + border-top: 1px solid #cbe7e5; + border-bottom: 1px solid #cbe7e5; + background-color: #e0f6f4; +} + +form dl { + color: #333; +} + +form dt { + clear: both; + float: left; + min-width: 110px; + margin-right: 10px; + padding-top: 2px; +} + +input#homepage { + display: none; +} + +div.error { + margin: 5px 20px 0 0; + padding: 5px; + border: 1px solid #d00; + font-weight: bold; +} + +/* :::: INLINE COMMENTS :::: */ + +div.inlinecomments { + position: absolute; + right: 20px; +} + +div.inlinecomments a.bubble { + display: block; + float: right; + background-image: url(style/comment.png); + background-repeat: no-repeat; + width: 25px; + height: 25px; + text-align: center; + padding-top: 3px; + font-size: 0.9em; + line-height: 14px; + font-weight: bold; + color: black; +} + +div.inlinecomments a.bubble span { + display: none; +} + +div.inlinecomments a.emptybubble { + background-image: url(style/nocomment.png); +} + +div.inlinecomments a.bubble:hover { + background-image: url(style/hovercomment.png); + text-decoration: none; + color: #3ca0a4; +} + +div.inlinecomments div.comments { + float: right; + margin: 25px 5px 0 0; + max-width: 50em; + min-width: 30em; + border: 1px solid #2eabb0; + background-color: #f2fbfd; + z-index: 150; +} + +div#comments { + border: 1px solid #2eabb0; + margin-top: 20px; +} + +div#comments div.nocomments { + padding: 10px; + font-weight: bold; +} + +div.inlinecomments div.comments h3, +div#comments h3 { + margin: 0; + padding: 0; + background-color: #2eabb0; + color: white; + border: none; + padding: 3px; +} + +div.inlinecomments div.comments div.actions { + padding: 4px; + margin: 0; + border-top: none; +} + +div#comments div.comment { + margin: 10px; + border: 1px solid #2eabb0; +} + +div.inlinecomments div.comment h4, +div.commentwindow div.comment h4, +div#comments div.comment h4 { + margin: 10px 0 0 0; + background-color: #2eabb0; + color: white; + border: none; + padding: 1px 4px 1px 4px; +} + +div#comments div.comment h4 { + margin: 0; +} + +div#comments div.comment h4 a { + color: #d5f4f4; +} + +div.inlinecomments div.comment div.text, +div.commentwindow div.comment div.text, +div#comments div.comment div.text { + margin: -5px 0 -5px 0; + padding: 0 10px 0 10px; +} + +div.inlinecomments div.comment div.meta, +div.commentwindow div.comment div.meta, +div#comments div.comment div.meta { + text-align: right; + padding: 2px 10px 2px 0; + font-size: 95%; + color: #538893; + border-top: 1px solid #cbe7e5; + background-color: #e0f6f4; +} + +div.commentwindow { + position: absolute; + width: 500px; + border: 1px solid #cbe7e5; + background-color: #f2fbfd; + display: none; + z-index: 130; +} + +div.commentwindow h3 { + margin: 0; + background-color: #2eabb0; + color: white; + border: none; + padding: 5px; + font-size: 1.5em; + cursor: pointer; +} + +div.commentwindow div.actions { + margin: 10px -10px 0 -10px; + padding: 4px 10px 4px 10px; + color: #538893; +} + +div.commentwindow div.actions input { + border: 1px solid #2eabb0; + background-color: white; + color: #135355; + cursor: pointer; +} + +div.commentwindow div.form { + padding: 0 10px 0 10px; +} + +div.commentwindow div.form input, +div.commentwindow div.form textarea { + border: 1px solid #3c9ea2; + background-color: white; + color: black; +} + +div.commentwindow div.error { + margin: 10px 5px 10px 5px; + background-color: #fbe5dc; + display: none; +} + +div.commentwindow div.form textarea { + width: 99%; +} + +div.commentwindow div.preview { + margin: 10px 0 10px 0; + background-color: #70d0d4; + padding: 0 1px 1px 25px; +} + +div.commentwindow div.preview h4 { + margin: 0 0 -5px -20px; + padding: 4px 0 0 4px; + color: white; + font-size: 1.3em; +} + +div.commentwindow div.preview div.comment { + background-color: #f2fbfd; +} + +div.commentwindow div.preview div.comment h4 { + margin: 10px 0 0 0!important; + padding: 1px 4px 1px 4px!important; + font-size: 1.2em; +} + +/* :::: SUGGEST CHANGES :::: */ +div#suggest-changes-box input, div#suggest-changes-box textarea { + border: 1px solid #ccc; + background-color: white; + color: black; +} + +div#suggest-changes-box textarea { + width: 99%; + height: 400px; +} + + +/* :::: PREVIEW :::: */ +div.preview { + background-image: url(style/preview.png); + padding: 0 20px 20px 20px; + margin-bottom: 30px; +} + + +/* :::: INDEX PAGE :::: */ + +table.contentstable { + width: 90%; +} + +table.contentstable p.biglink { + line-height: 150%; +} + +a.biglink { + font-size: 1.3em; +} + +span.linkdescr { + font-style: italic; + padding-top: 5px; + font-size: 90%; +} + +/* :::: INDEX STYLES :::: */ + +table.indextable td { + text-align: left; + vertical-align: top; +} + +table.indextable dl, table.indextable dd { + margin-top: 0; + margin-bottom: 0; +} + +table.indextable tr.pcap { + height: 10px; +} + +table.indextable tr.cap { + margin-top: 10px; +/* FIXME: Not sure */ + background-color: #f2f2f2; +} + +img.toggler { + margin-right: 3px; + margin-top: 3px; + cursor: pointer; +} + +form.pfform { + margin: 10px 0 20px 0; +} + +/* :::: GLOBAL STYLES :::: */ + +.docwarning { + background-color: #ffe4e4; + padding: 10px; + margin: 0 -20px 0 -20px; + border-bottom: 1px solid #f66; +} + +p.subhead { + font-weight: bold; + margin-top: 20px; +} + +a { + color: #944C14; + text-decoration: none; +} + +a:hover { + text-decoration: underline; +} + +div.body h1, +div.body h2, +div.body h3, +div.body h4, +div.body h5, +div.body h6 { + font-family: 'Trebuchet MS', sans-serif; + background-color: #f2f2f2; + font-weight: normal; + color: #000; + border-bottom: 1px solid #ccc; + margin: 20px -20px 10px -20px; + padding: 3px 0 3px 10px; +} + +div.body h1 { margin-top: 0; font-size: 200%; } +div.body h2 { font-size: 160%; } +div.body h3 { font-size: 140%; } +div.body h4 { font-size: 120%; } +div.body h5 { font-size: 110%; } +div.body h6 { font-size: 100%; } + +a.headerlink { +/* FIXME: consider */ + color: #c60f0f; + font-size: 0.8em; + padding: 0 4px 0 4px; + text-decoration: none; + visibility: hidden; +} + +h1:hover > a.headerlink, +h2:hover > a.headerlink, +h3:hover > a.headerlink, +h4:hover > a.headerlink, +h5:hover > a.headerlink, +h6:hover > a.headerlink, +dt:hover > a.headerlink { + visibility: visible; +} + +a.headerlink:hover { + background-color: #c60f0f; + color: white; +} + +div.body p, div.body dd, div.body li { + text-align: justify; + line-height: 130%; +} + +div.body p.caption { + text-align: inherit; +} + +div.body td { + text-align: left; +} + +ul.fakelist { + list-style: none; + margin: 10px 0 10px 20px; + padding: 0; +} + +.field-list ul { + padding-left: 1em; +} + +.first { + margin-top: 0 !important; +} + +/* "Footnotes" heading */ +p.rubric { + margin-top: 30px; + font-weight: bold; +} + +/* "Topics" */ + +div.topic { + background-color: #eee; + border: 1px solid #ccc; + padding: 0 7px 0 7px; + margin: 10px 0 10px 0; +} + +p.topic-title { + font-size: 1.1em; + font-weight: bold; + margin-top: 10px; +} + +/* Admonitions */ + +div.admonition { + margin-top: 10px; + margin-bottom: 10px; + padding: 7px; +} + +div.admonition dt { + font-weight: bold; +} + +div.admonition dl { + margin-bottom: 0; +} + +div.admonition p { + display: inline; +} + +div.seealso { + background-color: #ffc; + border: 1px solid #ff6; +} + +div.warning { + background-color: #ffe4e4; + border: 1px solid #f66; +} + +div.note { + background-color: #eee; + border: 1px solid #ccc; +} + +p.admonition-title { + margin: 0px 10px 5px 0px; + font-weight: bold; + display: inline; +} + +p.admonition-title:after { + content: ":"; +} + +div.body p.centered { + text-align: center; + margin-top: 25px; +} + +table.docutils { + border: 0; +} + +table.docutils td, table.docutils th { + padding: 1px 8px 1px 0; + border-top: 0; + border-left: 0; + border-right: 0; + border-bottom: 1px solid #aaa; +} + +table.field-list td, table.field-list th { + border: 0 !important; +} + +table.footnote td, table.footnote th { + border: 0 !important; +} + +.field-list ul { + margin: 0; + padding-left: 1em; +} + +.field-list p { + margin: 0; +} + +dl { + margin-bottom: 15px; + clear: both; +} + +dd p { + margin-top: 0px; +} + +dd ul, dd table { + margin-bottom: 10px; +} + +dd { + margin-top: 3px; + margin-bottom: 10px; + margin-left: 30px; +} + +.refcount { + color: #060; +} + +dt:target, +.highlight { +/* FIXME: consider */ + background-color: #fbe54e; +} + +dl.glossary dt { + font-weight: bold; + font-size: 1.1em; +} + +th { + text-align: left; + padding-right: 5px; +} + +pre { + padding: 5px; + background-color: #efc; + color: #333; + border: 1px solid #ac9; + border-left: none; + border-right: none; + overflow: auto; +} + +td.linenos pre { + padding: 5px 0px; + border: 0; + background-color: transparent; + color: #aaa; +} + +table.highlighttable { + margin-left: 0.5em; +} + +table.highlighttable td { + padding: 0 0.5em 0 0.5em; +} + +tt { + background-color: #ecf0f3; + padding: 0 1px 0 1px; + font-size: 0.95em; +} + +tt.descname { + background-color: transparent; + font-weight: bold; + font-size: 1.2em; +} + +tt.descclassname { + background-color: transparent; +} + +tt.xref, a tt { + background-color: transparent; + font-weight: bold; +} + +.footnote:target { background-color: #ffa } + +h1 tt, h2 tt, h3 tt, h4 tt, h5 tt, h6 tt { + background-color: transparent; +} + +.optional { + font-size: 1.3em; +} + +.versionmodified { + font-style: italic; +} + +form.comment { + margin: 0; + padding: 10px 30px 10px 30px; + background-color: #eee; +} + +form.comment h3 { + background-color: #326591; + color: white; + margin: -10px -30px 10px -30px; + padding: 5px; + font-size: 1.4em; +} + +form.comment input, +form.comment textarea { + border: 1px solid #ccc; + padding: 2px; + font-family: sans-serif; + font-size: 100%; +} + +form.comment input[type="text"] { + width: 240px; +} + +form.comment textarea { + width: 100%; + height: 200px; + margin-bottom: 10px; +} + +.system-message { + background-color: #fda; + padding: 5px; + border: 3px solid red; +} + +/* :::: PRINT :::: */ + at media print { + div.document, + div.documentwrapper, + div.bodywrapper { + margin: 0; + width : 100%; + } + + div.sphinxsidebar, + div.related, + div.footer, + div#comments div.new-comment-box, + #top-link { + display: none; + } +} Added: z3/deliverance/trunk/docs/_templates/layout.html ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/_templates/layout.html Sat Oct 4 23:53:10 2008 @@ -0,0 +1,21 @@ +{% extends "!layout.html" %} + +{% block extrahead %} +{{ super() }} + +{% endblock %} + +{% block sidebarsearch %} + +
+ + + +{{ super() }} + +{% endblock %} Modified: z3/deliverance/trunk/docs/license.txt ============================================================================== --- z3/deliverance/trunk/docs/license.txt (original) +++ z3/deliverance/trunk/docs/license.txt Sat Oct 4 23:53:10 2008 @@ -1,3 +1,6 @@ +License +======= + Copyright (c) 2008 The Open Planning Project Permission is hereby granted, free of charge, to any person obtaining Modified: z3/deliverance/trunk/docs/proxy.txt ============================================================================== --- z3/deliverance/trunk/docs/proxy.txt (original) +++ z3/deliverance/trunk/docs/proxy.txt Sat Oct 4 23:53:10 2008 @@ -1,3 +1,6 @@ +About the Proxy +=============== + Proxying can be setup inside the rules as well: .. code-block:: xml Modified: z3/deliverance/trunk/regen-docs ============================================================================== --- z3/deliverance/trunk/regen-docs (original) +++ z3/deliverance/trunk/regen-docs Sat Oct 4 23:53:10 2008 @@ -1,8 +1,13 @@ #!/bin/sh -mkdir -p docs/_static docs/_build -sphinx-build -E -b html docs/ docs/_build || exit 1 if [ "$1" = "publish" ] ; then + PUBLISH=1 + shift; +fi + +mkdir -p docs/_static docs/_build +sphinx-build "$@" -E -b html docs/ docs/_build || exit 1 +if [ "$PUBLISH" = "1" ] ; then cd docs/_build echo "Uploading files..." tar czvf - . | ssh flow.openplans.org 'ssh acura.openplans.org "cd /www/deliverance.openplans.org/; tar xzvf -"' From ianb at codespeak.net Mon Oct 6 05:17:59 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 05:17:59 +0200 (CEST) Subject: [z3-checkins] r58603 - in z3/deliverance/trunk/deliverance/paster-templates: . etc etc/supervisor.d logs theme var Message-ID: <20081006031759.18B5F16A0F5@codespeak.net> Author: ianb Date: Mon Oct 6 05:17:58 2008 New Revision: 58603 Added: z3/deliverance/trunk/deliverance/paster-templates/ z3/deliverance/trunk/deliverance/paster-templates/etc/ z3/deliverance/trunk/deliverance/paster-templates/etc/deliv-users.htpasswd z3/deliverance/trunk/deliverance/paster-templates/etc/deliverance.xml z3/deliverance/trunk/deliverance/paster-templates/etc/supervisor.d/ z3/deliverance/trunk/deliverance/paster-templates/etc/supervisor.d/deliverance.conf z3/deliverance/trunk/deliverance/paster-templates/etc/supervisord.conf z3/deliverance/trunk/deliverance/paster-templates/logs/ z3/deliverance/trunk/deliverance/paster-templates/theme/ z3/deliverance/trunk/deliverance/paster-templates/var/ Log: some template files (before moving around) Added: z3/deliverance/trunk/deliverance/paster-templates/etc/deliv-users.htpasswd ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster-templates/etc/deliv-users.htpasswd Mon Oct 6 05:17:58 2008 @@ -0,0 +1,8 @@ +# This file is read by for users that can access the logs with ?deliv_log +# and login via /.deliverance/login +# You may add new entries using the Apache htpasswd program. +{{py: +from devauth.htpasswd import apache_md5crypt +}} +admin: {{apache_md5crypt(password, 'admin')}} + Added: z3/deliverance/trunk/deliverance/paster-templates/etc/deliverance.xml ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster-templates/etc/deliverance.xml Mon Oct 6 05:17:58 2008 @@ -0,0 +1,49 @@ +{{py: +from paste.script.util.converters import asbool +}} + + + + {{host}} + true + 127.0.0.1 + deliv-users.htpasswd + + + + + + + + + + +{{if asbool(sub_rewrite_links):}} + +{{endif}} + + + + + + + + + + + + + Added: z3/deliverance/trunk/deliverance/paster-templates/etc/supervisor.d/deliverance.conf ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster-templates/etc/supervisor.d/deliverance.conf Mon Oct 6 05:17:58 2008 @@ -0,0 +1,2 @@ +[program:deliverance] +command = ../bin/deliverance-proxy ./deliverance.xml Added: z3/deliverance/trunk/deliverance/paster-templates/etc/supervisord.conf ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster-templates/etc/supervisord.conf Mon Oct 6 05:17:58 2008 @@ -0,0 +1,50 @@ +# This is for use with supervisord (http://supervisord.org) +# This is a daemon manager that you may find useful to use with +# Deliverance and other server processes in your stack. +# Put files for individual scripts/servers in supervisor.d + +[include] +files = ./supervisor.d/*.ini + +[unix_http_server] +file=../var/supervisor/supervisor.sock ; (the path to the socket file) +;chmod=0700 ; sockef file mode (default 0700) +;chown=nobody:nogroup ; socket file uid:gid owner +;username=user ; (default is no username (open server)) +;password=123 ; (default is no password (open server)) + +[inet_http_server] ; inet (TCP) server disabled by default +;port=127.0.0.1:PORT ; (ip_address:port specifier, *:port for all iface) +;username=user ; (default is no username (open server)) +;password=123 ; (default is no password (open server)) + +[supervisord] +logfile=../logs/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log) +logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB) +logfile_backups=10 ; (num of main logfile rotation backups;default 10) +loglevel=info ; (log level;default info; others: debug,warn,trace) +pidfile=../var/supervisor/supervisord.pid ; (supervisord pidfile;default supervisord.pid) +nodaemon=false ; (start in foreground if true;default false) +minfds=1024 ; (min. avail startup file descriptors;default 1024) +minprocs=200 ; (min. avail process descriptors;default 200) +;umask=022 ; (process file creation umask;default 022) +;user=USER ; (default is current user, required if root) +;identifier=supervisor ; (supervisord identifier, default is 'supervisor') +;nocleanup=true ; (don't clean up tempfiles at start;default false) +;childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP) +;environment=KEY=value ; (key value pairs to add to environment) +;strip_ansi=false ; (strip ansi escape codes in logs; def. false) + +; the below section must remain in the config file for RPC +; (supervisorctl/web interface) to work, additional interfaces may be +; added by defining them in separate rpcinterface: sections +[rpcinterface:supervisor] +supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface + +[supervisorctl] +serverurl=../var/supervisor/supervisor.sock ; use a unix:// URL for a unix socket +;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket +;username=chris ; should be same as http_username if set +;password=123 ; should be same as http_password if set +;prompt=mysupervisor ; cmd line prompt (default "supervisor") + From ianb at codespeak.net Mon Oct 6 05:18:29 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 05:18:29 +0200 (CEST) Subject: [z3-checkins] r58604 - in z3/deliverance/trunk/deliverance/paster-templates: deliverance deliverance/etc deliverance/logs deliverance/theme deliverance/var etc logs theme var Message-ID: <20081006031829.89EA416A0F5@codespeak.net> Author: ianb Date: Mon Oct 6 05:18:28 2008 New Revision: 58604 Added: z3/deliverance/trunk/deliverance/paster-templates/deliverance/ z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/ (props changed) - copied from r58603, z3/deliverance/trunk/deliverance/paster-templates/etc/ z3/deliverance/trunk/deliverance/paster-templates/deliverance/logs/ (props changed) - copied from r58603, z3/deliverance/trunk/deliverance/paster-templates/logs/ z3/deliverance/trunk/deliverance/paster-templates/deliverance/theme/ (props changed) - copied from r58603, z3/deliverance/trunk/deliverance/paster-templates/theme/ z3/deliverance/trunk/deliverance/paster-templates/deliverance/var/ (props changed) - copied from r58603, z3/deliverance/trunk/deliverance/paster-templates/var/ Removed: z3/deliverance/trunk/deliverance/paster-templates/etc/ z3/deliverance/trunk/deliverance/paster-templates/logs/ z3/deliverance/trunk/deliverance/paster-templates/theme/ z3/deliverance/trunk/deliverance/paster-templates/var/ Log: move into subdir From ianb at codespeak.net Mon Oct 6 05:35:21 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 05:35:21 +0200 (CEST) Subject: [z3-checkins] r58605 - in z3/deliverance/trunk/deliverance/paster-templates: deliverance/etc deliverance/etc/supervisor.d plone Message-ID: <20081006033521.45C32169F22@codespeak.net> Author: ianb Date: Mon Oct 6 05:35:20 2008 New Revision: 58605 Added: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd_tmpl (props changed) - copied unchanged from r58604, z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl (props changed) - copied unchanged from r58604, z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf_tmpl (props changed) - copied unchanged from r58604, z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisord.conf_tmpl (props changed) - copied unchanged from r58604, z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisord.conf z3/deliverance/trunk/deliverance/paster-templates/plone/ Removed: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisord.conf Log: templatize files Deleted: /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd ============================================================================== --- /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd Mon Oct 6 05:35:20 2008 +++ (empty file) @@ -1,8 +0,0 @@ -# This file is read by for users that can access the logs with ?deliv_log -# and login via /.deliverance/login -# You may add new entries using the Apache htpasswd program. -{{py: -from devauth.htpasswd import apache_md5crypt -}} -admin: {{apache_md5crypt(password, 'admin')}} - Deleted: /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml ============================================================================== --- /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml Mon Oct 6 05:35:20 2008 +++ (empty file) @@ -1,49 +0,0 @@ -{{py: -from paste.script.util.converters import asbool -}} - - - - {{host}} - true - 127.0.0.1 - deliv-users.htpasswd - - - - - - - - - - -{{if asbool(sub_rewrite_links):}} - -{{endif}} - - - - - - - - - - - - - Deleted: /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf ============================================================================== --- /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf Mon Oct 6 05:35:20 2008 +++ (empty file) @@ -1,2 +0,0 @@ -[program:deliverance] -command = ../bin/deliverance-proxy ./deliverance.xml Deleted: /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisord.conf ============================================================================== --- /z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisord.conf Mon Oct 6 05:35:20 2008 +++ (empty file) @@ -1,50 +0,0 @@ -# This is for use with supervisord (http://supervisord.org) -# This is a daemon manager that you may find useful to use with -# Deliverance and other server processes in your stack. -# Put files for individual scripts/servers in supervisor.d - -[include] -files = ./supervisor.d/*.ini - -[unix_http_server] -file=../var/supervisor/supervisor.sock ; (the path to the socket file) -;chmod=0700 ; sockef file mode (default 0700) -;chown=nobody:nogroup ; socket file uid:gid owner -;username=user ; (default is no username (open server)) -;password=123 ; (default is no password (open server)) - -[inet_http_server] ; inet (TCP) server disabled by default -;port=127.0.0.1:PORT ; (ip_address:port specifier, *:port for all iface) -;username=user ; (default is no username (open server)) -;password=123 ; (default is no password (open server)) - -[supervisord] -logfile=../logs/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log) -logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB) -logfile_backups=10 ; (num of main logfile rotation backups;default 10) -loglevel=info ; (log level;default info; others: debug,warn,trace) -pidfile=../var/supervisor/supervisord.pid ; (supervisord pidfile;default supervisord.pid) -nodaemon=false ; (start in foreground if true;default false) -minfds=1024 ; (min. avail startup file descriptors;default 1024) -minprocs=200 ; (min. avail process descriptors;default 200) -;umask=022 ; (process file creation umask;default 022) -;user=USER ; (default is current user, required if root) -;identifier=supervisor ; (supervisord identifier, default is 'supervisor') -;nocleanup=true ; (don't clean up tempfiles at start;default false) -;childlogdir=/tmp ; ('AUTO' child log dir, default $TEMP) -;environment=KEY=value ; (key value pairs to add to environment) -;strip_ansi=false ; (strip ansi escape codes in logs; def. false) - -; the below section must remain in the config file for RPC -; (supervisorctl/web interface) to work, additional interfaces may be -; added by defining them in separate rpcinterface: sections -[rpcinterface:supervisor] -supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface - -[supervisorctl] -serverurl=../var/supervisor/supervisor.sock ; use a unix:// URL for a unix socket -;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket -;username=chris ; should be same as http_username if set -;password=123 ; should be same as http_password if set -;prompt=mysupervisor ; cmd line prompt (default "supervisor") - From ianb at codespeak.net Mon Oct 6 06:28:16 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 06:28:16 +0200 (CEST) Subject: [z3-checkins] r58606 - in z3/deliverance/trunk: . deliverance deliverance/paster-templates/deliverance/etc deliverance/paster-templates/plone/etc Message-ID: <20081006042816.936DF169F24@codespeak.net> Author: ianb Date: Mon Oct 6 06:28:14 2008 New Revision: 58606 Added: z3/deliverance/trunk/deliverance/paster-templates/plone/etc/ z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl z3/deliverance/trunk/deliverance/paster_templates.py (contents, props changed) Modified: z3/deliverance/trunk/build-bundle z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd_tmpl z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl z3/deliverance/trunk/deliverance/proxy.py z3/deliverance/trunk/setup.py Log: Added some paster create templates Modified: z3/deliverance/trunk/build-bundle ============================================================================== --- z3/deliverance/trunk/build-bundle (original) +++ z3/deliverance/trunk/build-bundle Mon Oct 6 06:28:14 2008 @@ -20,6 +20,7 @@ pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz \ -e svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=Deliverance \ + supervisord PasteScript || fail 'Bundle failed' echo 'INSTALLING' Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliv-users.htpasswd_tmpl Mon Oct 6 06:28:14 2008 @@ -4,5 +4,5 @@ {{py: from devauth.htpasswd import apache_md5crypt }} -admin: {{apache_md5crypt(password, 'admin')}} +admin:{{apache_md5crypt(password, 'admin')}} Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl Mon Oct 6 06:28:14 2008 @@ -1,5 +1,5 @@ {{py: -from paste.script.util.converters import asbool +from deliverance.util.converters import asbool }} @@ -42,8 +42,13 @@ + + + Added: z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl Mon Oct 6 06:28:14 2008 @@ -0,0 +1,54 @@ +{{py: +from deliverance.util.converters import asbool +}} + + + + {{host}} + true + 127.0.0.1 + deliv-users.htpasswd + + + + + + + + + + +{{if asbool(sub_rewrite_links):}} + +{{endif}} + + + + + + + + + + + + + + Added: z3/deliverance/trunk/deliverance/paster_templates.py ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/paster_templates.py Mon Oct 6 06:28:14 2008 @@ -0,0 +1,104 @@ +from paste.script.templates import Template, var +from tempita import paste_script_template_renderer +import os +import posixpath +import urllib2 +from lxml.html import parse, tostring +import mimetypes + +class DeliveranceTemplate(Template): + _template_dir = 'paster-templates/deliverance' + template_renderer = staticmethod(paste_script_template_renderer) + summary = "Basic template for a deliverance-proxy setup" + vars = [ + var('host', 'The host/port to serve on', + 'localhost:8000'), + var('sub_host', 'The main host to connect to', + default='localhost:8080'), + var('sub_rewrite_links', 'Rewrite links from sub_host?', + default='n'), + var('password', 'The password for the deliverance admin console'), + var('theme_url', 'A URL to pull the initial theme from (optional)'), + ] + + def post(self, command, output_dir, vars): + theme_url = vars['theme_url'] + if not theme_url: + content = DEFAULT_THEME_CONTENT + files = DEFAULT_FILES + else: + content, files = self.get_content(theme_url) + command.ensure_file(os.path.join(output_dir, 'theme/theme.html'), content) + for filename, content in files: + command.ensure_file(os.path.join(output_dir, 'theme', filename), content) + + def get_content(self, url): + """Gets the content and all embedded content (images, CSS, etc)""" + page = parse(urllib2.urlopen(url)).getroot() + page.make_links_absolute() + files = [] + for element, attr, link, pos in page.iterlinks(): + if not self._embedded_link(element): + continue + filename, content = self.get_embedded(link) + files.append((filename, content)) + if attr is None: + old_value = element.text + else: + old_value = unicode(element.attrib[attr]) + new_value = old_value[:pos] + filename + old_value[pos+len(link):] + if attr is None: + element.text = new_value + else: + element.attrib[attr] = new_value + return tostring(page), files + + def _embedded_link(self, element): + """True if the element links to something embedded""" + if element.tag in ('script', 'img', 'style'): + return True + if element.tag == 'link' and element.attrib.get('rel', '').lower() == 'stylesheet': + return True + return False + + def get_embedded(self, url): + resp = urllib2.urlopen(url) + url = resp.geturl() + content = resp.read() + content_type = resp.info()['content-type'] + filename = posixpath.basename(url).split('?')[0] + filename, orig_ext = posixpath.splitext(filename) + if not filename: + filename = 'embedded' + ext = mimetypes.guess_extension(content_type) + if ext == '.jpeg': + ext = '.jpg' + ext = ext or orig_ext + return filename + ext, content + +DEFAULT_THEME_CONTENT = '''\ + + + + + + +
+ content that will be replaced. +
+ + +''' + +DEFAULT_FILES = [ + ('style.css', '/* put your styles here */'), + ] + +class PloneTemplate(Template): + _template_dir = 'paster-templates/plone' + required_templates = ['deliverance'] + template_renderer = staticmethod(paste_script_template_renderer) + summary = 'Plone-specific template for deliverance-proxy' + vars = [ + var('site_name', "The name of your Plone site (no /'s)"), + ] Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Mon Oct 6 06:28:14 2008 @@ -614,7 +614,7 @@ elif child.tag == 'dev-deny': dev_deny_ips.extend(cls.substitute(child.text, environ).split()) elif child.tag == 'dev-htpasswd': - dev_htpasswd = cls.substitute(child.text, environ) + dev_htpasswd = os.path.join(os.path.dirname(url_to_filename(source_location)), cls.substitute(child.text, environ)) elif child.tag == 'dev-expiration': dev_expiration = cls.substitute(child.text, environ) if dev_expiration: @@ -642,11 +642,15 @@ raise DeliveranceSyntaxError( "You can use or , but not both", element=el) + if not dev_users and not dev_htpasswd: + ## FIXME: not sure this is the best way to warn + print 'Warning: no or ; logging is inaccessible' ## FIXME: add a default allow_ips of 127.0.0.1? return cls(server_host, execute_pyref=execute_pyref, display_local_files=display_local_files, dev_allow_ips=dev_allow_ips, dev_deny_ips=dev_deny_ips, - dev_users=dev_users, dev_expiration=dev_expiration, + dev_users=dev_users, dev_htpasswd=dev_htpasswd, + dev_expiration=dev_expiration, source_location=source_location) @classmethod @@ -698,6 +702,9 @@ password_checker = None app = SecurityContext.middleware(app, execute_pyref=self.execute_pyref, display_local_files=self.display_local_files) + if password_checker is None and not self.dev_htpasswd: + print 'disabled: %r and %r' % (self.dev_users, self.dev_htpasswd) + return app app = DevAuth( app, allow=convert_ip_mask(self.dev_allow_ips), Modified: z3/deliverance/trunk/setup.py ============================================================================== --- z3/deliverance/trunk/setup.py (original) +++ z3/deliverance/trunk/setup.py Mon Oct 6 06:28:14 2008 @@ -45,5 +45,9 @@ entry_points=""" [console_scripts] deliverance-proxy = deliverance.proxycommand:main + + [paste.paster_create_template] + deliverance = deliverance.paster_templates:DeliveranceTemplate + deliverance_plone = deliverance.paster_templates:PloneTemplate """, ) From ianb at codespeak.net Mon Oct 6 06:29:05 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 06:29:05 +0200 (CEST) Subject: [z3-checkins] r58607 - z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d Message-ID: <20081006042905.A43EE169F56@codespeak.net> Author: ianb Date: Mon Oct 6 06:29:05 2008 New Revision: 58607 Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf_tmpl Log: don't use an absolute path for deliverance-proxy Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/supervisor.d/deliverance.conf_tmpl Mon Oct 6 06:29:05 2008 @@ -1,2 +1,2 @@ [program:deliverance] -command = ../bin/deliverance-proxy ./deliverance.xml +command = deliverance-proxy ./deliverance.xml From ianb at codespeak.net Mon Oct 6 06:50:55 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 06:50:55 +0200 (CEST) Subject: [z3-checkins] r58608 - z3/deliverance/trunk Message-ID: <20081006045055.AD309169FD9@codespeak.net> Author: ianb Date: Mon Oct 6 06:50:54 2008 New Revision: 58608 Modified: z3/deliverance/trunk/build-bundle Log: Add uploading to build-bundle script; install supervisor properly Modified: z3/deliverance/trunk/build-bundle ============================================================================== --- z3/deliverance/trunk/build-bundle (original) +++ z3/deliverance/trunk/build-bundle Mon Oct 6 06:50:54 2008 @@ -5,6 +5,17 @@ exit 1 } +if [ "$1" = "upload" ] ; then + shift + FILENAME="$1" + if [ -z "$FILENAME" ] ; then + echo "Usage: `basename $0` uplaod FILENAME" + exit 2 + fi + cat $FILENAME | ssh flow.openplans.org "ssh acura.openplans.org \"cd /www/deliverance.openplans.org/dist/ ; cat > $FILENAME ; rm Deliverance-snapshot-latest.pybundle ; ln -s $FILENAME Deliverance-snapshot-latest.pybundle\"" + exit +fi + REV=$(python -c 'import pyinstall; print pyinstall.get_svn_revision(".")') FILE="Deliverance-snapshot-r${REV}.pybundle" @@ -20,7 +31,7 @@ pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz \ -e svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=Deliverance \ - supervisord PasteScript + supervisor PasteScript \ || fail 'Bundle failed' echo 'INSTALLING' From ianb at codespeak.net Mon Oct 6 07:49:41 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 07:49:41 +0200 (CEST) Subject: [z3-checkins] r58609 - in z3/deliverance/trunk: deliverance deliverance/paster-templates/deliverance/etc deliverance/paster-templates/plone/etc docs Message-ID: <20081006054941.B6892169F69@codespeak.net> Author: ianb Date: Mon Oct 6 07:49:39 2008 New Revision: 58609 Added: z3/deliverance/trunk/docs/quickstart.txt (contents, props changed) Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl z3/deliverance/trunk/deliverance/paster_templates.py z3/deliverance/trunk/docs/index.txt Log: Added a quickstart document; renamed some paster create variables Modified: z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/deliverance/etc/deliverance.xml_tmpl Mon Oct 6 07:49:39 2008 @@ -21,8 +21,8 @@ --> - -{{if asbool(sub_rewrite_links):}} + +{{if asbool(proxy_rewrite_links):}} {{endif}} Modified: z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl Mon Oct 6 07:49:39 2008 @@ -21,8 +21,8 @@ --> - -{{if asbool(sub_rewrite_links):}} + +{{if asbool(proxy_rewrite_links):}} {{endif}} Modified: z3/deliverance/trunk/deliverance/paster_templates.py ============================================================================== --- z3/deliverance/trunk/deliverance/paster_templates.py (original) +++ z3/deliverance/trunk/deliverance/paster_templates.py Mon Oct 6 07:49:39 2008 @@ -13,9 +13,9 @@ vars = [ var('host', 'The host/port to serve on', 'localhost:8000'), - var('sub_host', 'The main host to connect to', - default='localhost:8080'), - var('sub_rewrite_links', 'Rewrite links from sub_host?', + var('proxy_url', 'The main site to connect/proxy to', + default='http://localhost:8080'), + var('proxy_rewrite_links', 'Rewrite links from sub_host?', default='n'), var('password', 'The password for the deliverance admin console'), var('theme_url', 'A URL to pull the initial theme from (optional)'), Modified: z3/deliverance/trunk/docs/index.txt ============================================================================== --- z3/deliverance/trunk/docs/index.txt (original) +++ z3/deliverance/trunk/docs/index.txt Mon Oct 6 07:49:39 2008 @@ -5,6 +5,7 @@ .. toctree:: + quickstart philosophy configuration pyref Added: z3/deliverance/trunk/docs/quickstart.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/quickstart.txt Mon Oct 6 07:49:39 2008 @@ -0,0 +1,83 @@ +Quickstart +========== + +This document will show you how to get started playing around with Deliverance quickly. This quickstart is written for a Linux, Mac, or BSD-using audience. Sorry Windows users. + +Starting with virtualenv +------------------------ + +If you are familiar with `virtualenv `_ and `easy_install `_ you can skip this section. + +We'll be setting everything up in an isolated environment. Nothing in this will effect anything on your system outside of the directories we set up in this tutorial -- so you can just delete the directory and forget about the whole thing if you don't like it. + +The first thing we'll do is get virtualenv and create an environment. Grab `virtualenv.py `_, and run:: + + $ python virtualenv.py DelivTest + +This will create an environment in ``DelivTest/`` and install ``easy_install``. There's also a new Python interpreter in ``DelivTest/bin/python`` -- anything in ``DelivTest/bin/`` will be tied to this environment. It'll use libraries from the environment and install libraries into the environment. + +Note you can run ``source DelivTest/bin/activate`` to change ``$PATH`` so that everytime you run ``python``, ``easy_install``, etc., you'll be running it from the environment you've created. + +Installing the Software +----------------------- + +We won't actually use ``easy_install`` much, we'll just install another installer:: + + $ DelivTest/bin/easy_install pyinstall + +And we'll use the installer to install Deliverance. This has the benefit over ``easy_install`` of installing all the source at once and with versions of the software that are known to work with each other. So, we'll install the Deliverance bundle:: + + $ DelivTest/bin/pyinstall.py http://deliverance.openplans.org/dist/Deliverance-snapshot-latest.pybundle + +To install lxml, you have to have ``libxml2`` installed, and the ``-dev`` packages for ``libxml2`` and Python itself. On Ubuntu this is ``libxml2-dev``, ``libxslt1-dev``, and ``python2.5-dev``. + +This can take a long time to crunch. + +Creating a Configuration +------------------------ + +We have the software installed, but not the configuration to run it. To create the configuration run:: + + $ DelivTest/bin/paster create -t deliverance DelivTest + Selected and implied templates: + deliverance#deliverance Basic template for a deliverance-proxy setup + + Variables: + egg: TestEnv + package: testenv + project: TestEnv + Enter host (The host/port to serve on) ['localhost:8000']: + Enter proxy_url (The main site to connect/proxy to) ['http://localhost:8080']: + Enter proxy_rewrite_links (Rewrite links from sub_host?) ['n']: + Enter password (The password for the deliverance admin console) ['']: test + Enter theme_url (A URL to pull the initial theme from (optional)) ['']: http://mysite.com + Creating template deliverance + ... + +It will ask you about some questions: + +``host``: + The host that Deliverance will serve from. Note ``localhost`` (or 127.0.0.1) means that you can only connect from the machine itself. If you want it to be externally visible use 0.0.0.0. +``proxy_host``: + This is the location where all requests will go to. ``http://localhost:8080`` is a common default for servers. You can also give a remote host and a path, like ``http://mysite.com/blog`` +``proxy_rewrite_links``: + If you are proxying to a site that doesn't really expect you to be proxying to it, the links will probably be broken. You can give Y here to turn on link rewriting. It's not 100% perfect (e.g., links put into Javascript), but it can be good for experimenting. +``password``: + The password to access the logging console. The username is always ``admin``. You can add more later too. +``theme_url``: + If you want to base your theme on an existing page, you can give the URL of that page here. It will fetch that page and all the CSS and images from that page, so you can locally edit them. Otherwise an extremely simply theme will be setup. + +Once you've entered these values, it will set up a basic layout with a file ``etc/deliverance.xml`` for the configuration, and the theme in ``theme/theme.html``. + +You can start the server with:: + + $ ./bin/deliverance-proxy ./etc/deliverance.xml + +The site will be at ``http://localhost:8000`` and you can login at ``http://localhost:8000/.deliverance/login`` + +Once you have logged in you can look at ``http://localhost:8000/?deliv_log`` to see a log of everything Deliverance is doing (at the bottom of the page). + +Editing the Rules +----------------- + +Here's where the quickstart ends for now; you'll have to read the rest of the documentation to understand the rules. From ianb at codespeak.net Mon Oct 6 21:14:32 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 21:14:32 +0200 (CEST) Subject: [z3-checkins] r58687 - z3/deliverance/trunk/deliverance Message-ID: <20081006191432.0EDC6169EA0@codespeak.net> Author: ianb Date: Mon Oct 6 21:14:30 2008 New Revision: 58687 Modified: z3/deliverance/trunk/deliverance/ruleset.py Log: X-Deliverance-Page-Class should be allowed in the headers Modified: z3/deliverance/trunk/deliverance/ruleset.py ============================================================================== --- z3/deliverance/trunk/deliverance/ruleset.py (original) +++ z3/deliverance/trunk/deliverance/ruleset.py Mon Oct 6 21:14:30 2008 @@ -38,7 +38,7 @@ classes = run_matches(self.matchers, req, resp, response_headers, log) except AbortTheme: return resp - if 'X-Deliverance-Page-Class' in resp.headers: + if 'X-Deliverance-Page-Class' in response_headers: classes.extend(resp.headers['X-Deliverance-Page-Class'].strip().split()) if 'deliverance.page_classes' in req.environ: classes.extend(req.environ['deliverance.page_classes']) From ianb at codespeak.net Mon Oct 6 21:14:48 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 21:14:48 +0200 (CEST) Subject: [z3-checkins] r58688 - z3/deliverance/trunk/deliverance Message-ID: <20081006191448.7ADCA169F03@codespeak.net> Author: ianb Date: Mon Oct 6 21:14:47 2008 New Revision: 58688 Modified: z3/deliverance/trunk/deliverance/pagematch.py z3/deliverance/trunk/deliverance/proxy.py Log: little cleanups Modified: z3/deliverance/trunk/deliverance/pagematch.py ============================================================================== --- z3/deliverance/trunk/deliverance/pagematch.py (original) +++ z3/deliverance/trunk/deliverance/pagematch.py Mon Oct 6 21:14:47 2008 @@ -8,8 +8,6 @@ from deliverance.pyref import PyReference from deliverance.security import execute_pyref -__all__ = ['MatchSyntaxError', 'Match'] - class AbstractMatch(object): """ Represents the tags. Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Mon Oct 6 21:14:47 2008 @@ -703,7 +703,7 @@ app = SecurityContext.middleware(app, execute_pyref=self.execute_pyref, display_local_files=self.display_local_files) if password_checker is None and not self.dev_htpasswd: - print 'disabled: %r and %r' % (self.dev_users, self.dev_htpasswd) + ## FIXME: warn here? return app app = DevAuth( app, From ianb at codespeak.net Mon Oct 6 21:15:15 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Mon, 6 Oct 2008 21:15:15 +0200 (CEST) Subject: [z3-checkins] r58689 - in z3/deliverance/trunk/docs: . modules Message-ID: <20081006191515.6B34D169F1B@codespeak.net> Author: ianb Date: Mon Oct 6 21:15:14 2008 New Revision: 58689 Added: z3/deliverance/trunk/docs/code-map.txt (contents, props changed) z3/deliverance/trunk/docs/modules/ z3/deliverance/trunk/docs/modules/exceptions.txt (contents, props changed) z3/deliverance/trunk/docs/modules/log.txt (contents, props changed) z3/deliverance/trunk/docs/modules/middleware.txt (contents, props changed) z3/deliverance/trunk/docs/modules/pagematch.txt (contents, props changed) z3/deliverance/trunk/docs/modules/proxy.txt (contents, props changed) z3/deliverance/trunk/docs/modules/proxycommand.txt (contents, props changed) z3/deliverance/trunk/docs/modules/pyref.txt (contents, props changed) z3/deliverance/trunk/docs/modules/rules.txt (contents, props changed) z3/deliverance/trunk/docs/modules/ruleset.txt (contents, props changed) z3/deliverance/trunk/docs/modules/security.txt (contents, props changed) z3/deliverance/trunk/docs/modules/selector.txt (contents, props changed) z3/deliverance/trunk/docs/modules/stringmatch.txt (contents, props changed) z3/deliverance/trunk/docs/modules/themeref.txt (contents, props changed) z3/deliverance/trunk/docs/modules/util.txt (contents, props changed) Modified: z3/deliverance/trunk/docs/conf.py z3/deliverance/trunk/docs/index.txt Log: Added in a code map, and extracted code documentation Added: z3/deliverance/trunk/docs/code-map.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/code-map.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,147 @@ +Map of the Deliverance Code +=========================== + +This document will help you navigate your way around the Deliverance code base. You don't need to know this level of detail if you are just *using* Deliverance -- you'll only need this information if you want to contribute to Deliverance development, or integrate Deliverance in another product. + +The Code +-------- + +Here's the code: + +.. toctree:: + + modules/exceptions + modules/log + modules/middleware + modules/pagematch + modules/proxycommand + modules/proxy + modules/pyref + modules/ruleset + modules/rules + modules/security + modules/selector + modules/stringmatch + modules/themeref + modules/util + +Startup Path with deliverance-proxy +----------------------------------- + +This describes what happens when you run ``deliverance-proxy``. + +1. ``deliverance-proxy`` calls :func:`deliverance.proxycommand.main()` + +2. ``main()`` parses the options and does some minor validation, then calls :func:`deliverance.proxycommand.run_command()` + +3. The configuration file is parsed for the server and proxy settings by :class:`deliverance.proxy.ProxySettings` + +4. The rule configuration is loaded by a wrapper class :class:`deliverance.proxycommand.ReloadingApp`. This is a class that wrap :class:`deliverance.proxy.ProxySet` and applies the middleware from `ProxySettings`. It checks the modification time of the configuration file on each request in case you edit the file. + +5. :class:`deliverance.proxy.ProxySet` represents all the ```` elements, which define how requests are mapped to remote hosts. + +6. :class:`deliverance.proxy.ProxySettings` represents ````, which defines things like the port to serve on, and developer console login methods. + +7. The actual WSGI application is :meth:`deliverance.proxy.ProxySet.application` and is wrapped by the authentication built by :meth:`deliverance.proxy.ProxySettings.middleware`. The authentication is implemented with :class:`devauth.DevAuth`. + +8. `run_command` applies an interactive debugger if you gave the option ``--interactive-debugger``. This is to debug problems with Deliverance itself. Alternately if you just gave ``--debug`` it applies a non-interactive debugger. (The interactive debugger allows arbitrary code execution.) + +9. `run_command` also applies :class:`wsgifilter.proxyapp.DebugHeaders` if you give ``--debug-headers``. This prints out incoming and outgoing headers, and if you provide ``--debug-headers --debug-headers`` it will show request and response bodies as well. + +10. Finally the server is started with the constructed application. It uses :mod:`paste.httpserver`, a threaded server. + +Request Path +------------ + +This describes what happens when a request comes in. + +1. We'll ignoring some of the debugging middleware applied by `run_command`, though it does get entered first. + +2. The request goes to :class:`devauth.DevAuth`, which checks IP addresses and cookies to see if you've logged in via ``/.deliverance/login``. If you are logged in it sets ``environ['x-wsgiorg.developer_user']`` + +3. The request then goes through :meth:`deliverance.security.SecurityContext.middleware` which instantiates an instance of :class:`deliverance.security.SecurityContext` and puts it in ``environ['deliverance.security_context']``. This security context is used later to allow or disallow ``pyref``, viewing files, and whether ``?deliv_log`` will show the developer console. + +4. Now we enter :meth:`deliverance.proxy.ProxySet.application`. This sets up :class:`deliverance.log.SavingLogger` which accepts log messages so we can display them later in the request. Log messages are all per-request. We then pass the request on to an instance of :class:`deliverance.middleware.DeliveranceMiddleware` -- it is instantiated with a callback into :meth:`deliverance.proxy.ProxySet.proxy_app` and a callback to get the rules. + +5. Deliverance really starts doing something in :meth:`deliverance.middleware.DeliveranceMiddleware.__call__`. Here it checks if the request should be unthemed (because of ``?deliv_notheme``). It also dispatches ``/.deliverance`` to :meth:`deliverance.middleware.DeliveranceMiddleware.internal_app` (which itself is fairly simple, and just implements some stuff like viewing files, logging in, etc). + +6. The request is passed on to ``self.app``, which :meth:`deliverance.proxy.ProxySet.proxy_app`. This goes through all the instances of :class:`deliverance.proxy.Proxy` (each of which represent one ````) and tries to match them against the request. If none matches it returns a 404. + +7. Assuming something matches, the request goes to :meth:`deliverance.proxy.Proxy.forward_request`. This fixes up the path based on the rules (possibly stripping some of the leading text). It applies any ```` modifications, forwards the request, and then applies any ```` modifications, including link rewriting. + +8. The request is forwarded to :meth:`deliverance.proxy.Proxy.proxy_to_dest`. If it sees the destination is a ``file:///`` URL then it passes the request to :meth:`deliverance.proxy.Proxy.proxy_to_file` (which itself is not very interesting). The method sets up some headers: ``X-Forwarded-For``, ``X-Forwarded-Scheme``, ``X-Forwarded-Server``, and ``X-Forwarded-Path``. It then actually forwards the request via :func:`wsgiproxy.exactproxy.proxy_exact_request`. + +9. The response is now complete, and we are back in :meth:`deliverance.middleware.DeliveranceMiddleware.__call__`. If the response wasn't of Content-Type text/html, then the request is passed back without any modification. + +10. The rule set is retrieved from `Proxy` -- an instance of :class:`deliverance.ruleset.RuleSet` (that was parsed directly from the configuration file). The response is mostly modified in-place by :meth:`deliverance.ruleset.RuleSet.apply_rules`. Finally :meth:`deliverance.log.SavingLogger.finish_request` is called to display the developer console if appropriate. + +11. :meth:`deliverance.ruleset.RuleSet.apply_rules` determines the response headers, which also includes headers declared in the body with ````. + +12. The classes are determined by calling :func:`deliverance.pagematch.run_matches`, which calls all the ```` elements that can add classes to the request. Also classes from the response header ``X-Deliverance-Page-Class`` are added, and any classes in the request in ``environ['deliverance.page_classes']`` (these last classes are set when you use ````, since ```` was handled earlier). If no classes are declared, then the single class ``default`` is used. + +13. The applicable rules are determined -- that is, all the classes from the previous step are used to select the rules. Also, a theme is determined from the ```` element in a rule, or defaulting to the ```` element inside ````. The theme is also resolved as it can be a URI Template. + +14. The document and theme are parsed. All of the rules are run against the document and theme in order. Because rules can contain match attributes, some rules may be skipped. + +15. If none of the rules has ``suppress-standard="1"`` then the "standard" rules are also applied. These are located in :data:`deliverance.ruleset.standard_rule`. + +16. Any of the rules can raise :exc:`deliverance.exceptions.AbortTheme`, which will cause the entire request to be unthemed. + +17. The request is serialized and returned. But we didn't describe how the rules work internally yet... + +18. Each ```` tag is an instance of :class:`deliverance.rules.Rule`. This is just a container for actions, which are applied in-order. + +18. Each action tag is a class: ```` is :class:`deliverance.rules.Replace`, ```` is :class:`deliverance.rules.Append`, ```` is :class:`deliverance.rules.Prepend``, and ```` is :class:`deliverance.rules.Drop`. + +19. The main method is :meth:`deliverance.rules.TransformAction.apply` (or for ```` it is :meth:`deliverance.rules.Drop.apply` -- ```` is a little different from the other actions). This method does some selection tasks that are common across the selections, as well as some error checking. It then calls the method ``self.apply_transformation`` which is implemented per-action. The implementation performs the actions (which isn't that hard) and tries to make good context-sensitive log messages. + +That summarizes pretty much everything involved. Hopefully this helps you poke into the code more easily. There's still lots of ancilliary methods and modules, but this is the core of the request path. + +Parsing the Configuration +------------------------- + +The Deliverance configuration is XML, and in most cases an element in the configuration maps to a class, and that class has a method ``.parse_xml(element, source_location)`` + +Here's a run through the elements and their classes: + +````: + :class:`deliverance.proxy.ProxySettings` -- all the sub-elements are packaged into this one instance. + +````: + :class:`deliverance.proxy.Proxy` (all the `Proxy` instances are packaged in one :class:`deliverance.proxy.ProxySet`). ```` element information is stored in `Proxy`. + +````: + :class:`deliverance.proxy.ProxyDest` + +````: + :class:`deliverance.proxy.ProxyRequestModification` + +````: + :class:`deliverance.proxy.ProxyResponseModification` + +````: + :class:`deliverance.ruleset.RuleSet` -- except ```` and ```` elements, which are contained separately (as noted above). + +````: + :class:`deliverance.pagematch.Match` + +```` (and ````): + :class:`deliverance.themeref.Theme` + +````: + :class:`deliverance.rules.Rule` + +````: + :class:`deliverance.rules.Replace` + +````: + :class:`deliverance.rules.Append` + +````: + :class:`deliverance.rules.Prepend` + +````: + :class:`deliverance.rules.Drop` + +Also many classes support request and response matching. These are defined with a common set of attributes on an arbitrary element. The parsing for these is done with :class:`deliverance.pagematch.AbstractMatch`. + Modified: z3/deliverance/trunk/docs/conf.py ============================================================================== --- z3/deliverance/trunk/docs/conf.py (original) +++ z3/deliverance/trunk/docs/conf.py Mon Oct 6 21:15:14 2008 @@ -21,8 +21,7 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -#extensions = ['sphinx.ext.autodoc'] -extensions = [] +extensions = ['sphinx.ext.autodoc'] # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] Modified: z3/deliverance/trunk/docs/index.txt ============================================================================== --- z3/deliverance/trunk/docs/index.txt (original) +++ z3/deliverance/trunk/docs/index.txt Mon Oct 6 21:15:14 2008 @@ -9,6 +9,7 @@ philosophy configuration pyref + code-map news license Added: z3/deliverance/trunk/docs/modules/exceptions.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/exceptions.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,18 @@ +:mod:`deliverance.exceptions` -- shared exception objects +========================================================= + +.. automodule:: deliverance.exceptions + +.. contents:: + +Module Contents +--------------- + +.. autoexception:: DeliveranceError +.. autoexception:: DeliveranceSyntaxError +.. autoexception:: AbortTheme + +Utilities +--------- + +.. autofunction:: add_exception_info Added: z3/deliverance/trunk/docs/modules/log.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/log.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,12 @@ +:mod:`deliverance.log` -- logging +================================= + +.. automodule:: deliverance.log + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: SavingLogger +.. autoclass:: PrintingLogger Added: z3/deliverance/trunk/docs/modules/middleware.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/middleware.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,12 @@ +:mod:`deliverance.middleware` -- the theming middleware +======================================================= + +.. automodule:: deliverance.middleware + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: DeliveranceMiddleware +.. autoclass:: SubrequestRuleGetter Added: z3/deliverance/trunk/docs/modules/pagematch.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/pagematch.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,13 @@ +:mod:`deliverance.pagematch` -- matching requests and responses +=============================================================== + +.. automodule:: deliverance.pagematch + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: Match +.. autoclass:: AbstractMatch +.. autofunction:: run_matches Added: z3/deliverance/trunk/docs/modules/proxy.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/proxy.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,17 @@ +:mod:`deliverance.proxy` -- objects to represent proxying +========================================================= + +.. automodule:: deliverance.proxy + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: ProxySet +.. autoclass:: Proxy +.. autoclass:: ProxyMatch +.. autoclass:: ProxyDest +.. autoclass:: ProxyRequestModification +.. autoclass:: ProxyResponseModification +.. autoclass:: ProxySettings Added: z3/deliverance/trunk/docs/modules/proxycommand.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/proxycommand.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,13 @@ +:mod:`deliverance.proxycommand` -- the ``deliverance-proxy`` command +==================================================================== + +.. automodule:: deliverance.proxycommand + +.. contents:: + +Module Contents +--------------- + +.. autofunction:: main +.. autofunction:: run_command +.. autoclass:: ReloadingApp Added: z3/deliverance/trunk/docs/modules/pyref.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/pyref.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,16 @@ +:mod:`deliverance.pyref` -- resolving ``pyref`` attributes +========================================================== + +.. automodule:: deliverance.pyref + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: PyReference + +Utilities +--------- + +.. autoclass:: DefaultDict Added: z3/deliverance/trunk/docs/modules/rules.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/rules.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,22 @@ +:mod:`deliverance.rules` -- rules and actions +============================================= + +.. automodule:: deliverance.rules + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: Rule +.. autoclass:: RuleMatch +.. autoclass:: Replace +.. autoclass:: Append +.. autoclass:: Prepend +.. autoclass:: Drop + +Abstract Classes +---------------- + +.. autoclass:: AbstractAction +.. autoclass:: TransformAction Added: z3/deliverance/trunk/docs/modules/ruleset.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/ruleset.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,16 @@ +:mod:`deliverance.ruleset` -- rule/action container +=================================================== + +.. automodule:: deliverance.ruleset + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: RuleSet + +Utility Functions +----------------- + +.. autofunction:: parse_meta_headers Added: z3/deliverance/trunk/docs/modules/security.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/security.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,14 @@ +:mod:`deliverance.security` -- security context/policy +====================================================== + +.. automodule:: deliverance.security + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: SecurityContext +.. autofunction:: display_logging +.. autofunction:: display_local_files +.. autofunction:: execute_pyref Added: z3/deliverance/trunk/docs/modules/selector.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/selector.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,11 @@ +:mod:`deliverance.selector` -- selectors (XPath, CSS, etc) +========================================================== + +.. automodule:: deliverance.selector + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: Selector Added: z3/deliverance/trunk/docs/modules/stringmatch.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/stringmatch.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,29 @@ +:mod:`deliverance.stringmatch` -- matcher for strings +===================================================== + +.. automodule:: deliverance.stringmatch + +.. contents:: + +Module Contents +--------------- + +.. autofunction:: compile_matcher +.. autofunction:: compile_header_matcher +.. autoexception:: MatchSyntaxError + +Internal Classes +---------------- + +.. autoclass:: Matcher +.. autoclass:: WildcardMatcher +.. autoclass:: WildcardInsensitiveMatcher +.. autoclass:: RegexMatcher +.. autoclass:: PathMatcher +.. autoclass:: ExactMatcher +.. autoclass:: ExactInsensitiveMatcher +.. autoclass:: ContainsMatcher +.. autoclass:: ContainsInsensitiveMatcher +.. autoclass:: BooleanMatcher +.. autoclass:: HeaderMatcher +.. autoclass:: HeaderWildcardMatcher Added: z3/deliverance/trunk/docs/modules/themeref.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/themeref.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,11 @@ +:mod:`deliverance.themeref` -- ```` elements +=================================================== + +.. automodule:: deliverance.themeref + +.. contents:: + +Module Contents +--------------- + +.. autoclass:: Theme Added: z3/deliverance/trunk/docs/modules/util.txt ============================================================================== --- (empty file) +++ z3/deliverance/trunk/docs/modules/util.txt Mon Oct 6 21:15:14 2008 @@ -0,0 +1,55 @@ +:mod:`deliverance.util` -- miscellaneous functions +================================================== + +.. contents:: + +Modules +------- + +converters +~~~~~~~~~~ + +.. automodule:: deliverance.util.converters + +.. autofunction:: asbool +.. autofunction:: html_quote + +filetourl +~~~~~~~~~ + +.. automodule:: deliverance.util.filetourl + +.. autofunction:: filename_to_url +.. autofunction:: url_to_filename + +importstring +~~~~~~~~~~~~ + +.. automodule:: deliverance.util.importstring + +.. autofunction:: eval_import +.. autofunction:: simple_import +.. autofunction:: import_module +.. autofunction:: try_import_module + +nesteddict +~~~~~~~~~~ + +.. automodule:: deliverance.util.nesteddict + +.. autoclass:: NestedDict + +uritemplate +~~~~~~~~~~~ + +.. automodule:: deliverance.util.uritemplate + +.. autofunction:: uri_template_substitute + +urlnormalize +~~~~~~~~~~~~ + +.. automodule:: deliverance.util.urlnormalize + +.. autofunction:: url_normalize + From ianb at codespeak.net Tue Oct 7 01:41:17 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Tue, 7 Oct 2008 01:41:17 +0200 (CEST) Subject: [z3-checkins] r58693 - in z3/deliverance/trunk: deliverance docs Message-ID: <20081006234117.25BC7169EED@codespeak.net> Author: ianb Date: Tue Oct 7 01:41:14 2008 New Revision: 58693 Modified: z3/deliverance/trunk/deliverance/proxy.py z3/deliverance/trunk/deliverance/security.py z3/deliverance/trunk/docs/configuration.txt Log: Add edit-local-files setting Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Tue Oct 7 01:41:14 2008 @@ -566,12 +566,14 @@ """ def __init__(self, server_host, execute_pyref=True, display_local_files=True, + edit_local_files=True, dev_allow_ips=None, dev_deny_ips=None, dev_htpasswd=None, dev_users=None, dev_expiration=0, source_location=None): self.server_host = server_host self.execute_pyref = execute_pyref self.display_local_files = display_local_files + self.edit_local_files = edit_local_files self.dev_allow_ips = dev_allow_ips self.dev_deny_ips = dev_deny_ips self.dev_htpasswd = dev_htpasswd @@ -596,6 +598,7 @@ ## FIXME: should these defaults be passed in: execute_pyref = True display_local_files = True + edit_local_files = True dev_allow_ips = [] dev_deny_ips = [] dev_htpasswd = None @@ -621,6 +624,8 @@ dev_expiration = int(dev_expiration) elif child.tag == 'display-local-files': display_local_files = asbool(cls.substitute(child.text, environ)) + elif child.tag == 'edit-local-files': + edit_local_files = asbool(cls.substitute(child.text, environ)) elif child.tag == 'dev-user': username = cls.substitute(child.get('username', ''), environ) ## FIXME: allow hashed password? @@ -648,6 +653,7 @@ ## FIXME: add a default allow_ips of 127.0.0.1? return cls(server_host, execute_pyref=execute_pyref, display_local_files=display_local_files, + edit_local_files=edit_local_files, dev_allow_ips=dev_allow_ips, dev_deny_ips=dev_deny_ips, dev_users=dev_users, dev_htpasswd=dev_htpasswd, dev_expiration=dev_expiration, @@ -701,7 +707,8 @@ else: password_checker = None app = SecurityContext.middleware(app, execute_pyref=self.execute_pyref, - display_local_files=self.display_local_files) + display_local_files=self.display_local_files, + edit_local_files=self.edit_local_files) if password_checker is None and not self.dev_htpasswd: ## FIXME: warn here? return app Modified: z3/deliverance/trunk/deliverance/security.py ============================================================================== --- z3/deliverance/trunk/deliverance/security.py (original) +++ z3/deliverance/trunk/deliverance/security.py Tue Oct 7 01:41:14 2008 @@ -29,10 +29,12 @@ """ def __init__(self, execute_pyref=False, display_logging=None, - display_local_files=None, force_dev_auth=False): + display_local_files=None, edit_local_files=None, + force_dev_auth=False): self._execute_pyref = execute_pyref self._display_logging = display_logging self._display_local_files = display_local_files + self._edit_local_files = edit_local_files self._force_dev_auth = force_dev_auth @classmethod @@ -61,9 +63,8 @@ return self._execute_pyref def edit_local_files(self, environ): - if self._display_logging is not None: - return self._display_logging - ## FIXME: this is not a restrictive enough rule; should be like pyref: + if self._edit_local_files is not None: + return self._edit_local_files return self.is_developer_user(environ) def is_developer_user(self, environ): Modified: z3/deliverance/trunk/docs/configuration.txt ============================================================================== --- z3/deliverance/trunk/docs/configuration.txt (original) +++ z3/deliverance/trunk/docs/configuration.txt Tue Oct 7 01:41:14 2008 @@ -368,6 +368,9 @@ ````: The `developer debugging console`_ will, by default, display any local files. If the person accessing that console generally has ssh or other access to the server this isn't a problem. But if not you should turn this off. +````: + You can edit files through the `developer debugging console`_ unless you include ``false``. + The ```` tags define access to the `developer console`_. ````: From ianb at codespeak.net Tue Oct 7 03:00:28 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Tue, 7 Oct 2008 03:00:28 +0200 (CEST) Subject: [z3-checkins] r58694 - z3/deliverance/trunk/docs Message-ID: <20081007010028.379F9168097@codespeak.net> Author: ianb Date: Tue Oct 7 03:00:25 2008 New Revision: 58694 Modified: z3/deliverance/trunk/docs/quickstart.txt Log: link quickstart to config doc Modified: z3/deliverance/trunk/docs/quickstart.txt ============================================================================== --- z3/deliverance/trunk/docs/quickstart.txt (original) +++ z3/deliverance/trunk/docs/quickstart.txt Tue Oct 7 03:00:25 2008 @@ -80,4 +80,4 @@ Editing the Rules ----------------- -Here's where the quickstart ends for now; you'll have to read the rest of the documentation to understand the rules. +Here's where the quickstart ends for now; you'll have to read the rest of the documentation to understand the rules, specifically the `rule and theme `_ section. From ianb at codespeak.net Tue Oct 7 03:02:53 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Tue, 7 Oct 2008 03:02:53 +0200 (CEST) Subject: [z3-checkins] r58695 - in z3/deliverance/trunk: deliverance deliverance/editor deliverance/editor/media deliverance/tests docs Message-ID: <20081007010253.DFEE8169EE7@codespeak.net> Author: ianb Date: Tue Oct 7 03:02:53 2008 New Revision: 58695 Added: z3/deliverance/trunk/deliverance/editor/view_dir_template.html (contents, props changed) Modified: z3/deliverance/trunk/deliverance/editor/editorapp.py z3/deliverance/trunk/deliverance/editor/media/delivxml.js z3/deliverance/trunk/deliverance/editor/media/style.css z3/deliverance/trunk/deliverance/log.py z3/deliverance/trunk/deliverance/proxy.py z3/deliverance/trunk/deliverance/tests/example-proxy-rule.xml z3/deliverance/trunk/docs/configuration.txt Log: Add , which lets you edit files through the developer console. Modified: z3/deliverance/trunk/deliverance/editor/editorapp.py ============================================================================== --- z3/deliverance/trunk/deliverance/editor/editorapp.py (original) +++ z3/deliverance/trunk/deliverance/editor/editorapp.py Tue Oct 7 03:02:53 2008 @@ -30,12 +30,22 @@ assert filename.startswith(self.base_dir) else: filename = self.filename - if req.method == 'POST': - resp = self.save_file(req, filename) - elif req.method == 'GET': - resp = self.edit_file(req, filename) + if req.method not in ('GET', 'POST'): + resp = exc.HTTPMethodNotAllowed('Bad method: %s' % req.method, + allow='GET,POST') + elif os.path.isdir(filename): + if req.method == 'POST': + resp = self.save_create(req, filename) + else: + if not req.path.endswith('/'): + resp = exc.HTTPMovedPermanently(add_slash=True) + else: + resp = self.view_dir(req, filename) else: - assert 0, 'bad method: %r' % req.method + if req.method == 'POST': + resp = self.save_file(req, filename) + elif req.method == 'GET': + resp = self.edit_file(req, filename) return resp(environ, start_response) def edit_url(self, req, filename): @@ -103,3 +113,46 @@ edit_template = HTMLTemplate.from_filename( os.path.join(os.path.dirname(__file__), 'editor_template.html')) + + def save_create(self, req, dir): + file = req.POST['file'] + filename = req.POST.get('filename') or file.filename + filename = filename.replace('\\', '/') + filename = os.path.basename(os.path.normpath(filename)) + filename = os.path.join(dir, filename) + if os.path.exists(filename): + return exc.HTTPForbidden( + "The file %s already exists, you cannot upload over it" % filename) + f = open(filename, 'wb') + f.write(file.value) + f.close() + return exc.HTTPFound( + location=self.edit_url(req, filename)) + + skip_files = ['.svn', 'CVS', '.hg'] + + def view_dir(self, req, dir): + dir = os.path.normpath(dir) + show_parent = dir != self.base_dir + children = [os.path.join(dir, name) for name in os.listdir(dir) + if name not in self.skip_files] + def edit_url(filename): + return self.edit_url(req, filename) + title = self.title or dir + body = self.view_dir_template.substitute( + req=req, + dir=dir, + show_parent=show_parent, + title=title, + basename=os.path.basename, + dirname=os.path.dirname, + isdir=os.path.isdir, + children=children, + edit_url=edit_url, + ) + resp = Response(body=body) + resp.cache_expires() + return resp + + view_dir_template = HTMLTemplate.from_filename( + os.path.join(os.path.dirname(__file__), 'view_dir_template.html')) Modified: z3/deliverance/trunk/deliverance/editor/media/delivxml.js ============================================================================== --- z3/deliverance/trunk/deliverance/editor/media/delivxml.js (original) +++ z3/deliverance/trunk/deliverance/editor/media/delivxml.js Tue Oct 7 03:02:53 2008 @@ -10,7 +10,7 @@ ,'KEYWORDS' : { 'values' : [ 'ruleset', - 'server-settings', 'server', 'execute-pyref', 'display-local-files', + 'server-settings', 'server', 'execute-pyref', 'display-local-files', 'edit-local-files', 'dev-allow', 'dev-deny', 'dev-htpasswd', 'dev-user', 'dev-expiration', 'proxy', 'dest', 'request', 'response', 'theme', 'rule', 'replace', 'append', 'prepend', 'drop' @@ -20,7 +20,7 @@ 'manytheme', 'nocontent', 'manycontent', 'href', 'move', 'suppress-standard', 'class', 'domain', 'path', 'header', 'rewrite-links', 'request-header', - 'response-header', 'environ'] + 'response-header', 'environ', 'editable'] } ,'OPERATORS' :[ ] Modified: z3/deliverance/trunk/deliverance/editor/media/style.css ============================================================================== --- z3/deliverance/trunk/deliverance/editor/media/style.css (original) +++ z3/deliverance/trunk/deliverance/editor/media/style.css Tue Oct 7 03:02:53 2008 @@ -1,3 +1,16 @@ body { font-family: sans-serif; } + +table.dir { + width: 100%; + padding-bottom: 1em; +} + +tr.odd td { + background-color: #ddd; +} + +h1 { + font-size: 100%; +} Added: z3/deliverance/trunk/deliverance/editor/view_dir_template.html ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/editor/view_dir_template.html Tue Oct 7 03:02:53 2008 @@ -0,0 +1,39 @@ + + +Directory: {{title}} + + + + + +

+{{if show_parent}} +{{dirname(dir)}} /{{basename(dir)}}/ +{{else}} +{{dir}}/ +{{endif}} +

+ + + + {{for loop, filename in looper(children):}} + + + + {{endfor}} +
+ {{basename(filename)}}{{if isdir(filename)}}/{{endif}} +
+ +
+ Upload a new file +
+Filename (optional):
+
+ + +
+
+ + Modified: z3/deliverance/trunk/deliverance/log.py ============================================================================== --- z3/deliverance/trunk/deliverance/log.py (original) +++ z3/deliverance/trunk/deliverance/log.py Tue Oct 7 03:02:53 2008 @@ -25,6 +25,8 @@ self.request = request # This is writable: self.theme_url = None + # Also writable: + self.edit_url = None def message(self, level, el, msg, *args, **kw): """Add one message at the given log level""" @@ -70,13 +72,16 @@
{{if log.theme_url}} - theme: {{log.theme_url}} + theme: {{theme_url}} {{else}} theme: no theme set {{endif}} | unthemed content | content source | browse content + {{if log.edit_url}} + | edit files + {{endif}} {{if edit_rules}} | edit rules {{endif}} Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Tue Oct 7 03:02:53 2008 @@ -24,10 +24,11 @@ from deliverance.log import SavingLogger from deliverance.util.uritemplate import uri_template_substitute from deliverance.util.nesteddict import NestedDict -from deliverance.security import execute_pyref +from deliverance.security import execute_pyref, edit_local_files from deliverance.pyref import PyReference from deliverance.util.filetourl import filename_to_url, url_to_filename from deliverance.util.urlnormalize import url_normalize +from deliverance.editor.editorapp import Editor class ProxySet(object): """ @@ -64,9 +65,11 @@ """ request = Request(environ) log = environ['deliverance.log'] - for proxy in self.proxies: + for index, proxy in enumerate(self.proxies): ## FIXME: obviously this is wonky: if proxy.match(request, None, None, log): + if proxy.editable: + log.edit_url = request.application_url + '/.deliverance/proxy-editor/%s/' % (index+1) try: return proxy.forward_request(environ, start_response) except AbortProxy, e: @@ -92,8 +95,18 @@ req = Request(environ) log = SavingLogger(req, self.deliverator) req.environ['deliverance.log'] = log + if req.path_info.startswith('/.deliverance/proxy-editor/'): + req.path_info_pop() + req.path_info_pop() + return self.proxy_editor(environ, start_response) return self.deliverator(environ, start_response) + def proxy_editor(self, environ, start_response): + req = Request(environ) + proxy = self.proxies[int(req.path_info_pop())-1] + return proxy.edit_app(environ, start_response) + + class Proxy(object): """Represents one ```` element. @@ -105,7 +118,7 @@ def __init__(self, match, dest, request_modifications, response_modifications, strip_script_name=True, keep_host=False, - source_location=None, classes=None): + source_location=None, classes=None, editable=False): self.match = match self.match.proxy = self self.dest = dest @@ -115,6 +128,7 @@ self.response_modifications = response_modifications self.source_location = source_location self.classes = classes + self.editable = editable def log_description(self, log=None): """The debugging description for use in log display""" @@ -129,6 +143,8 @@ parts.append('strip-script-name="0"') if self.keep_host: parts.append('keep-host="1"') + if self.editable: + parts.append('editable="1"') parts.append('>
\n') parts.append(' ' + self.dest.log_description(log)) parts.append('
\n') @@ -157,6 +173,7 @@ response_modifications = [] strip_script_name = True keep_host = False + editable = asbool(el.get('editable')) for child in el: if child.tag == 'dest': if dest is not None: @@ -183,10 +200,27 @@ raise DeliveranceSyntaxError( "Unknown tag in : %s" % xml_tostring(child), element=child, source_location=source_location) + if editable: + if not dest: + ## FIXME: should this always be a test? + raise DeliveranceSyntaxError("You must have a tag", + element=element, source_location=source_location) + try: + href = uri_template_substitute( + dest.href, dict(here=posixpath.dirname(source_location))) + except KeyError: + raise DeliveranceSyntaxError( + 'You can only use if you have a that only contains {here} (you have %s)' + % (dest.href)) + if not href.startswith('file:'): + raise DeliveranceSyntaxError( + 'You can only use if you have a (you have %s)' + % (dest)) classes = el.get('class', '').split() or None inst = cls(match, dest, request_modifications, response_modifications, strip_script_name=strip_script_name, keep_host=keep_host, - source_location=source_location, classes=classes) + source_location=source_location, classes=classes, + editable=editable) match.proxy = inst return inst @@ -314,6 +348,25 @@ # I don't really need a copied request here, because FileApp is so simple: resp = request.get_response(app) return resp, orig_base, dest, proxied_url + + def edit_app(self, environ, start_response): + try: + if not self.editable: + raise exc.HTTPForbidden('This proxy is not editable="1"') + if not edit_local_files(environ): + raise exc.HTTPForbidden('Editing is forbidden') + try: + dest_href = uri_template_substitute( + self.dest.href, dict(here=posixpath.dirname(self.source_location))) + except KeyError: + raise exc.HTTPForbidden('Not a static location: %s' % dest.href) + if not dest_href.startswith('file:/'): + raise exc.HTTPForbidden('Not local: %s' % self.dest.href) + filename = url_to_filename(dest_href) + editor = Editor(base_dir=filename) + return editor(environ, start_response) + except exc.HTTPException, e: + return e(environ, start_response) class ProxyMatch(AbstractMatch): """Represents the request matching for objects""" Modified: z3/deliverance/trunk/deliverance/tests/example-proxy-rule.xml ============================================================================== --- z3/deliverance/trunk/deliverance/tests/example-proxy-rule.xml (original) +++ z3/deliverance/trunk/deliverance/tests/example-proxy-rule.xml Tue Oct 7 03:02:53 2008 @@ -7,7 +7,7 @@ - + Modified: z3/deliverance/trunk/docs/configuration.txt ============================================================================== --- z3/deliverance/trunk/docs/configuration.txt (original) +++ z3/deliverance/trunk/docs/configuration.txt Tue Oct 7 03:02:53 2008 @@ -412,6 +412,8 @@ You can also add `page classes`_ for the proxied request, by using ````. +You can use ```` to allow the files references by the proxy to be editable through the `developer debugging console`_. If you do this you have to have a ```` that references a ``file:///...`` URL. + The proxy can contain several elements... proxy: ```` From ianb at codespeak.net Tue Oct 7 04:43:23 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Tue, 7 Oct 2008 04:43:23 +0200 (CEST) Subject: [z3-checkins] r58696 - z3/deliverance/sandbox/ianb/ploneconf2008 Message-ID: <20081007024323.6938F169E86@codespeak.net> Author: ianb Date: Tue Oct 7 04:43:21 2008 New Revision: 58696 Added: z3/deliverance/sandbox/ianb/ploneconf2008/ Log: space for a presentation From ianb at codespeak.net Thu Oct 9 15:35:53 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 15:35:53 +0200 (CEST) Subject: [z3-checkins] r58858 - z3/deliverance/trunk Message-ID: <20081009133553.6B038169E03@codespeak.net> Author: ianb Date: Thu Oct 9 15:35:51 2008 New Revision: 58858 Modified: z3/deliverance/trunk/build-bundle Log: Add PageCollector to bundle Modified: z3/deliverance/trunk/build-bundle ============================================================================== --- z3/deliverance/trunk/build-bundle (original) +++ z3/deliverance/trunk/build-bundle Thu Oct 9 15:35:51 2008 @@ -27,11 +27,13 @@ echo "CREATING BUNDLE: $FILE" echo +## FIXME: httplib2 still isn't being included ## httplib2 location is a temporary problem with a bad pypi entry pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz \ -e svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=Deliverance \ supervisor PasteScript \ + -e svn+http://svn.colorstudy.com/home/ianb/PageCollector/trunk#egg=PageCollector \ || fail 'Bundle failed' echo 'INSTALLING' From ianb at codespeak.net Thu Oct 9 15:36:22 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 15:36:22 +0200 (CEST) Subject: [z3-checkins] r58859 - z3/deliverance/trunk/deliverance Message-ID: <20081009133622.B36F3169E03@codespeak.net> Author: ianb Date: Thu Oct 9 15:36:22 2008 New Revision: 58859 Modified: z3/deliverance/trunk/deliverance/middleware.py Log: Fix a few little details in selection viewing Modified: z3/deliverance/trunk/deliverance/middleware.py ============================================================================== --- z3/deliverance/trunk/deliverance/middleware.py (original) +++ z3/deliverance/trunk/deliverance/middleware.py Thu Oct 9 15:36:22 2008 @@ -6,6 +6,7 @@ import mimetypes import os import urllib +import re from webob import Request, Response from webob import exc from wsgiproxy.exactproxy import proxy_exact_request @@ -208,11 +209,11 @@ selector = req.GET.get('selector', '') subresp = resource_fetcher(url) if source: - return self.view_source(req, subresp) + return self.view_source(req, subresp, url) elif browse: - return self.view_browse(req, subresp) + return self.view_browse(req, subresp, url) elif selector: - return self.view_selection(req, subresp) + return self.view_selection(req, subresp, url) else: return exc.HTTPBadRequest( "You must have a query variable source, browse, or selector") @@ -228,7 +229,7 @@ app = Editor(filename=filename, force_syntax='delivxml', title='rule file %s' % os.path.basename(filename)) return app - def view_source(self, req, resp): + def view_source(self, req, resp, url): """ View the highlighted source (from `action_view`). """ @@ -245,7 +246,7 @@ HtmlFormatter(full=True, linenos=True, lineanchors='code')) return Response(text) - def view_browse(self, req, resp): + def view_browse(self, req, resp, url): """ View the id/class browser (from `action_view`) """ @@ -286,14 +287,14 @@ body = extra_body + body return Response(body) - def view_selection(self, req, resp): + def view_selection(self, req, resp, url): """ View the highlighted selector (from `action_view`) """ from deliverance.selector import Selector doc = document_fromstring(resp.body) el = Element('base') - el.set('href', posixpath.dirname(req.GET['url']) + '/') + el.set('href', posixpath.dirname(url) + '/') doc.head.insert(0, el) selector = Selector.parse(req.GET['selector']) dummy_type, elements, dummy_attributes = selector(doc) @@ -337,13 +338,41 @@ def format_tag(tag): """Highlights the lxml HTML tag""" return highlight(tostring(tag).split('>')[0]+'>') + def wrap_html(html, width=100): + if isinstance(html, _Element): + html = tostring(html) + lines = html.splitlines() + new_lines = [] + def wrap_html_line(line): + if len(line) <= width: + return [line] + match_trail = re.search(r'^[^<]*', line, re.S) + if match_trail: + result = [match_trail.group(0)] + result.extend(wrap_html_line(line[match_trail.end():])) + return result + match1 = re.search(r'^[^<]*<[^>]*>', line, re.S) + match2 = re.search(r'<[^>]*>[^<>]*$', line, re.S) + if not match1 or not match2: + return [line] + result = [match1.group(0)] + result.extend(wrap_html_line(line[match1.end():match2.start()])) + result.append(match2.group(0)) + return result + for line in lines: + new_lines.extend(wrap_html_line(line)) + return '\n'.join(new_lines) + def mark_deliv_match(highlighted_text): + result = re.sub(r'(?:<[^/][^>]*>)*<.*?DELIVERANCE-MATCH=.*?>(?:]*>)*', lambda match: r'%s' % match.group(0), unicode(highlighted_text), re.S) + return html(result) text = template.substitute( - base_url=req.url, + base_url=url, els_in_head=els_in_head, doc=doc, elements=all_elements, selector=selector, - format_tag=format_tag, highlight=highlight) + format_tag=format_tag, highlight=highlight, + wrap_html=wrap_html, mark_deliv_match=mark_deliv_match) message = fromstring( - self._message_template.substitute(message=text, url=req.url)) + self._message_template.substitute(message=text, url=url)) if doc.body.text: message.tail = doc.body.text doc.body.text = '' @@ -388,14 +417,14 @@
Elements matched in head. Showing head:
- {{highlight(doc.head)}} + {{mark_deliv_match(highlight(wrap_html(doc.head)))}}
{{endif}} ''', 'deliverance.middleware.DeliveranceMiddleware._found_template') _message_template = HTMLTemplate('''\ -
+
Viewing {{url}}
{{message|html}} From ianb at codespeak.net Thu Oct 9 17:27:01 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 17:27:01 +0200 (CEST) Subject: [z3-checkins] r58860 - z3/deliverance/trunk Message-ID: <20081009152701.097DE169FAC@codespeak.net> Author: ianb Date: Thu Oct 9 17:26:59 2008 New Revision: 58860 Modified: z3/deliverance/trunk/build-bundle Log: get httplib2 in there properly Modified: z3/deliverance/trunk/build-bundle ============================================================================== --- z3/deliverance/trunk/build-bundle (original) +++ z3/deliverance/trunk/build-bundle Thu Oct 9 17:26:59 2008 @@ -30,7 +30,8 @@ ## FIXME: httplib2 still isn't being included ## httplib2 location is a temporary problem with a bad pypi entry pyinstall.py --bundle=$FILE --build=build-bundle-files/ --src=src-bundle-files/ \ - http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz \ + -f http://httplib2.googlecode.com/files/httplib2-0.4.0.tar.gz#egg=httplib2-0.4.0 \ + -e svn+http://httplib2.googlecode.com/svn/trunk/#egg=httplib2 \ -e svn+http://codespeak.net/svn/z3/deliverance/trunk#egg=Deliverance \ supervisor PasteScript \ -e svn+http://svn.colorstudy.com/home/ianb/PageCollector/trunk#egg=PageCollector \ From ianb at codespeak.net Thu Oct 9 17:27:16 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 17:27:16 +0200 (CEST) Subject: [z3-checkins] r58861 - z3/deliverance/trunk/docs Message-ID: <20081009152716.42379169FC8@codespeak.net> Author: ianb Date: Thu Oct 9 17:27:15 2008 New Revision: 58861 Modified: z3/deliverance/trunk/docs/index.txt Log: Make the ToC reasonable Modified: z3/deliverance/trunk/docs/index.txt ============================================================================== --- z3/deliverance/trunk/docs/index.txt (original) +++ z3/deliverance/trunk/docs/index.txt Thu Oct 9 17:27:15 2008 @@ -4,6 +4,7 @@ Deliverance is a tool to theme HTML, applying a consistent style to applications and static files regardless of how they are implemented, and separating site-wide styling from application-level templating. .. toctree:: + :maxdepth: 1 quickstart philosophy @@ -16,6 +17,7 @@ These are some documents that may or may not be accurate; they are a little out of date: .. toctree:: + :maxdepth: 1 page-classes rules From ianb at codespeak.net Thu Oct 9 17:28:13 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 17:28:13 +0200 (CEST) Subject: [z3-checkins] r58862 - z3/deliverance/trunk/deliverance Message-ID: <20081009152813.1A29B169FAC@codespeak.net> Author: ianb Date: Thu Oct 9 17:28:12 2008 New Revision: 58862 Modified: z3/deliverance/trunk/deliverance/paster_templates.py Log: Create .jpg files. Handle failures in embedded resources better Modified: z3/deliverance/trunk/deliverance/paster_templates.py ============================================================================== --- z3/deliverance/trunk/deliverance/paster_templates.py (original) +++ z3/deliverance/trunk/deliverance/paster_templates.py Thu Oct 9 17:28:12 2008 @@ -34,6 +34,7 @@ def get_content(self, url): """Gets the content and all embedded content (images, CSS, etc)""" + print 'Fetching theme at %s' % url page = parse(urllib2.urlopen(url)).getroot() page.make_links_absolute() files = [] @@ -41,6 +42,8 @@ if not self._embedded_link(element): continue filename, content = self.get_embedded(link) + if not filename: + continue files.append((filename, content)) if attr is None: old_value = element.text @@ -62,7 +65,12 @@ return False def get_embedded(self, url): - resp = urllib2.urlopen(url) + print ' fetching %s' % url + try: + resp = urllib2.urlopen(url) + except urllib2.HTTPError, e: + print 'Could not fetch %s: %s' % (url, e) + return None, None url = resp.geturl() content = resp.read() content_type = resp.info()['content-type'] @@ -71,7 +79,7 @@ if not filename: filename = 'embedded' ext = mimetypes.guess_extension(content_type) - if ext == '.jpeg': + if ext == '.jpeg' or ext == 'jpe': ext = '.jpg' ext = ext or orig_ext return filename + ext, content From ianb at codespeak.net Thu Oct 9 17:28:57 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 17:28:57 +0200 (CEST) Subject: [z3-checkins] r58863 - in z3/deliverance/trunk: . deliverance deliverance/util Message-ID: <20081009152857.25F84169FC8@codespeak.net> Author: ianb Date: Thu Oct 9 17:28:56 2008 New Revision: 58863 Added: z3/deliverance/trunk/deliverance/util/proxyrequest.py (contents, props changed) Modified: z3/deliverance/trunk/deliverance/proxy.py z3/deliverance/trunk/setup.py Log: Avoid encoded responses to the proxy. Handle encoding of proxy responses better. Modified: z3/deliverance/trunk/deliverance/proxy.py ============================================================================== --- z3/deliverance/trunk/deliverance/proxy.py (original) +++ z3/deliverance/trunk/deliverance/proxy.py Thu Oct 9 17:28:56 2008 @@ -9,7 +9,7 @@ import socket import os import string -from webob import Request, Response +from deliverance.util.proxyrequest import Request from webob import exc from wsgiproxy.exactproxy import proxy_exact_request from tempita import html_quote @@ -300,6 +300,7 @@ proxy_req.headers['X-Forwarded-Path'] = proxy_req.script_name proxy_req.script_name = '' proxied_url = url_normalize('%s://%s%s' % (scheme, netloc, proxy_req.path_qs)) + proxy_req.accept_encoding = None try: resp = proxy_req.get_response(proxy_exact_request) except socket.error, e: Added: z3/deliverance/trunk/deliverance/util/proxyrequest.py ============================================================================== --- (empty file) +++ z3/deliverance/trunk/deliverance/util/proxyrequest.py Thu Oct 9 17:28:56 2008 @@ -0,0 +1,33 @@ +import webob +import chardet + +class Response(webob.Response): + default_charset = None + unicode_errors = 'replace' + + def _unicode_body__get(self): + """ + Get/set the unicode value of the body (using the charset of the Content-Type) + """ + if not self.charset: + guess = chardet.detect(self.body) + self.charset = guess['encoding'] + body = self.body + return body.decode(self.charset, self.unicode_errors) + + def _unicode_body__set(self, value): + if not self.charset: + raise AttributeError( + "You cannot access Response.unicode_body unless charset is set") + if not isinstance(value, unicode): + raise TypeError( + "You can only set Response.unicode_body to a unicode string (not %s)" % type(value)) + self.body = value.encode(self.charset) + + def _unicode_body__del(self): + del self.body + + unicode_body = property(_unicode_body__get, _unicode_body__set, _unicode_body__del, doc=_unicode_body__get.__doc__) + +class Request(webob.Request): + ResponseClass = Response Modified: z3/deliverance/trunk/setup.py ============================================================================== --- z3/deliverance/trunk/setup.py (original) +++ z3/deliverance/trunk/setup.py Thu Oct 9 17:28:56 2008 @@ -38,6 +38,7 @@ "DevAuth", "Paste", "WSGIFilter", + "chardet", ], dependency_links=[ "https://svn.openplans.org/svn/DevAuth/trunk#egg=DevAuth-dev", From ianb at codespeak.net Thu Oct 9 18:23:45 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Thu, 9 Oct 2008 18:23:45 +0200 (CEST) Subject: [z3-checkins] r58864 - z3/deliverance/trunk/deliverance/paster-templates/plone/etc Message-ID: <20081009162345.80125169E6E@codespeak.net> Author: ianb Date: Thu Oct 9 18:23:42 2008 New Revision: 58864 Modified: z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl Log: Fix up the Plone default config some; add /editor Modified: z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl ============================================================================== --- z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl (original) +++ z3/deliverance/trunk/deliverance/paster-templates/plone/etc/deliverance.xml_tmpl Thu Oct 9 18:23:42 2008 @@ -20,8 +20,19 @@ --> + + + +{{if asbool(proxy_rewrite_links):}} + +{{endif}} + + + + + - + {{if asbool(proxy_rewrite_links):}} {{endif}} @@ -45,8 +56,8 @@ - - From ianb at codespeak.net Fri Oct 10 00:15:39 2008 From: ianb at codespeak.net (ianb at codespeak.net) Date: Fri, 10 Oct 2008 00:15:39 +0200 (CEST) Subject: [z3-checkins] r58866 - in z3/deliverance/sandbox/ianb/ploneconf2008: . ui ui/default Message-ID: <20081009221539.53F3F16A071@codespeak.net> Author: ianb Date: Fri Oct 10 00:15:37 2008 New Revision: 58866 Added: z3/deliverance/sandbox/ianb/ploneconf2008/index.html (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/index.txt (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/ z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/ z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/blank.gif (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/framing.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/iepngfix.htc (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/opera.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/outline.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/pretty.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/print.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/s5-core.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/slides.css (contents, props changed) z3/deliverance/sandbox/ianb/ploneconf2008/ui/default/slides.js (contents, props changed) Log: presentation Added: z3/deliverance/sandbox/ianb/ploneconf2008/index.html ============================================================================== --- (empty file) +++ z3/deliverance/sandbox/ianb/ploneconf2008/index.html Fri Oct 10 00:15:37 2008 @@ -0,0 +1,853 @@ + + + + + + + +Deliverance: Theming Decoupled + + + + + + + + + + + + + + + + +
+
+
+ + +
+
+
+

Deliverance: Theming Decoupled

+ +++ + + + + + + + +
Author:Ian Bicking
Organization:The Open Planning Project
site:http://deliverance.openplans.org
+ +
+
+

Outline Of This Talk

+
    +
  1. A demonstration of what Deliverance does
  2. +
  3. Why Deliverance
  4. +
  5. How Deliverance Works (from the outside)
  6. +
+
+
+

Installation

+
    +
  • Installation:
      +
    • "Quickstart" on deliverance.openplans.org
    • +
    • Uses pyinstall/pybundles
    • +
    • Regular snapshots, but not automated
    • +
    +
  • +
+
+
+

paster startup

+
+$ pyinstall.py -E DelivTest Deliverance-snapshot-latest.pybundle
+$ cd DelivTest
+$ source bin/activate
+$ paster create -t deliverance_plone .
+
+
+
+

Demo

+

See the demo...

+
+
+

Why Deliverance?

+
+
+

Why Do I Work With Deliverance?

+
    +
  • I've been through a lot of frameworks
  • +
  • Using a new framework, platform, or existing application shouldn't be a repudiation of any other system
  • +
+
+
+

Why Do I Work With Deliverance?

+
    +
  • This lets me do things with Plone without poking around in the internals
  • +
  • It is built on technologies I know will stay relevant:
      +
    • HTTP
    • +
    • HTML
    • +
    +
  • +
+
+
+

Why Should You Care?

+
    +
  • Theming with no (visible) Python
  • +
  • Rules and content are entirely separate (vs. Page Templates)
  • +
  • Aggregation is relatively straight-forward
  • +
+
+
+

Why Should You Care?

+
    +
  • Clear path for creating exceptions (e.g., class="sphinx")
  • +
  • Everything is based on visible information: URLs and markup
  • +
+
+
+

Why You Should Want Heterogeneity

+
    +
  • Plone, the community, can't be everything to everyone
      +
    • For example, Plone will never have the best blog
    • +
    +
  • +
  • Without heterogeneity, every platform and specialization of a platform creates smaller and smaller audiences
  • +
+
+
+

Heterogeneous Stacks Are Hard

+

(I'm not going to deny it)

+
+
+

How Deliverance Works

+
+
+

Moving Boxes Around

+
    +
  • Deliverance moves boxes around between the content and theme, and some external resources (like the sidebar)
  • +
  • Deliverance produces no content itself
  • +
  • Deliverance doesn't care how the content was produced
  • +
+
+
+

From Content To Theme

+
    +
  • "Content" is the response that was generated for the request -- what you'd see if there was no Deliverance and no theming
  • +
  • "Theme" is the theme you've created
  • +
+
+
+

From Content To Theme

+
    +
  • The response starts with the theme page
  • +
  • Rules move stuff from the content into the theme
  • +
  • If there were no rules, every response would be the identical theme page
  • +
+
+
+

The Rules

+

Four rules:

+
    +
  • <replace>
  • +
  • <append>
  • +
  • <prepend>
  • +
  • <drop>
  • +
+
+
+

replace, append, prepend

+
+
<replace theme="..." content="..." />:
+
Remove some stuff from the theme, and replace it with stuff from the content
+
<append theme="..." content="..." />:
+
Take some stuff from the content and put it right after some stuff in the theme
+
<prepend theme="..." content="..." />:
+
Like append, but prepend
+
+
+
+

Rules

+

<replace> looks like:

+
+<replace
+ content="...selector..."
+ theme="...selector...">
+
+
+
+

Rules

+

<replace> looks like:

+
+<replace
+ content="children:#content"
+ theme="children:#content">
+
+

Replace all the children of #content in the theme, with the children of #content in the content.

+
+
+

Selectors

+
    +
  • Selectors can be CSS selectors (#content) or XPath (//*[@id='content'])
  • +
  • Selectors select elements
  • +
  • Modifiers make it more clear what you want to do
  • +
+
+
+

Selectors (modifiers)

+
+
element:...:
+
Selects the element
+
children:...:
+
Selects the children of the element
+
tag:...:
+
Selects the tag, but not its children
+
+
+
+

Selectors (modifiers)

+
+
attributes:...:
+
Selects the attributes of the elmeent
+
attributes(class):...:
+
Selects just the class attribute of the element
+
+
+
+

drop

+

Mostly to fix up problems:

+
+
<drop theme="..." />:
+
Remove some stuff from the theme
+
<drop content="..." />:
+
Remove some stuff from the content (only meaningful if you have later rules)
+
+
+
+

Examples

+
+<append theme="chidren:#content" content="elements:.content" />
+
+
    +
  • Take the elements from the content with the class .content
  • +
  • The target is children:#content -- the element with the id content (e.g., <div id="content">
  • +
  • Because the target is children, it will be appended inside the <div id="content"> tag
  • +
+
+
+

Examples

+
+<append theme="chidren:#content" content="elements:.content" />
+
+

(continued...)

+
    +
  • It will also preserve the children of that div; if you don't want to preserve that content (e.g., it looks like <div id="content">content goes here</div> use <replace>
  • +
+
+
+

Examples

+
+<drop content="tag:font" />
+
+
    +
  • This will remove the <font> tags, but not their children (will just clean the content)
  • +
+
+
+

Examples

+
+<prepend content="attributes(class):body" theme="attributes:body" />
+
+
    +
  • This will move the class attribute from <body> in the content, into <body> in the theme.
  • +
  • With <prepend> the content attributes get priority, with <append> the theme attributes get priority
  • +
+
+
+

Examples

+
+<append theme="elements:div#footer" content="elements:div#copyright" />
+
+
    +
  • This adds the element <div id="copyright"> from the content immediate after <div id="footer"> in the theme
  • +
  • This is probably not what you want
  • +
  • Usually you want children: for the theme, and elements: or children: for the content
  • +
+
+
+

if-content

+
    +
  • Rules can be conditional
  • +
  • The only condition is "if this selection matches something in the content"
  • +
+
+<drop theme="#copyright" if-content=".creative-commons" />
+<append content="elements:.creative-commons"
+        theme="elements:#content" />
+
+
+
+

Runtime rule errors/exceptions

+

Consider the rule:

+
+<replace content="children:#content"
+         theme="children:#content" />
+
+

Your 404 page may not include <div id="content">. You do not want a 404 to turn into a theme page with no information

+
+
+

Runtime rule errors/exceptions

+
+<replace content="children:#content"
+         theme="children:#content"
+         nocontent="abort" />
+
+

If children:#content matches no elements, abort all theming.

+

Other corner cases can be specified.

+
+
+

href

+
    +
  • How do you make the theme dynamic?
  • +
  • Deliverance produces no content
      +
    • It only moves content around
    • +
    +
  • +
  • If everything you want to display is in your content or theme, you can move it all around appropriately, otherwise...
  • +
+
+
+

href

+
+<append href="/sidebar" content="children:#content"
+        theme="children:#sidebar" />
+
+

Take the resource from /sidebar (which can be dynamic)

+

(In the future, this will be a URI Template, so you can implement more dynamicism)

+
+
+

Exceptional Places In The Site

+
    +
  • Usually the front page

    +
  • +
  • The simple way, don't theme it:

    +
    +<match path="exact:/" abort="1" />
    +
    +
  • +
+
+
+

Exceptional Places In The Site

+

Or change the theme:

+
+<match path="exact:/" class="frontpage" />
+<rule class="frontpage">
+  <theme href="/_themes/frontpage.html" />
+  ... rules ...
+</rule>
+
+
+
+

Exceptional Places In The Site

+

Or maybe you just want some extra rules:

+
+<match path="exact;/" class="frontpage default" />
+<rule class="frontpage">
+    ... some extra rules ...
+</rule>
+<rule> <!-- no class means "default" -->
+    <theme href="..." />
+    ... other rules ...
+</rule>
+
+
+
+

Exceptional Places In The Site

+

Some application has a PITA bit of CSS:

+
+<match path="/trac" class="trac default" />
+<rule class="trac" />
+    <drop content="link[href$='/trac.css']" />
+</rule>
+
+
+
+

Workarounds

+

Avoid theming...

+

A page name (any path that ends with /blank.html):

+
+<match path="wildcard:*/blank.html" abort="1">
+
+
+
+

Workarounds

+

Avoid theming...

+

Based on a request header:

+
+<match request-header="User-Agent: contains-insensitive:slurp" abort="1">
+
+
+
+

Workarounds

+

Avoid theming...

+

Based on a response header:

+
+<match response-header="X-No-Deliverate: bool:true" abort="1">
+
+

This includes "headers" that are in the page, like <meta http-equiv="X-No-Deliverate" content="1">

+
+
+

CSS vs. XPath

+

link[href$='trac.css'] is CSS 3 which matches all <link> tags with an href attribute that ends with /trac.css (CSS 3).

+

Compare the CSS 3 with XPath 1.0:

+
+link[href$='trac.css']
+//link[substring(@href, string-length(@href)-len('/trac.css')) = '/trac.css']
+
+
+
+

CSS

+
    +
  • CSS is great, except for it being horrible
  • +
  • CSS classes are the horrible part. And #id
  • +
+
+
+

CSS

+
    +
  • CSS is:
      +
    • A way of mapping (vaguely) semantic HTML to a presentation (styles on tags)
    • +
    • A page layout description (layout, based mostly on ids)
    • +
    • A mapping from a custom semantic document language to presentation (styles on classes)
    • +
    +
  • +
+
+
+

CSS

+
    +
  • Unfortunately, CSS is not portable
      +
    • Page layout and CSS classes use an application-specific vocabulary
    • +
    +
  • +
+
+
+

CSS

+
    +
  • And CSS is not encapsulated: you can't easily apply CSS to just a portion of a site
      +
    • Rules are flat
    • +
    • HTML generally is flat
    • +
    • This is frustrating, I don't have a solution, I'm not sure what the solution would be (mostly not sure)
    • +
    +
  • +
+
+
+

CSS

+
    +
  • Tag-based CSS (i.e., selectors without class or id conditions) should be dropped from applications, implemented by theme
  • +
  • Large-scale page layout CSS (e.g., sidebars, etc.) should also be in the theme
  • +
  • Small-scale page layout should remain with the application (e.g., laying out a form); this is typically done partially with ids and partially with classes
  • +
+
+
+

CSS

+
    +
  • The custom semantic language should be unified across all your applications, and defined by CSS in the theme
      +
    • Like a CSS framework, but better than the CSS frameworks I've seen so far
    • +
    • Current CSS frameworks seem focused on making tricky designer problems easier, not to creating a consistent and application agnostic language
    • +
    +
  • +
+
+
+

CSS & HTML 5

+
    +
  • HTML 5 increases the number of tags to increase the expressiveness of content without using CSS classes
      +
    • <article>, <section>...
    • +
    +
  • +
+
+
+

The Many-Places Problem

+
    +
  • When someone sees a page, and wants to change something on the page, they now have yet more places to look for the content
  • +
  • Deliverance has no solutions yet, this is just an acknowledgment
  • +
  • What I'd like is if elements had attributes to show where the element was generated (typically a file or URL where the template that generated the content came from)
  • +
+
+
+

The Many-Places Problem

+
    +
  • Deliverance would have to mix and match these attributes as it moves things around, but I can only add that feature if there's something worth moving around
      +
    • But I'd really like to add this feature
    • +
    • This should be applicable beyond Plone without trouble
    • +
    +
  • +
+
+
+

Development Cycle

+
    +
  • Theming is entirely separate from the aapplication
      +
    • That also means you can many many themes for one application
    • +
    • Meaning, you can do theme development with a production application
    • +
    +
  • +
  • Link rewriting allows you to theme a live site
  • +
+
+
+

Development Cycle

+
    +
  • You can change rules on the fly (though not the server and proxy settings without a restart)
  • +
  • You can change the theme on the fly (always)
  • +
+
+
+

Deployment

+
    +
  • Why the emphasis on separate server processes?
      +
    • Deploy products in isolation (buildout, virtualenv, etc -- even PHP/apache)
    • +
    • No shared code
    • +
    • Avoid a version/release lockstep between large pieces of your stack
    • +
    +
  • +
+
+
+

Deployment

+
    +
  • Why the emphasis on separate server processes?
      +
    • Upgrade pieces when you want to, get bug fixes, new features, etc., without cascading effects across your entire site
    • +
    • No migrations needed to update Deliverance
    • +
    +
  • +
+
+
+

Opportunities: Caching

+
    +
  • Everything talks HTTP
      +
    • This means you can put a cache between any two pieces
    • +
    +
  • +
  • Pages can be assembled from multiple pieces
      +
    • Some of the pieces might be cachable, even if the aggregate is not
    • +
    +
  • +
+
+
+

The Many Implementations of Deliverance

+
    +
  • Paul Everitt is working on an XSLT implementation
  • +
+
+
+ + Added: z3/deliverance/sandbox/ianb/ploneconf2008/index.txt ============================================================================== --- (empty file) +++ z3/deliverance/sandbox/ianb/ploneconf2008/index.txt Fri Oct 10 00:15:37 2008 @@ -0,0 +1,568 @@ +Deliverance: Theming Decoupled +============================== + +:author: Ian Bicking +:organization: `The Open Planning Project `_ +:site: http://deliverance.openplans.org + +Outline Of This Talk +-------------------- + +1. A demonstration of what Deliverance does +2. Why Deliverance +3. How Deliverance Works (from the outside) + +Installation +------------ + +* Installation: + + - "Quickstart" on deliverance.openplans.org + - Uses pyinstall/pybundles + - Regular snapshots, but not automated + +paster startup +-------------- + +:: + + $ pyinstall.py -E DelivTest Deliverance-snapshot-latest.pybundle + $ cd DelivTest + $ source bin/activate + $ paster create -t deliverance_plone . + +Demo +---- + +See the demo... + +Why Deliverance? +---------------- + +Why Do *I* Work With Deliverance? +--------------------------------- + +* I've been through a lot of frameworks + +* Using a new framework, platform, or existing application shouldn't be a repudiation of any other system + + +Why Do *I* Work With Deliverance? +--------------------------------- + +* This lets me do things with Plone without poking around in the internals + +* It is built on technologies I **know** will stay relevant: + + - HTTP + + - HTML + + +Why Should *You* Care? +---------------------- + +* Theming with no (visible) Python + +* Rules and content are entirely separate (vs. Page Templates) + +* Aggregation is relatively straight-forward + + +Why Should *You* Care? +---------------------- + +* Clear path for creating exceptions (e.g., ``class="sphinx"``) + +* Everything is based on visible information: URLs and markup + + +Why You Should Want Heterogeneity +--------------------------------- + +* Plone, the community, can't be everything to everyone + + - For example, Plone will never have the best blog + +* Without heterogeneity, every platform and specialization of a platform creates smaller and smaller audiences + + +Heterogeneous Stacks Are Hard +----------------------------- + +(I'm not going to deny it) + + +How Deliverance Works +--------------------- + +Moving Boxes Around +------------------- + +* Deliverance moves boxes around between the content and theme, and some external resources (like the sidebar) + +* Deliverance produces no content itself + +* Deliverance doesn't care how the content was produced + + +From Content To Theme +--------------------- + +* "Content" is the response that was generated for the request -- what you'd see if there was no Deliverance and no theming + +* "Theme" is the theme you've created + +From Content To Theme +--------------------- + +* The response starts with the theme page + +* Rules move stuff from the content into the theme + +* If there were no rules, every response would be the identical theme page + + + + +The Rules +--------- + +Four rules: + +* ```` + +* ```` + +* ```` + +* ```` + +replace, append, prepend +------------------------ + +````: + Remove some stuff from the theme, and replace it with stuff from the content + +````: + Take some stuff from the content and put it right after some stuff in the theme + +````: + Like ``append``, but prepend + +Rules +----- + +```` looks like:: + + + + +Rules +----- + +```` looks like:: + + + +Replace all the children of ``#content`` in the theme, with the children of ``#content`` in the content. + + +Selectors +--------- + +* Selectors can be CSS selectors (``#content``) or XPath (``//*[@id='content']``) + +* Selectors select *elements* + +* Modifiers make it more clear what you want to do + + +Selectors (modifiers) +--------------------- + +``element:...``: + Selects the element + +``children:...``: + Selects the children of the element + +``tag:...``: + Selects the *tag*, but not its children + +Selectors (modifiers) +--------------------- + +``attributes:...``: + Selects the attributes of the elmeent + +``attributes(class):...``: + Selects just the ``class`` attribute of the element + + +drop +---- + +Mostly to fix up problems: + +````: + Remove some stuff from the theme + +````: + Remove some stuff from the content (only meaningful if you have later rules) + + + +Examples +-------- + +:: + + + +* Take the elements from the content with the class ``.content`` + +* The target is ``children:#content`` -- the element with the id ``content`` (e.g., ``
`` + +* Because the target is ``children``, it will be appended *inside* the ``
`` tag + +Examples +-------- + +:: + + + +(continued...) + +* It will also preserve the children of that div; if you don't want to preserve that content (e.g., it looks like ``
content goes here
`` use ```` + +Examples +-------- + +:: + + + +* This will remove the ```` tags, but *not* their children (will just clean the content) + +Examples +-------- + +:: + + + +* This will move the ``class`` attribute from ```` in the content, into ```` in the theme. + +* With ```` the *content* attributes get priority, with ```` the *theme* attributes get priority + + +Examples +-------- + +:: + + + +* This adds the element ``