[py-svn] r36249 - in py/dist/py/documentation: . future
hpk at codespeak.net
hpk at codespeak.net
Mon Jan 8 12:39:13 CET 2007
Author: hpk
Date: Mon Jan 8 12:39:06 2007
New Revision: 36249
Added:
py/dist/py/documentation/future/ (props changed)
py/dist/py/documentation/future/code_template.txt
- copied unchanged from r36246, py/dist/py/documentation/code_template.txt
py/dist/py/documentation/future/future.txt
- copied, changed from r36246, py/dist/py/documentation/future.txt
Removed:
py/dist/py/documentation/code_template.txt
py/dist/py/documentation/future.txt
Modified:
py/dist/py/documentation/coding-style.txt
py/dist/py/documentation/getting-started.txt
py/dist/py/documentation/home.txt
py/dist/py/documentation/index.txt
py/dist/py/documentation/misc.txt
py/dist/py/documentation/why_py.txt
Log:
creating a future subdir and starting to move
things into it to have a cleaner "documentation" directory
Deleted: /py/dist/py/documentation/code_template.txt
==============================================================================
--- /py/dist/py/documentation/code_template.txt Mon Jan 8 12:39:06 2007
+++ (empty file)
@@ -1,640 +0,0 @@
-===============================================================
-py.code_template: Lightweight and flexible code template system
-===============================================================
-
-.. contents::
-.. sectnum::
-
-Motivation
-==========
-
-There are as many python templating systems as there are web frameworks
-(a lot). This is partly because it is so darned easy to write a templating
-system in Python. What are the distinguishing characteristics of the
-py.code_template templating system?
-
- * Optimized for generating code (Python, C, bash scripts, etc.),
- not XML or HTML
-
- * Designed for use by Python programmers, not by web artists
-
- + Aesthetic sensibilities are different
-
- + The templates should be an organic part of a module -- just more code
-
- + Templates do not need to be incredibly full-featured, because
- programmers are perfectly capable of escaping to Python for
- advanced features.
-
- - No requirement to support inheritance
- - No requirement to support exec
-
- * Designed so that templates can be coded in the most natural way
- for the task at hand
-
- + Generation of code and scripts often does not follow MVC paradigm!
-
- + Small template fragments are typically coded *inside* Python modules
-
- + Sometimes it is natural to put strings inside code; sometimes it is
- natural to put code inside strings. Both should be supported as
- reasonably and naturally as possible.
-
-Imaginary-world examples
-========================
-
-These would be real-world examples, but, not only is this module not yet
-implemented, as of now, PyPy is not incredibly useful to the average
-programmer...
-
-translator/c/genc.py
---------------------
-
-The original function::
-
- def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]):
- #
- # All declarations
- #
- structdeflist = database.getstructdeflist()
- print >> f
- print >> f, '/***********************************************************/'
- print >> f, '/*** Structure definitions ***/'
- print >> f
- for node in structdeflist:
- print >> f, 'struct %s;' % node.name
- print >> f
- for node in structdeflist:
- for line in node.definition():
- print >> f, line
- print >> f
- print >> f, '/***********************************************************/'
- print >> f, '/*** Forward declarations ***/'
- print >> f
- for node in database.globalcontainers():
- for line in node.forward_declaration():
- print >> f, line
-
- #
- # Implementation of functions and global structures and arrays
- #
- print >> f
- print >> f, '/***********************************************************/'
- print >> f, '/*** Implementations ***/'
- print >> f
- for line in preimplementationlines:
- print >> f, line
- print >> f, '#include "src/g_include.h"'
- print >> f
- blank = True
- for node in database.globalcontainers():
- if blank:
- print >> f
- blank = False
- for line in node.implementation():
- print >> f, line
- blank = True
-
-This could be refactored heavily. An initial starting point
-would look something like this, although later, the template
-instance could be passed in and reused directly, rather than
-passing the file handle around::
-
- def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]):
- def container_implementation():
- # Helper function designed to introduce blank lines
- # between container implementations
-
- blank = True
- for node in database.globalcontainers():
- if blank:
- yield ''
- blank = False
- for line in node.implementation():
- yield line
- blank = True
-
- t = code_template.Template()
- #
- # All declarations
- #
- structdeflist = database.getstructdeflist()
- t.write(dedent=8, text='''
-
- /***********************************************************/
- /*** Structure definitions ***/
-
- {for node in structdeflist}
- struct {node.name};
- {endfor}
-
- {for node in structdeflist}
- {for line in node.definition}
- {line}
- {endfor}
- {endfor}
-
- /***********************************************************/
- /*** Forward declarations ***/
-
- {for node in database.globalcontainers()}
- {for line in node.forward_declaration()}
- {line}
- {endfor}
- {endfor}
-
- {**
- ** Implementation of functions and global structures and arrays
- **}
-
- /***********************************************************/
- /*** Implementations ***/
-
- {for line in preimplementationlines}
- {line}
- {endfor}
-
- #include "src/g_include.h"
-
- {for line in container_implementation()}
- {line}
- {endfor}
- """)
- t.output(f)
-
-translator/c/genc.py gen_makefile
----------------------------------
-
-The original code::
-
- MAKEFILE = '''
- CC = gcc
-
- $(TARGET): $(OBJECTS)
- \t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS)
-
- %.o: %.c
- \t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS)
-
- clean:
- \trm -f $(OBJECTS)
- '''
-
- def gen_makefile(self, targetdir):
- def write_list(lst, prefix):
- for i, fn in enumerate(lst):
- print >> f, prefix, fn,
- if i < len(lst)-1:
- print >> f, '\\'
- else:
- print >> f
- prefix = ' ' * len(prefix)
-
- compiler = self.getccompiler(extra_includes=['.'])
- cfiles = []
- ofiles = []
- for fn in compiler.cfilenames:
- fn = py.path.local(fn).basename
- assert fn.endswith('.c')
- cfiles.append(fn)
- ofiles.append(fn[:-2] + '.o')
-
- f = targetdir.join('Makefile').open('w')
- print >> f, '# automatically generated Makefile'
- print >> f
- print >> f, 'TARGET =', py.path.local(compiler.outputfilename).basename
- print >> f
- write_list(cfiles, 'SOURCES =')
- print >> f
- write_list(ofiles, 'OBJECTS =')
- print >> f
- args = ['-l'+libname for libname in compiler.libraries]
- print >> f, 'LIBS =', ' '.join(args)
- args = ['-L'+path for path in compiler.library_dirs]
- print >> f, 'LIBDIRS =', ' '.join(args)
- args = ['-I'+path for path in compiler.include_dirs]
- write_list(args, 'INCLUDEDIRS =')
- print >> f
- print >> f, 'CFLAGS =', ' '.join(compiler.compile_extra)
- print >> f, 'LDFLAGS =', ' '.join(compiler.link_extra)
- print >> f, MAKEFILE.strip()
- f.close()
-
-
-Could look something like this::
-
- MAKEFILE = '''
- # automatically generated Makefile
-
- TARGET = {py.path.local(compiler.outputfilename).basename}
-
- {for line in write_list(cfiles, 'SOURCES =')}
- {line}
- {endfor}
-
- {for line in write_list(ofiles, 'OBJECTS =')}
- {line}
- {endfor}
-
- LIBS ={for libname in compiler.libraries} -l{libname}{endfor}
- LIBDIRS ={for path in compiler.library_dirs} -L{path}{endfor}
- INCLUDEDIRS ={for path in compiler.include_dirs} -I{path}{endfor}
-
- CFLAGS ={for extra in compiler.compile_extra} {extra}{endfor}
- LDFLAGS ={for extra in compiler.link_extra} {extra}{endfor}
-
- CC = gcc
-
- $(TARGET): $(OBJECTS)
- \t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS)
-
- %.o: %.c
- \t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS)
-
- clean:
- \trm -f $(OBJECTS)
- '''
-
- def gen_makefile(self, targetdir):
- def write_list(lst, prefix):
- for i, fn in enumerate(lst):
- yield '%s %s %s' % (prefix, fn, i < len(list)-1 and '\\' or '')
- prefix = ' ' * len(prefix)
-
- compiler = self.getccompiler(extra_includes=['.'])
- cfiles = []
- ofiles = []
- for fn in compiler.cfilenames:
- fn = py.path.local(fn).basename
- assert fn.endswith('.c')
- cfiles.append(fn)
- ofiles.append(fn[:-2] + '.o')
-
- code_template.Template(MAKEFILE).output(targetdir.join('Makefile'))
-
-
-translator/llvm/module/excsupport.py
-------------------------------------
-
-The original string::
-
- invokeunwind_code = '''
- ccc %(returntype)s%%__entrypoint__%(entrypointname)s {
- %%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception
-
- no_exception:
- store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type
- ret %(returntype)s %%result
-
- exception:
- ret %(noresult)s
- }
-
- ccc int %%__entrypoint__raised_LLVMException() {
- %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type
- %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int
- ret int %%result
- }
-
- internal fastcc void %%unwind() {
- unwind
- }
- '''
-
-Could look something like this if it was used in conjunction with a template::
-
- invokeunwind_code = '''
- ccc {returntype}%__entrypoint__{entrypointname} {
- %result = invoke {cconv} {returntype}%{entrypointname} to label %no_exception except label %exception
-
- no_exception:
- store %RPYTHON_EXCEPTION_VTABLE* null, %RPYTHON_EXCEPTION_VTABLE** %last_exception_type
- ret {returntype} %result
-
- exception:
- ret {noresult}
- }
-
- ccc int %__entrypoint__raised_LLVMException() {
- %tmp = load %RPYTHON_EXCEPTION_VTABLE** %last_exception_type
- %result = cast %RPYTHON_EXCEPTION_VTABLE* %tmp to int
- ret int %result
- }
-
- internal fastcc void %unwind() {
- unwind
- }
- '''
-
-
-Template syntax
-===============
-
-Design decision
----------------
-
-As all programmers must know by now, all the special symbols on the keyboard
-are quite heavily overloaded. Often, template systems work around this fact
-by having special notation like `<*` ... `*>` or {% ... %}. Some template systems
-even have multiple special notations -- one for comments, one for statements,
-one for expressions, etc.
-
-I find these hard to type and ugly. Other markups are either too lightweight,
-or use characters which occur so frequently in the target languages that it
-becomes hard to distinguish marked-up content from content which should be
-rendered as-is.
-
-The compromise taken by *code_template* is to use braces (**{}**) for markup.
-
-This immediately raises the question: what about when the marked-up language
-is C or C++? The answer is that if the leading brace is immediately followed
-by whitespace, it is normal text; if not it is the start of markup.
-
-To support normal text which has a leading brace immediately followed by
-an identifier, if the first whitespace character after the brace is a space
-character (e.g. not a newline or tab), it will be removed from the output.
-
-Examples::
-
- { This is normal text and the space between { and This will be removed}
- {'this must be a valid Python expression' + ' because it is treated as markup'}
- {
- This is normal text, but nothing is altered (the newline is kept intact)
- }
-
- {{1:'Any valid Python expression is allowed as markup'}[1].ljust(30)}
-
-.. _`Code element`:
-
-Elements
---------
-
-Templates consist of normal text and code elements.
-(Comments are considered to be code elements.)
-
-All code elements start with a `left brace`_ which is not followed by
-whitespace.
-
-Keyword element
-~~~~~~~~~~~~~~~
-
-A keyword element is a `code element`_ which starts with a keyword_.
-
-For example, *{if foo}* is a keyword element, but *{foo}* is a `substituted expression`_.
-
-Keyword
-~~~~~~~
-
-A keyword is a word used in `conditional text`_ or in `repeated text`_, e.g.
-one of *if*, *elif*, *else*, *endif*, *for*, or *endfor*.
-
-Keywords are designed to match their Python equivalents. However, since
-templates cannot use spacing to indicate expression nesting, the additional
-keywords *endif* and *endfor* are required.
-
-Left brace
-~~~~~~~~~~
-
-All elements other than normal text start with a left brace -- the symbol '{',
-sometimes known as a 'curly bracket'. A left brace is itself considered
-to be normal text if it is followed by whitespace. If the whitespace starts
-with a space character, that space character will be stripped from the output.
-If the whitespace starts with a tab or linefeed character, the whitespace will
-be left in the output.
-
-Normal Text
-~~~~~~~~~~~
-
-Normal text remains unsubstituted. Transition from text to the other elements
-is effected by use of a `left brace`_ which is not followed by whitespace.
-
-Comment
-~~~~~~~
-
-A comment starts with a left brace followed by an asterisk ('{`*`'), and
-ends with an asterisk followed by a right brace ('`*`}')::
-
- This is a template -- this text will be copied to the output.
- {* This is a comment and this text will not be copied to the output *}
-
- {*
- Comments can span lines,
- but cannot be nested
- *}
-
-Substituted expression
-~~~~~~~~~~~~~~~~~~~~~~
-
-Any python expression may be used::
-
- Dear {record.name},
- we are sorry to inform you that you did not win {record.contest}.
-
-The expression must be surrounded by braces, and there must not be any
-whitespace between the leftmost brace and the start of the expression.
-
-The expression will automatically be converted to a string with str().
-
-Conditional text
-~~~~~~~~~~~~~~~~
-
-The following template has text which is included conditionally::
-
- This text will always be included in the output
- {if foo}
- This text will be included if foo is true
- {elif bar}
- This text will be included if foo is not true but bar is true
- {else}
- This text will be included if neither foo nor bar is true
- {endif}
-
-The {elif} and {else} elements are optional.
-
-Repeated text
-~~~~~~~~~~~~~
-
-The following template shows how to pull multiple items out of a list::
-
- {for student, score in sorted(scorelist)}
- {student.ljust(20)} {score}
- {endfor}
-
-Whitespace removal or modification
-----------------------------------
-
-In general, whitespace in `Normal Text`_ is transferred unchanged to the
-output. There are three exceptions to this rule:
-
-Line separators
-~~~~~~~~~~~~~~~
-
-Each newline is converted to the final output using os.linesep.
-
-Beginning or end of string
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-py.code_template is designed to allow easy use of templates inside of python
-modules. The canonical way to write a template is inside a triple-quoted
-string, e.g.::
-
- my_template = '''
- This is my template. It can have any text at all in it except
- another triple-single-quote.
- '''
-
-To support this usage, if the first character is a newline, it will be
-removed, and if the last line consists solely of whitespace with no
-trailing newline, it will also be removed.
-
-A comment or single keyword element on a line
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Whenever a `keyword element`_ or comment_ is on a line
-*by itself*, that line will not be copied to the output.
-
-This happens when:
- - There is nothing on the line before the keyword element
- or comment except whitespace (spaces and/or tabs).
-
- - There is nothing on the line after the keyword element
- or comment except a newline.
-
-Note that even a multi-line comment or keyword element can
-have the preceding whitespace and subsequent newline stripped
-by this rule.
-
-The primary purpose of this rule is to allow the Python
-programmer to use indentation, **even inside a template**::
-
- This is a template
-
- {if mylist}
- List items:
- {for item in mylist}
- - {item}
- {endfor}
- {endif}
-
-Template usage
-==============
-
-Templates are used by importing the Template class from py.code_template,
-constructing a template, and then sending data with the write() method.
-
-In general, there are four methods for getting the formatted data back out
-of the template object:
-
- - read() reads all the data currently in the object
-
- - output(fobj) outputs the data to a file
-
- fobj can either be an open file object, or a string. If it is
- a string, the file will be opened, written, and closed.
-
- - open(fobj) (or calling the object constructor with a file object)
-
- If the open() method is used, or if a file object is passed to
- the constructor, each write() will automatically flush the data
- out to the file. If the fobj is a string, it is considered to
- be *owned*, otherwise it is considered to be *borrowed*. *Owned*
- file objects are closed when the class is deleted.
-
- - write() can be explicitly called with a file object, in which case
- it will invoke output() on that object after it generates the data.
-
-Template instantiation and methods
-==================================
-
-template = code_template.Template(outf=None, cache=None)
-
-If outf is given, it will be passed to the open() method
-
-cache may be given as a mapping. If not given, the template will use
-the shared default cache. This is not thread safe.
-
-template.open
--------------
-
-template.open(outf, borrowed = None)
-
-The open method closes the internal file object if it was already open,
-and then re-opens it on the given file. It is an error to call open()
-if there is data in the object left over from previous writes. (Call
-output() instead.)
-
-borrowed defaults to 0 if outf is a string, and 1 if it is a file object.
-
-borrowed can also be set explicitly if required.
-
-template.close
---------------
-
-close() disassociates the file from the template, and closes the file if
-it was not borrowed. close() is automatically called by the destructor.
-
-template.write
---------------
-
-template.write(text='', outf=None, dedent=0, localvars=None, globalvars=None,
-framelevel=1)
-
-The write method has the following parameters:
-
- - text is the template itself
-
- - if outf is not None, the output method will be invoked on the object
- after the current template is processed. If no outf is given, data
- will be accumulated internal to the instance until a write() with outf
- is processed, or read() or output() is called, whichever comes first, if
- there is no file object. If there is a file object, data will be flushed
- to the file after every write.
-
- - dedent, if given is applied to each line in the template, to "de-indent"
-
- - localvars and globalvars default to the dictionaries of the caller. A copy
- of localvars is made so that the __TrueSpace__ identifier can be added.
-
- - cache may be given as a mapping. If not given, the template will use
- the shared default cache. This is not thread safe.
-
- - framelevel is used to determine which stackframe to access for globals
- and locals if localvars and/or globalvars are not specified. The default
- is to use the caller's frame.
-
-The write method supports the print >> file protocol by deleting the softspace
-attribute on every invocation. This allows code like::
-
- t = code_template.Template()
- print >> t, "Hello, world"
-
-
-template.read
---------------
-
-This method reads and flushes all accumulated data in the object. Note that
-if a file has been associated with the object, there will never be any data
-to read.
-
-template.output
----------------
-
-This method takes one parameter, outf. template.output() first
-invokes template.read() to read and flush all accumulated data,
-and then outputs the data to the file specified by outf.
-
-If outf has a write() method, that will be invoked with the
-data. If outf has no write() method, it will be treated as
-a filename, and that file will be replaced.
-
-Caching and thread safety
-=========================
-
-The compiled version of every template is cached internal to the
-code_template module (unless a separate cache object is specified).
-
-This allows efficient template reuse, but is not currently thread-safe.
-Alternatively, each invocation of a template object can specify a
-cache object. This is thread-safe, but not very efficient. A shared
-model could be implemented later.
-
Modified: py/dist/py/documentation/coding-style.txt
==============================================================================
--- py/dist/py/documentation/coding-style.txt (original)
+++ py/dist/py/documentation/coding-style.txt Mon Jan 8 12:39:06 2007
@@ -70,4 +70,4 @@
.. _`PEP 8 Style Guide for Python Code`: http://www.python.org/peps/pep-0008.html
.. _`py-dev mailing list`: http://codespeak.net/mailman/listinfo/py-dev
-.. _`future`: future.html
+.. _`future`: future/future.html
Deleted: /py/dist/py/documentation/future.txt
==============================================================================
--- /py/dist/py/documentation/future.txt Mon Jan 8 12:39:06 2007
+++ (empty file)
@@ -1,423 +0,0 @@
-=======================================================
-Visions and ideas for further development of the py lib
-=======================================================
-
-.. contents::
-.. sectnum::
-
-This document tries to describe directions and guiding ideas
-for the near-future development of the py lib. *Note that all
-statements within this document - even if they sound factual -
-mostly just express thoughts and ideas. They not always refer to
-real code so read with some caution. This is not a reference guide
-(tm). Moreover, the order in which appear here in the file does
-not reflect the order in which they may be implemented.*
-
-.. _`general-path`:
-.. _`a more general view on path objects`:
-
-A more general view on ``py.path`` objects
-==========================================
-
-Seen from a more general persective, the current ``py.path.extpy`` path
-offers a way to go from a file to the structured content of
-a file, namely a python object. The ``extpy`` path retains some
-common ``path`` operations and semantics but offers additional
-methods, e.g. ``resolve()`` gets you a true python object.
-
-But apart from python files there are many other examples
-of structured content like xml documents or INI-style
-config files. While some tasks will only be convenient
-to perform in a domain specific manner (e.g. applying xslt
-etc.pp) ``py.path`` offers a common behaviour for
-structured content paths. So far only ``py.path.extpy``
-is implemented and used by py.test to address tests
-and traverse into test files.
-
-*You are in a maze of twisty passages, all alike*
--------------------------------------------------
-
-Now, for the sake of finding out a good direction,
-let's consider some code that wants to find all
-*sections* which have a certain *option* value
-within some given ``startpath``::
-
- def find_option(startpath, optionname):
- for section in startpath.listdir(dir=1):
- opt = section.join(optionname)
- if opt.check(): # does the option exist here?
- print section.basename, "found:", opt.read()
-
-Now the point is that ``find_option()`` would obviously work
-when ``startpath`` is a filesystem-like path like a local
-filesystem path or a subversion URL path. It would then see
-directories as sections and files as option-names and the
-content of the file as values.
-
-But it also works (today) for ``extpy`` paths if you put the following
-python code in a file::
-
- class Section1:
- someoption = "i am an option value"
-
- class Section2:
- someoption = "i am another option value"
-
-An ``extpy()`` path maps classes and modules to directories and
-name-value bindings to file/read() operations.
-
-And it could also work for 'xml' paths if you put
-the following xml string in a file::
-
- <xml ...>
- <root>
- <section1>
- <someoption>value</name></section1>
- <section2>
- <someoption>value</name></section2></root>
-
-where tags containing non-text tags map to directories
-and tags with just text-children map to files (which
-upon read() return the joined content of the text
-tags possibly as unicode.
-
-Now, to complete the picture, we could make Config-Parser
-*ini-style* config files also available::
-
- [section1]
- name = value
-
- [section2]
- othername = value
-
-where sections map to directories and name=value mappings
-to file/contents.
-
-So it seems that our above ``find_option()`` function would
-work nicely on all these *mappings*.
-
-Of course, the somewhat open question is how to make the
-transition from a filesystem path to structured content
-useful and unified, as much as possible without overdoing it.
-
-Again, there are tasks that will need fully domain specific
-solutions (DOM/XSLT/...) but i think the above view warrants
-some experiments and refactoring. The degree of uniformity
-still needs to be determined and thought about.
-
-path objects should be stackable
---------------------------------
-
-Oh, and btw, a ``py.path.extpy`` file could live on top of a
-'py.path.xml' path as well, i.e. take::
-
- <xml ...>
- <code>
- <py>
- <magic>
- <assertion>
- import py
- ... </assertion>
- <exprinfo>
- def getmsg(x): pass </exprino></magic></py></code>
-
-and use it to have a ``extpy`` path living on it::
-
- p = py.path.local(xmlfilename)
- xmlp = py.path.extxml(p, 'py/magic/exprinfo')
- p = py.path.extpy(xmlp, 'getmsg')
-
- assert p.check(func=1, basename='getmsg')
- getmsg = p.resolve()
- # we now have a *live* getmsg() function taken and compiled from
- # the above xml fragment
-
-There could be generic converters which convert between
-different content formats ... allowing configuration files to e.g.
-be in XML/Ini/python or filesystem-format with some common way
-to find and iterate values.
-
-*After all the unix filesystem and the python namespaces are
-two honking great ideas, why not do more of them? :-)*
-
-
-.. _importexport:
-
-Revising and improving the import/export system
-===============================================
-
- or let's wrap the world all around
-
-the export/import interface
----------------------------
-
-The py lib already incorporates a mechanism to select which
-namespaces and names get exposed to a user of the library.
-Apart from reducing the outside visible namespaces complexity
-this allows to quickly rename and refactor stuff in the
-implementation without affecting the caller side. This export
-control can be used by other python packages as well.
-
-However, all is not fine as the import/export has a
-few major deficiencies and shortcomings:
-
-- it doesn't allow to specify doc-strings
-- it is a bit hackish (see py/initpkg.py)
-- it doesn't present a complete and consistent view of the API.
-- ``help(constructed_namespace)`` doesn't work for the root
- package namespace
-- when the py lib implementation accesses parts of itself
- it uses the native python import mechanism which is
- limiting in some respects. Especially for distributed
- programs as encouraged by `py.execnet`_ it is not clear
- how the mechanism can nicely integrate to support remote
- lazy importing.
-
-Discussions have been going on for a while but it is
-still not clear how to best tackle the problem. Personally,
-i believe the main missing thing for the first major release
-is the docstring one. The current specification
-of exported names is dictionary based. It would be
-better to declare it in terms of Objects.
-
-
-Example sketch for a new export specification
----------------------------------------------
-
-Here is a sketch of how the py libs ``__init__.py`` file
-might or should look like::
-
- """
- the py lib version 0.8
- http://codespeak.net/py/0.8
- """
-
- from py import pkg
- pkg.export(__name__,
- pkg.Module('path',
- '''provides path objects for local filesystem,
- subversion url and working copy, and extension paths.
- ''',
- pkg.Class('local', '''
- the local filesystem path offering a single
- point of interaction for many purposes.
- ''', extpy='./path/local.LocalPath'),
-
- pkg.Class('svnurl', '''
- the subversion url path.
- ''', extpy='./path/local/svn/urlcommand.SvnUrlPath'),
- ),
- # it goes on ...
- )
-
-The current ``initpkg.py`` code can be cleaned up to support
-this new more explicit style of stating things. Note that
-in principle there is nothing that stops us from retrieving
-implementations over the network, e.g. a subversion repository.
-
-
-Let there be alternatives
--------------------------
-
-We could also specify alternative implementations easily::
-
- pkg.Class('svnwc', '''
- the subversion working copy.
- ''', extpy=('./path/local/svn/urlbinding.SvnUrlPath',
- './path/local/svn/urlcommand.SvnUrlPath',)
- )
-
-This would prefer the python binding based implementation over
-the one working through he 'svn' command line utility. And
-of course, it could uniformly signal if no implementation is
-available at all.
-
-
-Problems problems
------------------
-
-Now there are reasons there isn't a clear conclusion so far.
-For example, the above approach has some implications, the
-main one being that implementation classes like
-``py/path/local.LocalPath`` are visible to the caller side but
-this presents an inconsistency because the user started out with
-``py.path.local`` and expects that the two classes are really much
-the same. We have the same problem today, of course.
-
-The naive solution strategy of wrapping the "implementation
-level" objects into their exported representations may remind
-of the `wrapping techniques PyPy uses`_. But it
-*may* result in a slightly heavyweight mechanism that affects
-runtime speed. However, I guess that this standard strategy
-is probably the cleanest.
-
-
-Every problem can be solved with another level ...
---------------------------------------------------
-
-The wrapping of implementation level classes in their export
-representations objects adds another level of indirection.
-But this indirection would have interesting advantages:
-
-- we could easily present a consistent view of the library
-- it could take care of exceptions as well
-- it provides natural interception points for logging
-- it enables remote lazy loading of implementations
- or certain versions of interfaces
-
-And quite likely the extra indirection wouldn't hurt so much
-as it is not much more than a function call and we cared
-we could even generate some c-code (with PyPy :-) to speed
-it up.
-
-But it can lead to new problems ...
------------------------------------
-
-However, it is critical to avoid to burden the implementation
-code of being aware of its wrapping. This is what we have
-to do in PyPy but the import/export mechanism works at
-a higher level of the language, i think.
-
-Oh, and we didn't talk about bootstrapping :-)
-
-.. _`py.execnet`: execnet.html
-.. _`wrapping techniques PyPy uses`: http://codespeak.net/pypy/index.cgi?doc/wrapping.html
-.. _`lightweight xml generation`:
-
-Extension of py.path.local.sysexec()
-====================================
-
-The `sysexec mechanism`_ allows to directly execute
-binaries on your system. Especially after we'll have this
-nicely integrated into Win32 we may also want to run python
-scripts both locally and from the net::
-
- vadm = py.path.svnurl('http://codespeak.net/svn/vadm/dist/vadm/cmdline.py')
- stdoutput = vadm.execute('diff')
-
-To be able to execute this code fragement, we need either or all of
-
-- an improved import system that allows remote imports
-
-- a way to specify what the "neccessary" python import
- directories are. for example, the above scriptlet will
- require a certain root included in the python search for module
- in order to execute something like "import vadm".
-
-- a way to specify dependencies ... which opens up another
- interesting can of worms, suitable for another chapter
- in the neverending `future book`_.
-
-.. _`sysexec mechanism`: misc.html#sysexec
-.. _`compile-on-the-fly`:
-
-we need a persistent storage for the py lib
--------------------------------------------
-
-A somewhat open question is where to store the underlying
-generated pyc-files and other files generated on the fly
-with `CPython's distutils`_. We want to have a
-*persistent location* in order to avoid runtime-penalties
-when switching python versions and platforms (think NFS).
-
-A *persistent location* for the py lib would be a good idea
-maybe also for other reasons. We could cache some of the
-expensive test setups, like the multi-revision subversion
-repository that is created for each run of the tests.
-
-.. _`CPython's distutils`: http://www.python.org/dev/doc/devel/lib/module-distutils.html
-
-.. _`getting started`: getting-started.html
-.. _`restructured text`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
-.. _`python standard library`: http://www.python.org/doc/2.3.4/lib/lib.html
-.. _`xpython EuroPython 2004 talk`: http://codespeak.net/svn/user/hpk/talks/xpython-talk.txt
-.. _`under the xpy tree`: http://codespeak.net/svn/user/hpk/xpy/xml.py
-.. _`future book`: future.html
-.. _`PEP-324 subprocess module`: http://www.python.org/peps/pep-0324.html
-.. _`subprocess implementation`: http://www.lysator.liu.se/~astrand/popen5/
-.. _`py.test`: test.html
-
-Refactor path implementations to use a Filesystem Abstraction
-=============================================================
-
-It seems like a good idea to refactor all python implementations to
-use an internal Filesystem abstraction. The current code base
-would be transformed to have Filesystem implementations for e.g.
-local, subversion and subversion "working copy" filesystems. Today
-the according code is scattered through path-handling code.
-
-On a related note, Armin Rigo has hacked `pylufs`_ which allows to
-implement kernel-level linux filesystems with pure python. Now
-the idea is that the mentioned filesystem implementations would
-be directly usable for such linux-filesystem glue code.
-
-In other words, implementing a `memoryfs`_ or a `dictfs`_ would
-give you two things for free: a filesystem mountable at kernel level
-as well as a uniform "path" object allowing you to access your
-filesystem in convenient ways. (At some point it might
-even become interesting to think about interfacing to
-`reiserfs v4 features`_ at the Filesystem level but that
-is a can of subsequent worms).
-
-.. _`memoryfs`: http://codespeak.net/svn/user/arigo/hack/pyfuse/memoryfs.py
-.. _`dictfs`: http://codespeak.net/pipermail/py-dev/2005-January/000191.html
-.. _`pylufs`: http://codespeak.net/svn/user/arigo/hack/pylufs/
-.. _`reiserfs v4 features`: http://www.namesys.com/v4/v4.html
-
-
-Improve and unify Path API
-==========================
-
-visit() grows depth control
----------------------------
-
-Add a ``maxdepth`` argument to the path.visit() method,
-which will limit traversal to subdirectories. Example::
-
- x = py.path.local.get_tmproot()
- for x in p.visit('bin', stop=N):
- ...
-
-This will yield all file or directory paths whose basename
-is 'bin', depending on the values of ``stop``::
-
- p # stop == 0 or higher (and p.basename == 'bin')
- p / bin # stop == 1 or higher
- p / ... / bin # stop == 2 or higher
- p / ... / ... / bin # stop == 3 or higher
-
-The default for stop would be `255`.
-
-But what if `stop < 0`? We could let that mean to go upwards::
-
- for x in x.visit('py/bin', stop=-255):
- # will yield all parent direcotires which have a
- # py/bin subpath
-
-visit() returning a lazy list?
-------------------------------
-
-There is a very nice "no-API" `lazy list`_ implementation from
-Armin Rigo which presents a complete list interface, given some
-iterable. The iterable is consumed only on demand and retains
-memory efficiency as much as possible. The lazy list
-provides a number of advantages in addition to the fact that
-a list interface is nicer to deal with than an iterator.
-For example it lets you do::
-
- for x in p1.visit('*.cfg') + p2.visit('*.cfg'):
- # will iterate through all results
-
-Here the for-iter expression will retain all lazyness (with
-the result of adding lazy lists being another another lazy
-list) by internally concatenating the underlying
-lazylists/iterators. Moreover, the lazylist implementation
-will know that there are no references left to the lazy list
-and throw away iterated elements. This makes the iteration
-over the sum of the two visit()s as efficient as if we had
-used iterables to begin with!
-
-For this, we would like to move the lazy list into the
-py lib's namespace, most probably at `py.builtin.lazylist`.
-
-.. _`lazy list`: http://codespeak.net/svn/user/arigo/hack/misc/collect.py
Copied: py/dist/py/documentation/future/future.txt (from r36246, py/dist/py/documentation/future.txt)
==============================================================================
--- py/dist/py/documentation/future.txt (original)
+++ py/dist/py/documentation/future/future.txt Mon Jan 8 12:39:06 2007
@@ -280,7 +280,7 @@
Oh, and we didn't talk about bootstrapping :-)
-.. _`py.execnet`: execnet.html
+.. _`py.execnet`: ../execnet.html
.. _`wrapping techniques PyPy uses`: http://codespeak.net/pypy/index.cgi?doc/wrapping.html
.. _`lightweight xml generation`:
@@ -308,7 +308,7 @@
interesting can of worms, suitable for another chapter
in the neverending `future book`_.
-.. _`sysexec mechanism`: misc.html#sysexec
+.. _`sysexec mechanism`: ../misc.html#sysexec
.. _`compile-on-the-fly`:
we need a persistent storage for the py lib
@@ -327,7 +327,7 @@
.. _`CPython's distutils`: http://www.python.org/dev/doc/devel/lib/module-distutils.html
-.. _`getting started`: getting-started.html
+.. _`getting started`: ../getting-started.html
.. _`restructured text`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
.. _`python standard library`: http://www.python.org/doc/2.3.4/lib/lib.html
.. _`xpython EuroPython 2004 talk`: http://codespeak.net/svn/user/hpk/talks/xpython-talk.txt
@@ -335,7 +335,7 @@
.. _`future book`: future.html
.. _`PEP-324 subprocess module`: http://www.python.org/peps/pep-0324.html
.. _`subprocess implementation`: http://www.lysator.liu.se/~astrand/popen5/
-.. _`py.test`: test.html
+.. _`py.test`: ../test.html
Refactor path implementations to use a Filesystem Abstraction
=============================================================
Modified: py/dist/py/documentation/getting-started.txt
==============================================================================
--- py/dist/py/documentation/getting-started.txt (original)
+++ py/dist/py/documentation/getting-started.txt Mon Jan 8 12:39:06 2007
@@ -104,7 +104,7 @@
.. _`zope3`: http://zope3.zwiki.org/
.. _twisted: http://www.twistedmatrix.org
-.. _future: future.html
+.. _future: future/future.html
.. _`get an account`:
Modified: py/dist/py/documentation/home.txt
==============================================================================
--- py/dist/py/documentation/home.txt (original)
+++ py/dist/py/documentation/home.txt Mon Jan 8 12:39:06 2007
@@ -4,9 +4,3 @@
documentation issues - seen primarily from the perspective
of a FOSS (Free and Open Source) developer.
-Heading towards a first 0.8.0 py.test/py lib release!
-======================================================
-
-``py.test`` and the py lib are heading towards their first release.
-The main missing feature is proper testing and integration with win32
-platforms and improved documentation. *holger (06/11/2005)*
Modified: py/dist/py/documentation/index.txt
==============================================================================
--- py/dist/py/documentation/index.txt (original)
+++ py/dist/py/documentation/index.txt Mon Jan 8 12:39:06 2007
@@ -27,7 +27,7 @@
.. _`py.test`: test.html
.. _`py.xml`: xml.html
.. _`Why What how py?`: why_py.html
-.. _`future`: future.html
+.. _`future`: future/future.html
.. _`getting started`: getting-started.html
.. _`miscellaneous features`: misc.html
Modified: py/dist/py/documentation/misc.txt
==============================================================================
--- py/dist/py/documentation/misc.txt (original)
+++ py/dist/py/documentation/misc.txt Mon Jan 8 12:39:06 2007
@@ -136,11 +136,11 @@
will have to install the `pywin32 package`_.
-.. _`compile-on-the-fly`: future.html#compile-on-the-fly
-.. _`future book`: future.html
+.. _`compile-on-the-fly`: future/future.html#compile-on-the-fly
+.. _`future book`: future/future.html
.. _`PEP-324 subprocess module`: http://www.python.org/peps/pep-0324.html
.. _`subprocess implementation`: http://www.lysator.liu.se/~astrand/popen5/
-.. _`a more general view on path objects`: future.html#general-path
+.. _`a more general view on path objects`: future/future.html#general-path
.. _`pywin32 package`: http://pywin32.sourceforge.net/
finding an executable local path
Modified: py/dist/py/documentation/why_py.txt
==============================================================================
--- py/dist/py/documentation/why_py.txt (original)
+++ py/dist/py/documentation/why_py.txt Mon Jan 8 12:39:06 2007
@@ -266,8 +266,8 @@
.. _`py-dev mailing list`: http://codespeak.net/mailman/listinfo/py-dev
.. _`test environment`: test.html
.. _`PyPy`: http://codespeak.net/pypy
-.. _`envisioned import/export system`: future.html#importexport
-.. _future: future.html
+.. _`envisioned import/export system`: future/future.html#importexport
+.. _future: future/future.html
.. _`py.test tool and library`: test.html
.. _`py API`: api.html
.. _`great work that Fred L. Drake, Jr. is doing`: http://www.python.org/doc/2.3.4/lib/lib.html
More information about the py-svn
mailing list