[Cython] Problems with creating a package of extension modules

[Simon King]

Dear Robert,

Zitat von Robert Bradshaw <robertwb at math.washington.edu>:
> Hmm... could you point me to that bit of documentation? I think it's
> wrong (or at least not implemented as explained), we do Extension
> ("package.module",...) all the time in sage and that seems to mix
> well with pickling.

Admittedly it is not Cython-documentation, but Python:
At http://docs.python.org/distutils/setupscript.html, they say:

---------------------
If you have a number of extensions all in the same package (or all under the
same base package), use the ext_package keyword argument to setup(). For
example,

setup(...,
      ext_package='pkg',
      ext_modules=[Extension('foo', ['foo.c']),
                   Extension('subpkg.bar', ['bar.c'])],
     )

will compile foo.c to the extension pkg.foo, and bar.c to pkg.subpkg.bar.
---------------------

Anyway. As I mentioned in my original post, I also tried it *without*
ext_package but with explicit package names.

Five minutes ago I found that (as a leftover of a previous built) I had mtx.so
not only in the pGroupCohomology folder, but also one level up. Perhaps this
was confusing for Sage.

A propos confusing: It is still not clear to me what the role of the
setup-options "name", "packages", and "ext_package". In order to learn (one of
the main advantages of open source, imho), I was looking at the setup.py of the
optional pil-package for sage. They have
        name="PIL",
        package_dir={"": "PIL"},
        packages=[""],
        ext_modules = [Extension("_imaging", ["_imaging.c"])], # dummy
and apparently it results in _imaging.so in the folder PIL.

If I do, analogously,
  name='pGroupCohomology',
  packages=[""],
  package_dir={"":"CohoSrc"},
    Extension("mtx",
then my (dummy) __init__.py is copied at the top level (folder
SAGE_LOCAL/lib/python2.5/site-packages/), and in the same folder mtx.so shows
up -- I hope this did not destroy something important. Anyway, I wonder why the
PIL package works.

If I do
  name='pGroupCohomology'
  packages=[""],
  package_dir={"":"CohoSrc"},
    Extension("pGroupCohomology.mtx",
then still my __init__.py runs amok. mtx.so seems in the correct place, but of
course I can't import mtx if there is no __init__.py in the package folder.

If I do
  name='pGroupCohomology'
  packages=["pGroupCohomology"],
  package_dir={"pGroupCohomology":"CohoSrc"},
  ext_package="pGroupCohomology",
    Extension("mtx",
then everything seems in the right place, in accordance with the above-mentioned
documentation. I can import mtx
 sage: from pGroupCohomology.mtx import MTX
and construct some instance M, but again
 sage: M.__class__
 <type 'mtx.MTX'>
and
 sage: save(M,'Test')
 ---------------------------------------------------------------------------
 PicklingError                             Traceback (most recent call last)
 ...
 PicklingError: Can't pickle mtx.MTX_unpickle_class: import of module mtx failed

Note that the error is slightly different from my original post. Apparently, at
that time, I had a copy of mtx.so in the site-packages top folder, so that the
import of mtx worked, but was of course not the same as
pGroupCohomology/mtx.so.

Conclusion:
- I have a folder pGroupCohomology in the site-packages,
- it contains __init__.py, hence, gives rise to a package
- it contains the module mtx.so
- but mtx.so believes that it is not contained in a package.
Why?

Cheers,
  Simon


----------------------------------------------------------------
This mail was sent through http://webmail.uni-jena.de