[Cython] cython / numpy / mac(?) issue

[Zachary Pincus]

Hello all,

I've run into a problem using some cython+numpy code that some folks  
recently made available. After some back and forth with the original  
authors, we've determined that the problem appears to be with cython  
on my box. I've attached the offending code and will post a test-case  
below, but first some configuration details:

I'm using OS X 10.5.6 and a python 2.5.2 that's installed as a  
"framework" (the standard OS X way of installing Python) -- the only  
major difference is that the python directories live in /Library/ 
Frameworks/[etc.] instead of /usr/local.

First, I should note that on my system, using both the latest stable  
Cython release as well as the tip of cython-devel from hg, cython's  
numpy tests fail utterly, due to some sort of error with include paths:
hg clone http://hg.cython.org/cython-devel/
cd cython-devel
make local
make test
[...output...]
compiling (c) and running numpy_test ... numpy_test.c: 
129:31:numpy_test.c:129:31:  error: error: numpy/arrayobject.h: No  
such file or directorynumpy/arrayobject.h: No such file or directory

The error using cython 0.10.3 is pretty much the same, except with the  
spacing errors fixed:
compiling (c) and running numpy_test ... numpy_test.c:112:31: error:  
numpy/arrayobject.h: No such file or directory

Also, I should point out that the default OS X filesystem is case- 
insensitive, so using a directory named 'BUILD' for building the tests  
is not a great idea, as it is the same as the python 'build'  
directory, so the latter gets nuked by the 'rm -rf BUILD' in 'make  
test'.

Anyhow, if I install cython despite these issues, I can build numpy- 
using projects fine... for some reason, the right include directories  
get pulled in. However, thereafter I have further issues.

I've attached the troublesome code; it's pretty simple but I don't  
really know enough cython yet to pull it into a minimal test case. (I  
suspect from the nature of the errors that this should be easy.)

Anyhow, here's the test case and error:
cd stefanv-bilateral
python setup.py build_ext -i
python

Python 2.5.2 (r252:60911, Feb 22 2008, 07:57:53)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import numpy
 >>> print numpy.version.version
1.3.0.dev6359
 >>> import Cython.Compiler.Version
 >>> print Cython.Compiler.Version.version
0.11.beta
 >>> import bilateral
 >>> bilateral.bilateral(numpy.zeros((4,4)), 4, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "bilateral/bilateral.py", line 38, in bilateral
    filter_fcn = _BB.Bilat_fcn(xy_sig, z_sig, filter_size)
  File "bilateral_base.pyx", line 40, in  
bilateral.bilateral_base.Bilat_fcn.__init__ (bilateral/ 
bilateral_base.c:752)
NameError: np

Now, bilateral_base.pyx has the line 'cimport numpy as np' at the top,  
but somehow a NameError gets thrown the first time that 'np' is used,  
at the end of this snippet:
    def __init__(self, spat_sig, inten_sig, filter_size=None):
        if filter_size is not None and filter_size >= 2:
            self.xy_size = int(filter_size)
        else:
            self.xy_size = int(round(spat_sig*4))
            # Make filter size odd
            self.xy_size += 1-self.xy_size%2
        x = np.arange(self.xy_size, dtype=float)


Just for fun, I tried adding 'import numpy as np' to  
bilateral_base.pyx after the cimport line (this is what numpy_text.pyx  
does in the cython tests). Here then is the error that I get:

 >>> bilateral.bilateral(numpy.zeros((4,4)), 4, 4)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "bilateral/bilateral.py", line 40, in bilateral
    return generic_filter(mat, filter_fcn.cfilter, size=size, mode=mode)
  File "/Library/Frameworks/Python.framework/Versions/2.5/lib/ 
python2.5/site-packages/scipy/ndimage/filters.py", line 992, in  
generic_filter
    cval, origins, extra_arguments, extra_keywords)
  File "bilateral_base.pyx", line 73, in  
bilateral.bilateral_base.Bilat_fcn.cfilter (bilateral/bilateral_base.c: 
1443)
AttributeError: 'numpy.ndarray' object has no attribute 'dimensions'

This corresponds to the last line in this snippet:
    def cfilter(self, np.ndarray data):
        'An optimized implementation'
        cdef np.ndarray kernel = self.xy_ker
        cdef double sigma   = self.inten_sig
        cdef double weight_i, weight, result, centre, dat_i
        cdef double *pdata=<double *>data.data, *pker=<double  
*>kernel.data
        cdef int i, dim = data.dimensions[0]

Now, numpy arrays don't expose a dimensions attribute in python, but  
the array struct has such a member, which I suppose should be exposed  
to cython via 'cimport numpy'.

So, clearly something is going wrong with cython's numpy support on my  
system -- the tests fail, as well as actual use of numpy, but the  
failures are different. The authors of this code can't reproduce this  
with the same versions of cython and numpy on their linux systems, so  
I suspect that this might be a mac issue, perhaps relating to some  
assumptions about where the numpy headers live?

On a framework install, Python.h lives in:
/Library/Frameworks/Python.framework/Versions/2.5/include
but numpy/arrayobject.h is in:
/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site- 
packages/numpy/core/include


Can anyone offer any suggestions here?

Thanks,


Zach Pincus
Postdoctoral Fellow, lab of Frank Slack
Molecular, Cellular and Developmental Biology
Yale University




-------------- next part --------------
A non-text attachment was scrubbed...
Name: stefanv-bilateral.zip
Type: application/zip
Size: 3225 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20090301/ced4ab19/attachment-0001.zip