From dalcinl at gmail.com  Wed Oct  1 00:53:48 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 30 Sep 2008 19:53:48 -0300
Subject: [Cython] cython-devel broken for mpi4py
Message-ID: <e7ba66e40809301553w40360d41l2e0e5a382283e38e@mail.gmail.com>

I just got this, sorry, not time today to figure out what's going on:

Error converting Pyrex file to C:
------------------------------------------------------------
...

    int MPI_Type_get_attr(MPI_Datatype, int, void* , int*)
    int MPI_Type_set_attr(MPI_Datatype, int, void*)
    int MPI_Type_delete_attr(MPI_Datatype, int)

    ctypedef int
(MPI_Type_copy_attr_function)(MPI_Datatype,int,void*,void*,void*,int*)
                ^
------------------------------------------------------------

include/mpi4py/mpi.pxi:182:17: Function cannot return a function
Traceback (most recent call last):
  File "/u/dalcinl/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/u/dalcinl/lib/python/Cython/Compiler/Main.py", line 701, in main
    result = compile(sources, options)
  File "/u/dalcinl/lib/python/Cython/Compiler/Main.py", line 678, in compile
    return compile_multiple(source, options)
  File "/u/dalcinl/lib/python/Cython/Compiler/Main.py", line 648, in
compile_multiple
    result = run_pipeline(source, options)
  File "/u/dalcinl/lib/python/Cython/Compiler/Main.py", line 510, in
run_pipeline
    err, enddata = context.run_pipeline(pipeline, source)
  File "/u/dalcinl/lib/python/Cython/Compiler/Main.py", line 172, in
run_pipeline
    data = phase(data)
  File "/u/dalcinl/lib/python/Cython/Compiler/ParseTreeTransforms.py",
line 491, in __call__
    return super(AnalyseDeclarationsTransform, self).__call__(root)
  File "/u/dalcinl/lib/python/Cython/Compiler/Visitor.py", line 170, in __call__
    return super(CythonTransform, self).__call__(node)
  File "/u/dalcinl/lib/python/Cython/Compiler/Visitor.py", line 156, in __call__
    return self.visit(root)
  File "/u/dalcinl/lib/python/Cython/Compiler/Visitor.py", line 36, in visit
    return m(obj)
  File "/u/dalcinl/lib/python/Cython/Compiler/ParseTreeTransforms.py",
line 494, in visit_ModuleNode
    node.analyse_declarations(self.env_stack[-1])
  File "/u/dalcinl/lib/python/Cython/Compiler/ModuleNode.py", line 57,
in analyse_declarations
    self.body.analyse_declarations(env)
  File "/u/dalcinl/lib/python/Cython/Compiler/Nodes.py", line 312, in
analyse_declarations
    stat.analyse_declarations(env)
  File "/u/dalcinl/lib/python/Cython/Compiler/Nodes.py", line 312, in
analyse_declarations
    stat.analyse_declarations(env)
  File "/u/dalcinl/lib/python/Cython/Compiler/Nodes.py", line 367, in
analyse_declarations
    self.body.analyse_declarations(env)
  File "/u/dalcinl/lib/python/Cython/Compiler/Nodes.py", line 312, in
analyse_declarations
    stat.analyse_declarations(env)
  File "/u/dalcinl/lib/python/Cython/Compiler/Nodes.py", line 864, in
analyse_declarations
    cname = cname, visibility = self.visibility)
  File "/u/dalcinl/lib/python/Cython/Compiler/Symtab.py", line 328, in
declare_typedef
    type.qualified_name = entry.qualified_name
AttributeError: Entry instance has no attribute 'qualified_name'


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From fperez.net at gmail.com  Wed Oct  1 01:59:01 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 30 Sep 2008 16:59:01 -0700
Subject: [Cython] docs.cython.org
In-Reply-To: <E7F6549F-0E9E-4D02-A25F-1A02C0B86357@math.washington.edu>
References: <20080927144913.GA20932@encolpuis>
	<85b5c3130809290630w4c1055dma2ce4a4fbb7d5831@mail.gmail.com>
	<20080929142813.GA8804@encolpuis>
	<85b5c3130809290735r62bb6434oe725c8eb2df24f2b@mail.gmail.com>
	<476444D5-5A11-444B-879D-867BFE0A34BE@math.washington.edu>
	<20080929215931.GA11292@encolpuis>
	<48E1D66D.7080408@student.matnat.uio.no>
	<E7F6549F-0E9E-4D02-A25F-1A02C0B86357@math.washington.edu>
Message-ID: <db6b5ecc0809301659x225ca128k93da882bf9c5d465@mail.gmail.com>

On Tue, Sep 30, 2008 at 10:35 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:

>> b) A script to build them is shipped with Cython (setup.py could do
>> this
>> I suppose)
>
> If Sphinx is detected, and it's fast. I certainly don't want it to be
> a requirement.

Please feel free to pillage anything from the ipython sources you may
want for this.  We build the docs only when
'sdist' or 'bdist_rpm' are given, so that only package distributors
need to have the toolchain installed:

http://bazaar.launchpad.net/%7Eipython-dev/ipython/trunk/annotate/1147?file_id=setup.py-20080216095032-xb0is4a97lmosv2z-14

lines 97-120.

There's also a bit of nasty trickery in

http://bazaar.launchpad.net/%7Eipython-dev/ipython/trunk/annotate/1147?file_id=setupbase.py-20080606221622-xwbqrw3mlwq01vyb-1

line 146, func make_dir_struct() to be able to easily produce the
necessary data layout for distutils to ship the nested doc tree.

You guys already have the makefile and sphinx code, but take any of
this you may find useful.

Anything for better docs :)

Cheers,

f

From uschmitt at mineway.de  Wed Oct  1 13:08:46 2008
From: uschmitt at mineway.de (Uwe Schmitt)
Date: Wed, 01 Oct 2008 13:08:46 +0200
Subject: [Cython] pyximport
Message-ID: <48E35A3E.5080403@mineway.de>

Hi,

I had trouble to import pyximport and found
only two postings about this problem from august.

The problem was that "import pyximport" fails,
which I could fix (as suggested in the postings
above) by making pyximport/ a python packgage,
which means I had to add a "__init__.py" file to
.../site-packages/cython.../pyximport/

I wonder that I found not more about that.
Is nobody using pyximport or did I miss something ?


Greetings, Uwe

-- 
Dr. rer. nat. Uwe Schmitt
F&E Mathematik
 
mineway GmbH
Science Park 2
D-66123 Saarbr?cken
 
Telefon: +49 (0)681 8390 5334
Telefax: +49 (0)681 830 4376
 
uschmitt at mineway.de
www.mineway.de
 
Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer
Amtsgericht Saarbr?cken HRB 12339



From dalcinl at gmail.com  Wed Oct  1 15:47:31 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 10:47:31 -0300
Subject: [Cython] pyximport
In-Reply-To: <48E35A3E.5080403@mineway.de>
References: <48E35A3E.5080403@mineway.de>
Message-ID: <e7ba66e40810010647g707314feqb0c2076198b22997@mail.gmail.com>

On Wed, Oct 1, 2008 at 8:08 AM, Uwe Schmitt <uschmitt at mineway.de> wrote:
> The problem was that "import pyximport" fails,
> which I could fix (as suggested in the postings
> above) by making pyximport/ a python packgage,
> which means I had to add a "__init__.py" file to
> .../site-packages/cython.../pyximport/

Yes, add a __init__.py file with the following line inside 'from
pyximport import *'

This still need to be fixed in cython-devel repo. I'll try to make a patch.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Oct  1 15:52:57 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 10:52:57 -0300
Subject: [Cython] PATCH: fix pyximport installation as a package
Message-ID: <e7ba66e40810010652u77fe0376ud388b549a2ea00e@mail.gmail.com>

This should fix the installation of pyximport as a package. Please,
review and apply.



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pyximport.diff
Type: text/x-patch
Size: 382 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081001/a79fa014/attachment.bin 

From dalcinl at gmail.com  Wed Oct  1 20:13:00 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 15:13:00 -0300
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
Message-ID: <e7ba66e40810011113r653738b7ve21afe5b4803973@mail.gmail.com>

Consider the following code inside a one-line pyx file:

cdef object someint = 7

Then Cython generates the following inside the module init function:

  /*--- Global init code ---*/
  __pyx_v_9refleaks2_someint = Py_None; Py_INCREF(Py_None);

  /* "/u/dalcinl/Devel/Cython/sandbox/refleaks2.pyx":1
 * cdef object someint = 7             # <<<<<<<<<<<<<<
 *
 */
  Py_INCREF(__pyx_int_7);
  __pyx_v_9refleaks2_someint = __pyx_int_7;


Clearly, Py_None references are being leaked. All this is because of
bad interaction between this two methods:

* ModuleNode.generate_global_init_code(...) (in ModuleNode.py)
* FinalOptimizePhase.visit_SingleAssignmentNode(...) (in Optimize.py)



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Oct  1 20:19:53 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 15:19:53 -0300
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
In-Reply-To: <e7ba66e40810011113r653738b7ve21afe5b4803973@mail.gmail.com>
References: <e7ba66e40810011113r653738b7ve21afe5b4803973@mail.gmail.com>
Message-ID: <e7ba66e40810011119o3b5fbcbi3fff71fd606b71b8@mail.gmail.com>

BTW, this patch fix the issue by not initializing cdef'ed module
globals to None. But this is not the safest approach (I can imagine
some nasty ways to originate segfaults). Perhaps we should try to fix
the isue in FinalOptimizePhase.visit_SingleAssignmentNode

On Wed, Oct 1, 2008 at 3:13 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Consider the following code inside a one-line pyx file:
>
> cdef object someint = 7
>
> Then Cython generates the following inside the module init function:
>
>  /*--- Global init code ---*/
>  __pyx_v_9refleaks2_someint = Py_None; Py_INCREF(Py_None);
>
>  /* "/u/dalcinl/Devel/Cython/sandbox/refleaks2.pyx":1
>  * cdef object someint = 7             # <<<<<<<<<<<<<<
>  *
>  */
>  Py_INCREF(__pyx_int_7);
>  __pyx_v_9refleaks2_someint = __pyx_int_7;
>
>
> Clearly, Py_None references are being leaked. All this is because of
> bad interaction between this two methods:
>
> * ModuleNode.generate_global_init_code(...) (in ModuleNode.py)
> * FinalOptimizePhase.visit_SingleAssignmentNode(...) (in Optimize.py)
>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fixinitnoneleaks.diff
Type: text/x-patch
Size: 1166 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081001/8c5e15b7/attachment.bin 

From thomas.e.keller at gmail.com  Wed Oct  1 20:29:02 2008
From: thomas.e.keller at gmail.com (Thomas Keller)
Date: Wed, 1 Oct 2008 13:29:02 -0500
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
Message-ID: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>

This may be a foolish question, but I'm still getting used to to
Cython. I was under the impression that you can cdef all the basic
python types like list, dict,tuple etc. However, when I try to cdef variables
of type set, Cython gives a compile error. Is it simply unsupported at
this time or am I completely missing something? I imagine either way
it wouldn't speed things up that much.
Regards,
Thomas Keller

From dalcinl at gmail.com  Wed Oct  1 21:08:57 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 16:08:57 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
Message-ID: <e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>

Dear Thomas,

This should be fixed in 'cython-devel' repo. All numeric, string and
container builtin types (plus slice and file) are supported for all
Python versions. Let us know if you give a try to cython-devel and
something does not work.

Of course,  in Py2.3 your C compiler will fail if you use 'set',
perhaps this could be emulated somewhat with sets.Set and
set.InmutableSet, but I do not tried that when I wrote the patch
fixing your problem.

Regards,

On Wed, Oct 1, 2008 at 3:29 PM, Thomas Keller <thomas.e.keller at gmail.com> wrote:
> This may be a foolish question, but I'm still getting used to to
> Cython. I was under the impression that you can cdef all the basic
> python types like list, dict,tuple etc. However, when I try to cdef variables
> of type set, Cython gives a compile error. Is it simply unsupported at
> this time or am I completely missing something? I imagine either way
> it wouldn't speed things up that much.
> Regards,
> Thomas Keller
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Wed Oct  1 21:28:00 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: 01 Oct 2008 21:28:00 +0200
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
Message-ID: <3305741288.1706043@smtp.netcom.no>

I say let them leak? The only real risk (that I can see at least) is overflowing the refcount, and if I understand and remember an informal chat correctly even that is eliminated in newer Python versions where Py_None and friends have special deallocation which never deallocate...

Dag Sverre Seljebotn
-----Original Message-----
From: "Lisandro Dalcin" <dalcinl at gmail.com>
Date: Wednesday, Oct 1, 2008 8:20 pm
Subject: Re: [Cython] cython leaks references to Py_None for cdef'ed globals
To: cython-dev <cython-dev at codespeak.net>Reply-To: cython-dev at codespeak.net

BTW, this patch fix the issue by not initializing cdef'ed module
>globals to None. But this is not the safest approach (I can imagine
>some nasty ways to originate segfaults). Perhaps we should try to fix
>the isue in FinalOptimizePhase.visit_SingleAssignmentNode
>
>On Wed, Oct 1, 2008 at 3:13 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> Consider the following code inside a one-line pyx file:
>>
>> cdef object someint = 7
>>
>> Then Cython generates the following inside the module init function:
>>
>>  /*--- Global init code ---*/
>>  __pyx_v_9refleaks2_someint = Py_None; Py_INCREF(Py_None);
>>
>>  /* "/u/dalcinl/Devel/Cython/sandbox/refleaks2.pyx":1
>>  * cdef object someint = 7             # <<<<<<<<<<<<<<
>>  *
>>  */
>>  Py_INCREF(__pyx_int_7);
>>  __pyx_v_9refleaks2_someint = __pyx_int_7;
>>
>>
>> Clearly, Py_None references are being leaked. All this is because of
>> bad interaction between this two methods:
>>
>> * ModuleNode.generate_global_init_code(...) (in ModuleNode.py)
>> * FinalOptimizePhase.visit_SingleAssignmentNode(...) (in Optimize.py)
>>
>>
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
>
>
>-- 
>Lisandro Dalc?n
>---------------
>Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>Tel/Fax: +54-(0)342-451.1594
>


From michael.abshoff at googlemail.com  Wed Oct  1 21:34:41 2008
From: michael.abshoff at googlemail.com (Michael Abshoff)
Date: Wed, 01 Oct 2008 12:34:41 -0700
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
In-Reply-To: <3305741288.1706043@smtp.netcom.no>
References: <3305741288.1706043@smtp.netcom.no>
Message-ID: <48E3D0D1.6050909@gmail.com>

Dag Sverre Seljebotn wrote:

Hi,

> I say let them leak? The only real risk (that I can see at least) is overflowing the refcount,
 > and if I understand and remember an informal chat correctly even that 
is eliminated in newer
 > Python versions where Py_None and friends have special deallocation 
which never deallocate...

I have had the pleasure to debug refcount reference overflows and let me 
say it is not something I would like to repeat since it happened only on 
32 bit test systems (the count would also wraps on a 64 bit box, but 
that would take a while longer). So I would highly suggest to fix any 
reference count leak since you never want code to crash for stupid 
reasons like this.

> Dag Sverre Seljebotn

<SNIP>

Cheers,

Michael

From thomas.e.keller at gmail.com  Wed Oct  1 21:41:15 2008
From: thomas.e.keller at gmail.com (Thomas Keller)
Date: Wed, 1 Oct 2008 19:41:15 +0000 (UTC)
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
Message-ID: <loom.20081001T193537-418@post.gmane.org>

Lisandro Dalcin <dalcinl at ...> writes:

> 
> Dear Thomas,
> 
> This should be fixed in 'cython-devel' repo. All numeric, string and
> container builtin types (plus slice and file) are supported for all
> Python versions. Let us know if you give a try to cython-devel and
> something does not work.
> 

Thanks much for the reply. I realized after the email I provided absolutely no
information about my build. I was using Cython from within SAGE 3.1.1, so I
believe it's not as updated as what is in cython-devel. I'll try that repo and
see how things go.






From dalcinl at gmail.com  Wed Oct  1 21:47:42 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 16:47:42 -0300
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
In-Reply-To: <48E3D0D1.6050909@gmail.com>
References: <3305741288.1706043@smtp.netcom.no> <48E3D0D1.6050909@gmail.com>
Message-ID: <e7ba66e40810011247j582193a2ma0f2314e336efbfb@mail.gmail.com>

The particular case being discussed, this leak occurs only one time,
at extension module import time.

Anyway, Michael, I'm on your side. In the past, refleaks in numpy made
it very hard for my to make sure in my unittesting that I was not
generating leaks on my code.

So, Dag, I think we should try hard to fix this. My patch fix the
problem, but it's a bit unsafe.  I'm so pedantic (because of bad
experiences in the past) about ref leaks that I always run Cython with
'--cleanup 9' flag. And I always run my full testsuite inside a loop
with debug python builds just to make sure I'm never leaking.


On Wed, Oct 1, 2008 at 4:34 PM, Michael Abshoff
<michael.abshoff at googlemail.com> wrote:
> Dag Sverre Seljebotn wrote:
>
> Hi,
>
>> I say let them leak? The only real risk (that I can see at least) is overflowing the refcount,
>  > and if I understand and remember an informal chat correctly even that
> is eliminated in newer
>  > Python versions where Py_None and friends have special deallocation
> which never deallocate...
>
> I have had the pleasure to debug refcount reference overflows and let me
> say it is not something I would like to repeat since it happened only on
> 32 bit test systems (the count would also wraps on a 64 bit box, but
> that would take a while longer). So I would highly suggest to fix any
> reference count leak since you never want code to crash for stupid
> reasons like this.
>
>> Dag Sverre Seljebotn
>
> <SNIP>
>
> Cheers,
>
> Michael
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Oct  1 22:10:05 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 17:10:05 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <loom.20081001T193537-418@post.gmane.org>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
Message-ID: <e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>

On Wed, Oct 1, 2008 at 4:41 PM, Thomas Keller <thomas.e.keller at gmail.com> wrote:
> Thanks much for the reply. I realized after the email I provided absolutely no
> information about my build. I was using Cython from within SAGE 3.1.1, so I
> believe it's not as updated as what is in cython-devel. I'll try that repo and
> see how things go.

Yep, the suport for your request was pushed a couple of days ago, so
it not availabe even in the latest official release.

BTW, access to the 'set' methods is not currently optimized in any
way. If you can help me a bit, we can optimize this, ie. map
'someset.add(obj)' to a direct Python C-API call. If you make a heavy
use of sets this could give you some speedup.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From thomas.e.keller at gmail.com  Wed Oct  1 23:05:00 2008
From: thomas.e.keller at gmail.com (Thomas Keller)
Date: Wed, 1 Oct 2008 21:05:00 +0000 (UTC)
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
Message-ID: <loom.20081001T210058-967@post.gmane.org>

Lisandro Dalcin <dalcinl at ...> writes:


> 
> BTW, access to the 'set' methods is not currently optimized in any
> way. If you can help me a bit, we can optimize this, ie. map
> 'someset.add(obj)' to a direct Python C-API call. If you make a heavy
> use of sets this could give you some speedup.
> 


I'm not sure how useful I'll be, but I'd be glad to try helping. The program I
am trying to use this with centers around a loop that heavily uses various set
operations, so I am definitely interested. What files should I look at to see
what to follow? As a disclaimer, I've never tried directly interfacing with the
Python C-API.


From robertwb at math.washington.edu  Wed Oct  1 23:13:03 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Oct 2008 14:13:03 -0700
Subject: [Cython] cython leaks references to Py_None for cdef'ed globals
In-Reply-To: <e7ba66e40810011119o3b5fbcbi3fff71fd606b71b8@mail.gmail.com>
References: <e7ba66e40810011113r653738b7ve21afe5b4803973@mail.gmail.com>
	<e7ba66e40810011119o3b5fbcbi3fff71fd606b71b8@mail.gmail.com>
Message-ID: <41440546-AAA6-433E-8FC1-3B6E043B6C93@math.washington.edu>

On Oct 1, 2008, at 11:19 AM, Lisandro Dalcin wrote:

> BTW, this patch fix the issue by not initializing cdef'ed module
> globals to None. But this is not the safest approach (I can imagine
> some nasty ways to originate segfaults). Perhaps we should try to fix
> the isue in FinalOptimizePhase.visit_SingleAssignmentNode

I agree that this should be fixed (I remember debugging a nasty bug  
where the refcount of the integer 0 wrapped around, ugh..). However,  
FinalOptimizePhase is the right place to do it.

>
> On Wed, Oct 1, 2008 at 3:13 PM, Lisandro Dalcin <dalcinl at gmail.com>  
> wrote:
>> Consider the following code inside a one-line pyx file:
>>
>> cdef object someint = 7
>>
>> Then Cython generates the following inside the module init function:
>>
>>  /*--- Global init code ---*/
>>  __pyx_v_9refleaks2_someint = Py_None; Py_INCREF(Py_None);
>>
>>  /* "/u/dalcinl/Devel/Cython/sandbox/refleaks2.pyx":1
>>  * cdef object someint = 7             # <<<<<<<<<<<<<<
>>  *
>>  */
>>  Py_INCREF(__pyx_int_7);
>>  __pyx_v_9refleaks2_someint = __pyx_int_7;
>>
>>
>> Clearly, Py_None references are being leaked. All this is because of
>> bad interaction between this two methods:
>>
>> * ModuleNode.generate_global_init_code(...) (in ModuleNode.py)
>> * FinalOptimizePhase.visit_SingleAssignmentNode(...) (in Optimize.py)
>>
>>
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
>
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0) 
> 342-451.1594<fixinitnoneleaks.diff>___________________________________ 
> ____________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Wed Oct  1 23:17:43 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Oct 2008 14:17:43 -0700
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <loom.20081001T210058-967@post.gmane.org>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
Message-ID: <9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>

On Oct 1, 2008, at 2:05 PM, Thomas Keller wrote:

> Lisandro Dalcin <dalcinl at ...> writes:
>
>
>>
>> BTW, access to the 'set' methods is not currently optimized in any
>> way. If you can help me a bit, we can optimize this, ie. map
>> 'someset.add(obj)' to a direct Python C-API call. If you make a heavy
>> use of sets this could give you some speedup.
>
> I'm not sure how useful I'll be, but I'd be glad to try helping.  
> The program I
> am trying to use this with centers around a loop that heavily uses  
> various set
> operations, so I am definitely interested. What files should I look  
> at to see
> what to follow? As a disclaimer, I've never tried directly  
> interfacing with the
> Python C-API.

You could look at how list and dict are handled at the bottom of

http://hg.cython.org/cython-devel/file/c9f3badef047/Cython/Compiler/ 
Builtin.py

- Robert


From dfdeshom at gmail.com  Wed Oct  1 23:25:51 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Wed, 1 Oct 2008 17:25:51 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
Message-ID: <4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>

On Wed, Oct 1, 2008 at 5:17 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
>
> You could look at how list and dict are handled at the bottom of
>
> http://hg.cython.org/cython-devel/file/c9f3badef047/Cython/Compiler/
> Builtin.py

Hi Robert,
What is the meaning of lines like this:
("append", "OO",   "i", "PyList_Append")

I'm especially curious about the "OO" and "i" stuff. Is this the only
code change needed to map these calls?

didier

>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From robertwb at math.washington.edu  Wed Oct  1 23:29:32 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Oct 2008 14:29:32 -0700
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
Message-ID: <2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>

On Oct 1, 2008, at 2:25 PM, didier deshommes wrote:

> On Wed, Oct 1, 2008 at 5:17 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>>
>> You could look at how list and dict are handled at the bottom of
>>
>> http://hg.cython.org/cython-devel/file/c9f3badef047/Cython/Compiler/
>> Builtin.py
>
> Hi Robert,
> What is the meaning of lines like this:
> ("append", "OO",   "i", "PyList_Append")
>
> I'm especially curious about the "OO" and "i" stuff.

"OO" means PyList_Append takes two arguments of type O = object. "i"  
means the return value is an int (for error checking, never actually  
revealed to Python).

> Is this the only code change needed to map these calls?

Yep, it's that easy now that we have support for this kind of thing.  
Looking forward to a patch :).

- Robert


From dalcinl at gmail.com  Wed Oct  1 23:37:45 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 18:37:45 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
Message-ID: <e7ba66e40810011437s4700005at947d4aa0de64fe00@mail.gmail.com>

In Python2.5:
PyAPI_FUNC(int) PySet_Clear(PyObject *set);
PyAPI_FUNC(int) PySet_Contains(PyObject *anyset, PyObject *key);
PyAPI_FUNC(int) PySet_Discard(PyObject *set, PyObject *key);
PyAPI_FUNC(int) PySet_Add(PyObject *set, PyObject *key);
PyAPI_FUNC(int) _PySet_Next(PyObject *set, Py_ssize_t *pos, PyObject **key);
PyAPI_FUNC(int) _PySet_NextEntry(PyObject *set, Py_ssize_t *pos,
PyObject **key, long *hash);
PyAPI_FUNC(PyObject *) PySet_Pop(PyObject *set);
PyAPI_FUNC(int) _PySet_Update(PyObject *set, PyObject *iterable);

then you should use (check it!!!)

"clear", "O", "i", "PySet_Clear"

"discard", "OO", "i", "PySet_Discard"

"add "OO", "i", "PySet_Add"

"pop", "O", "O", "PySet_Pop"



On Wed, Oct 1, 2008 at 6:29 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 1, 2008, at 2:25 PM, didier deshommes wrote:
>
>> On Wed, Oct 1, 2008 at 5:17 PM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>>
>>> You could look at how list and dict are handled at the bottom of
>>>
>>> http://hg.cython.org/cython-devel/file/c9f3badef047/Cython/Compiler/
>>> Builtin.py
>>
>> Hi Robert,
>> What is the meaning of lines like this:
>> ("append", "OO",   "i", "PyList_Append")
>>
>> I'm especially curious about the "OO" and "i" stuff.
>
> "OO" means PyList_Append takes two arguments of type O = object. "i"
> means the return value is an int (for error checking, never actually
> revealed to Python).
>
>> Is this the only code change needed to map these calls?
>
> Yep, it's that easy now that we have support for this kind of thing.
> Looking forward to a patch :).
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Wed Oct  1 23:44:31 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Oct 2008 14:44:31 -0700
Subject: [Cython] Python-compatible syntax
In-Reply-To: <50542.85.165.152.85.1222448145.squirrel@webmail.uio.no>
References: <3305292230.603409@smtp.netcom.no>
	<85b5c3130809260812u4b9a24bfn84871c1e734a2fe2@mail.gmail.com>
	<50542.85.165.152.85.1222448145.squirrel@webmail.uio.no>
Message-ID: <836CF7B1-3029-4E9E-AE1C-9C17B56B1647@math.washington.edu>

On Sep 26, 2008, at 9:55 AM, Dag Sverre Seljebotn wrote:

> Ondrej wrote:
>> On Fri, Sep 26, 2008 at 4:43 PM, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>> I can write more later when I'm not on my phone, but: I'd suggest
>>> starting with implementing
>>>
>>> @cython.earlybind
>>> def f(x, y): ...
>>>
>>> and have it turn to
>>>
>>> cpdef f(x, y): ...
>>>
>>> (note on the name below)
>>>
>>> Just getting this working, with arguments and result typed to  
>>> "object",
>>> should get your feet wet. Then I suppose the next step is  
>>> implementing
>>> the Py3 PEP supporting decorators on arguments and return type.  
>>> (I think
>>> one should aim for Py2.6+ here, though I suppose arguments to the
>>> decorator could work too: @cython.earlybind(x=cython.uint,
>>> localvar=cython.float)...opinions?
>>
>> I need it to be working with python2.4+, so I'll choose something  
>> that
>> works in python2.4. Debian just switched to python2.5 recently, so
>> it's a long, long way before python2.6 could become a de facto
>> standard. But generally I agree one should design this so that things
>> are especially easy in python2.6+.
>
> (Writing this just so that you can look it up when you are ready for
> types, ideally when declaring cpdef without any non-object types works
> already).
>
> We've had long discussions about options for this earlier and it is
> something people care strongly about, so please check with the mailing
> list before fixing too hard on a given syntax.
>
> Until something else is proposed then I'm +1 on my following  
> proposal (but
> at least Stefan or Robert should +1 it before it has any kind of  
> official
> status):
>
> @cython.earlybind(cython.uint, x=cython.uint, v=cython.float)
> def f(x):
>     v = x
>     return x
>
> The first positional argument is the return type, while any keyword
> arguments are for either arguments or local variables. Support for the
> Py2.6+ syntax can be added in addition later.
>
> We had a discussion already about the type names, I think we agreed on
> something like (though others should comment too):
> - cython.uint, cython.int, cython.float, cython.longdouble, etc.
> - cython.p_uint for unsigned*. Up to three pointers provided like  
> this:
> cython.ppp_uint is unsigned***. (Hmm, in fact, as this is likely  
> parsed as
> a string and not declared anywhere, one can make the number of p-s
> arbitrary).
> - For people preferring verbosity: cython.ptr(cython.uint), with  
> nesting
> of ptr as many times as one like. Finally, cython.ptr(cython.uint,
> levels=4) for unsigned****...
> - Pointers to functions provided in a similar fashion, i.e.
> cython.func_ptr(cython.int, cython.float) etc.
>
> Notes you can look at again when you get as far (I enjoy thinking  
> about
> how to do stuff, coding it is the boring part :-) of course, if you do
> things in your own way instead there's no harm to it though):
>
> 1) This is already parseable by Parsing.py, so you should be able to
> implement this purely by staying in the transform I wrote about and  
> output
> type nodes in the appropriate places.
>
> 2) It would work by having the transform simply inserting "cdef  
> unsigned
> int v" statements (i.e. CVarDeclNode) at the beginning of the function
> body, or changing info in the argument list of the function  
> (anything in
> the argument list goes there, the rest is inserted at beginning of
> function without checking whether it is used).
>
> 3) As cpdef functions cannot take pointer arguments, one should  
> generate
> cdef functions instead when there are pointer arguments. IMO this  
> should
> be automagic, i.e. cpdef is selected whenever the arguments are  
> coercable
> from objects and cdef otherwise. (An alternative I would like better,
> though, is to have cpdef functions taking pointer arguments be  
> allowed in
> the language, but lead to a "def" version containing a stub raising a
> run-time error! This way things could be more consistent with good  
> error
> reporting. Long term, ctypes pointers could be coercable to Cython
> pointers too.)
>
>>> 2) The ResolveDirectives transform (or something like that)  
>>> already has
>>> code in it to intercept @cython.X-decorators on functions for  
>>> compiler
>>> directives, I think you want to extend that transform, in time  
>>> changing
>>> the name appropriately to give it a wider scope.
>
> Having thought about this, the transform should be renamed and
> redocumented to "Handle the parts of the cython scope that are not  
> normal
> run-time functions, i.e. any 'magic' functions and decorators in the
> cython cimportable module." Something like CythonScopeTransform  
> perhaps.

Some preliminary support for this is up at http://hg.cython.org/ 
cython-devel/rev/cfc5c05e0292 and http://hg.cython.org/cython-devel/ 
rev/c9f3badef047 . There's *lots* of room for improvement (e.g. no  
support for the p*_int notation, etc) but that should be fairly easy  
once the framework is in place. One can now do

import cython

@cython.locals(s=cython.double, i=int, n=int)
def harmonic_sum(n):
     s = 0
     for i in range(1, n+1):
         s += 1.0 / i
     return s

and it will work in Python or Cython (there's a new "shadow" module).  
One can also do

@cython.locals(ptr="void*")
def my_id(x):
     ptr = cython.cast("void*", x)
     return cython.cast(long, ptr)

though of course that won't work in Python. This does not handle def - 
 > c[p]def. Until we have auto-generation of .pxd files, this is  
probably easiest to handle by letting the declarations in the .pxd  
file "mutate" the def nodes in the .py file into the right thing.

- Robert


From robertwb at math.washington.edu  Thu Oct  2 00:02:55 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 1 Oct 2008 15:02:55 -0700
Subject: [Cython] pyximport
In-Reply-To: <48E35A3E.5080403@mineway.de>
References: <48E35A3E.5080403@mineway.de>
Message-ID: <FDF7F4D0-B767-48CD-B6D1-A6BC1EB46D13@math.washington.edu>


On Oct 1, 2008, at 4:08 AM, Uwe Schmitt wrote:

> Hi,
>
> I had trouble to import pyximport and found
> only two postings about this problem from august.
>
> The problem was that "import pyximport" fails,
> which I could fix (as suggested in the postings
> above) by making pyximport/ a python packgage,
> which means I had to add a "__init__.py" file to
> .../site-packages/cython.../pyximport/

Thanks. We are tracking this at http://trac.cython.org/cython_trac/ 
ticket/91

> I wonder that I found not more about that.
> Is nobody using pyximport or did I miss something ?

It is very new, so not many people are using it yet.

- Robert



From greg.ewing at canterbury.ac.nz  Thu Oct  2 02:51:47 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Oct 2008 12:51:47 +1200
Subject: [Cython] quick guide moved ???
In-Reply-To: <C622577D-9A2C-4B47-A215-5440A5FA349B@mineway.de>
References: <C622577D-9A2C-4B47-A215-5440A5FA349B@mineway.de>
Message-ID: <48E41B23.8060509@canterbury.ac.nz>

Uwe Schmitt wrote:

> does anybody know what happened
> to "michaels quick guide to pyrex"
> at http://iopen.net/pyrex-guide/ ???

Not sure about the original, but there's a copy of it available here:

http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/mpj17-pyrex-guide/

-- 
Greg

From dalcinl at gmail.com  Thu Oct  2 03:42:13 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 1 Oct 2008 22:42:13 -0300
Subject: [Cython] Robert, please help!
Message-ID: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>

Robert, in this changeset

changeset:   1184:515f94dc16f2
user:        Robert Bradshaw <robertwb at math.washington.edu>
date:        Mon Sep 29 19:08:45 2008 -0700
summary:     Refactoring of type parsing

you cannot parse C function typedef like this:

cdef extern from *:
    void (somefunc)(int)

Please note that this declaration is not of the form "void
(*somefuncptr)(int)", and AFAIK the former is valid C code.

I was looking at the changes you pushed, but I was unable to discover
what's going on. So I really need your help here.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Thu Oct  2 04:40:19 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Oct 2008 14:40:19 +1200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
Message-ID: <48E43493.4040009@canterbury.ac.nz>

Robert Bradshaw wrote:
> On Oct 1, 2008, at 2:25 PM, didier deshommes wrote:
 >
> > I'm especially curious about the "OO" and "i" stuff.
> 
> "OO" means PyList_Append takes two arguments of type O = object. "i"  
> means the return value is an int (for error checking, never actually  
> revealed to Python).

That should really be "r", not "i" ("r" is an int used only
to signal an error, "i" is an actual int to be used).

There's a comment at the top of TypeSlots.py explaining
what all the letters mean.

-- 
Greg

From dfdeshom at gmail.com  Thu Oct  2 05:28:57 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Wed, 1 Oct 2008 23:28:57 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
Message-ID: <4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>

On Wed, Oct 1, 2008 at 5:29 PM, Robert Bradshaw
>> Is this the only code change needed to map these calls?
>
> Yep, it's that easy now that we have support for this kind of thing.
> Looking forward to a patch :).

Ok, I've got a patch and I've looked at the generated C code, compiled
the module and verified the behavior of the function. How do I write
tests for cython? Do I just edit tests/run/dict.pyx and look at the
generated code?

didier

From kevinar18 at hotmail.com  Thu Oct  2 05:36:29 2008
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Wed, 1 Oct 2008 23:36:29 -0400
Subject: [Cython] quick guide moved ???
In-Reply-To: <C622577D-9A2C-4B47-A215-5440A5FA349B@mineway.de>
References: <C622577D-9A2C-4B47-A215-5440A5FA349B@mineway.de>
Message-ID: <BLU110-W453268B61709A4EB013082AA3D0@phx.gbl>



> Hi,
> 
> does anybody know what happened
> to "michaels quick guide to pyrex"
> at http://iopen.net/pyrex-guide/ ???
> 
> Greetings, Uwe
It's my fault.  I asked him for a non-copyleft documentation.  When he said there wasn't one, I thanked him and told him about the super secret docs at http://www.mudskipper.ca/cython-doc/
 I'd say I won't do it again ... but I know I will. :)

/end sarcasm

On a serious note, I might want to work up some kind of Cython reference; I know it will help me, and I suspect it would help others.  So, who can I ask lots of questions? :)
_________________________________________________________________
See how Windows connects the people, information, and fun that are part of your life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/

From dalcinl at gmail.com  Thu Oct  2 05:50:36 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 00:50:36 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
Message-ID: <e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>

On Thu, Oct 2, 2008 at 12:28 AM, didier deshommes <dfdeshom at gmail.com> wrote:
>
> Ok, I've got a patch and I've looked at the generated C code, compiled
> the module and verified the behavior of the function. How do I write
> tests for cython? Do I just edit tests/run/dict.pyx and look at the
> generated code?

More or less, yes. Write a 'set.pyx' file like this:

__doc__ = u"""
>>> test_set()
True
"""

def test_set():
    cdef set s1 = set()
    cdef set s2 = set()
    # then do stuff with s1 and s2, like s1.add(1)
    # finally return True
    return True


Of course, I do not know if this can be readily incorporated in Cython
test suite, provided that this code is going to fail  in the case of a
Python 2.3 runtime.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Oct  2 05:58:36 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 00:58:36 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E43493.4040009@canterbury.ac.nz>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<48E43493.4040009@canterbury.ac.nz>
Message-ID: <e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>

On Wed, Oct 1, 2008 at 11:40 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Robert Bradshaw wrote:
>> "OO" means PyList_Append takes two arguments of type O = object. "i"
>
> That should really be "r", not "i" ("r" is an int used only
> to signal an error, "i" is an actual int to be used).
>

Mmm.. Perhaps some black magic in Cython? "i" is being used, and the
generated code interprets this as a return flag for errors, with
checks like if(unlikely(__pyx_1 ==  -1))) ...


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Thu Oct  2 06:06:42 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 02 Oct 2008 16:06:42 +1200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<48E43493.4040009@canterbury.ac.nz>
	<e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>
Message-ID: <48E448D2.1080506@canterbury.ac.nz>

Lisandro Dalcin wrote:

> Mmm.. Perhaps some black magic in Cython? "i" is being used, and the
> generated code interprets this as a return flag for errors, with
> checks like if(unlikely(__pyx_1 ==  -1))) ...

On further investigation, it seems it doesn't make a
difference in this situation. But "r" is conceptually
more correct, and it may become important in a future
version of Pyrex.

-- 
Greg

From ondrej at certik.cz  Thu Oct  2 09:39:10 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 2 Oct 2008 09:39:10 +0200
Subject: [Cython] Does not accept Unicode docstrings
Message-ID: <85b5c3130810020039x4f11e7a6md28e8a747875e2b0@mail.gmail.com>

Hi,

Cython doesn't seem to accept unicode strings:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500847


Ondrej

From dalcinl at gmail.com  Thu Oct  2 15:11:50 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 10:11:50 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E448D2.1080506@canterbury.ac.nz>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<48E43493.4040009@canterbury.ac.nz>
	<e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>
	<48E448D2.1080506@canterbury.ac.nz>
Message-ID: <e7ba66e40810020611m1a0f380elfcec472b287c465a@mail.gmail.com>

On Thu, Oct 2, 2008 at 1:06 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>
>> Mmm.. Perhaps some black magic in Cython? "i" is being used, and the
>> generated code interprets this as a return flag for errors, with
>> checks like if(unlikely(__pyx_1 ==  -1))) ...
>
> On further investigation, it seems it doesn't make a
> difference in this situation. But "r" is conceptually
> more correct, and it may become important in a future
> version of Pyrex.
>

OK, good to know! Robert, should we fix this in Cython?



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Oct  2 15:13:38 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 10:13:38 -0300
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <85b5c3130810020039x4f11e7a6md28e8a747875e2b0@mail.gmail.com>
References: <85b5c3130810020039x4f11e7a6md28e8a747875e2b0@mail.gmail.com>
Message-ID: <e7ba66e40810020613k11f71d26v46c483bbdc61e8ed@mail.gmail.com>

Did you try to write your docstrings as  u"somestuff"", ie, use the u"" prefix?

On Thu, Oct 2, 2008 at 4:39 AM, Ondrej Certik <ondrej at certik.cz> wrote:
> Hi,
>
> Cython doesn't seem to accept unicode strings:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500847
>
>
> Ondrej
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Thu Oct  2 15:38:41 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 02 Oct 2008 15:38:41 +0200
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <85b5c3130810020039x4f11e7a6md28e8a747875e2b0@mail.gmail.com>
References: <85b5c3130810020039x4f11e7a6md28e8a747875e2b0@mail.gmail.com>
Message-ID: <48E4CEE1.2060402@student.matnat.uio.no>

Ondrej Certik wrote:
> Hi,
>
> Cython doesn't seem to accept unicode strings:
>
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=500847
>
>   
I just tried it against cython-devel and it worked just fine as long as 
the coding: utf-8 header is declared. This is consistent with how Python 
works (at least Python 2.5).

The user might have used a different encoding etc. but without a better 
bug report (binary file submitted, encoding of system, encoding 
behaviour of Python for the given file etc. etc.) it is difficult to say 
if anything really is wrong here.

Dag Sverre

From uschmitt at mineway.de  Thu Oct  2 17:25:17 2008
From: uschmitt at mineway.de (Uwe Schmitt)
Date: Thu, 02 Oct 2008 17:25:17 +0200
Subject: [Cython] resolving name conflict -- does not work for enums !?
Message-ID: <48E4E7DD.1040906@mineway.de>

Hi,

in order to resolve a name conflict concerning enums I followed the 
description from
http://docs.cython.org/docs/sharing_declarations.html#using-cimport-to-resolve-naming-conflicts

Here comes my code:

    ----- minimal.pxd ------------------------

        cdef extern from "minimal.h":

            cdef enum test:
                A

    ----- minimal.pyx -------------------------

        cimport minimal

        A = minimal.A

    -------------------------------------------


which does not work. Cython says "Assignment to non-lvalue 'A'"
If I modify it to "cdef int A = minimal.A" I get "'A' redeclared".

So, how can I provide the user of my module the same enums
as defined internally ???

Greetings, Uwe

-- 
Dr. rer. nat. Uwe Schmitt
F&E Mathematik
 
mineway GmbH
Science Park 2
D-66123 Saarbr?cken
 
Telefon: +49 (0)681 8390 5334
Telefax: +49 (0)681 830 4376
 
uschmitt at mineway.de
www.mineway.de
 
Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer
Amtsgericht Saarbr?cken HRB 12339


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081002/dd889387/attachment.htm 

From pagx13 at yahoo.gr  Thu Oct  2 17:46:14 2008
From: pagx13 at yahoo.gr (P M)
Date: Thu, 2 Oct 2008 18:46:14 +0300
Subject: [Cython] Does not accept Unicode docstrings
Message-ID: <200810021846.14875.pagx13@yahoo.gr>

Hello, I am the original reporter of the bug with description "Does not accept 
Unicode docstrings" in the Debian BTS. I tried using the encoding: UTF-8 
header and using the u'' prefix, but the problem still persists. I'll try to 
post more information with this email.

$ locale
LANG=el_GR.UTF-8
LC_CTYPE="el_GR.UTF-8"
LC_NUMERIC="el_GR.UTF-8"
LC_TIME="el_GR.UTF-8"
LC_COLLATE="el_GR.UTF-8"
LC_MONETARY="el_GR.UTF-8"
LC_MESSAGES="el_GR.UTF-8"
LC_PAPER="el_GR.UTF-8"
LC_NAME="el_GR.UTF-8"
LC_ADDRESS="el_GR.UTF-8"
LC_TELEPHONE="el_GR.UTF-8"
LC_MEASUREMENT="el_GR.UTF-8"
LC_IDENTIFICATION="el_GR.UTF-8"
LC_ALL=
$ head -n1 `which cython`
#!/usr/bin/python
$ python --version
Python 2.5.2
$ cython --version
Cython version 0.9.8
$ file bug.pyx
bug.pyx: UTF-8 Unicode text
$ cat bug.pyx
# -*- encoding: UTF-8 -*-
def hello():
        u"""???? ???, ?????!"""
        print 'Hello, world!'
$ cython bug.pyx
Traceback (most recent call last):
  File "/usr/bin/cython", line 8, in <module>
    main(command_line = 1)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 527, 
in main
    result = compile(sources, options)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 505, 
in compile
    return compile_multiple(source, options)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 472, 
in compile_multiple
    result = context.compile(source, options)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 327, 
in compile
    tree.process_implementation(scope, options, result)
  File "/var/lib/python-support/python2.5/Cython/Compiler/ModuleNode.py", line 
59, in process_implementation
    self.generate_c_code(env, options, result)
  File "/var/lib/python-support/python2.5/Cython/Compiler/ModuleNode.py", line 
243, in generate_c_code
    self.body.generate_function_definitions(env, code, options.transforms)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Nodes.py", line 839, 
in generate_function_definitions
    with_pymethdef = env.is_py_class_scope)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Nodes.py", line 
1442, in generate_function_header
    self.entry.doc))
  File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py", line 52, 
in putln
    self.put(code)
  File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py", line 69, 
in put
    self._write(code)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 38-41: 
ordinal not in range(128)
$ echo $?
1
$

I hope this helps :)

From dfdeshom at gmail.com  Thu Oct  2 20:06:48 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Thu, 2 Oct 2008 14:06:48 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
Message-ID: <4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>

I've attached a patch for cdef'ed sets. It should work whether you're
running 2.3 or 2.5 (In 2.3 nothing will happen and "cdef set s" should
give you a syntax error). I also had to modify  PyrexTypes.py to make
sure that PyAnySet_CheckExact() was being passed, instead of
PySet_CheckExact() (there are 2 ways to check for the type of a set:
PyAnySet_CheckExact and PyFrozenSet_CheckExact). Reviews welcome!

didier

On Wed, Oct 1, 2008 at 11:50 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> On Thu, Oct 2, 2008 at 12:28 AM, didier deshommes <dfdeshom at gmail.com> wrote:
>>
>> Ok, I've got a patch and I've looked at the generated C code, compiled
>> the module and verified the behavior of the function. How do I write
>> tests for cython? Do I just edit tests/run/dict.pyx and look at the
>> generated code?
>
> More or less, yes. Write a 'set.pyx' file like this:
>
> __doc__ = u"""
>>>> test_set()
> True
> """
>
> def test_set():
>    cdef set s1 = set()
>    cdef set s2 = set()
>    # then do stuff with s1 and s2, like s1.add(1)
>    # finally return True
>    return True
>
>
> Of course, I do not know if this can be readily incorporated in Cython
> test suite, provided that this code is going to fail  in the case of a
> Python 2.3 runtime.
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: sets.txt
Url: http://codespeak.net/pipermail/cython-dev/attachments/20081002/f4eb378a/attachment-0001.txt 

From dalcinl at gmail.com  Thu Oct  2 20:40:49 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 15:40:49 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
Message-ID: <e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>

n Thu, Oct 2, 2008 at 3:06 PM, didier deshommes <dfdeshom at gmail.com> wrote:
> I've attached a patch for cdef'ed sets. It should work whether you're
> running 2.3 or 2.5 (In 2.3 nothing will happen and "cdef set s" should
> give you a syntax error).

Please, clarify my something: At wich stage the syntax error is shown
for Python 2.3? Do you mean the C compiler fails? Or is Cython
failing?

> I also had to modify  PyrexTypes.py to make
> sure that PyAnySet_CheckExact() was being passed, instead of
> PySet_CheckExact() (there are 2 ways to check for the type of a set:
> PyAnySet_CheckExact and PyFrozenSet_CheckExact). Reviews welcome!

Your patch seems fine, but I believe the fix for 'type_test_code()'
method is wrong for the case of 'frozenset', 'frozenset'.capitalize()
-> 'Frozenset', and you need 'FrozenSet', right? I would add these two
lines

elif type == 'Frozenset'
    type = 'FrozenSet'

As a final note, It would be great if you could also provide a test
(pyx file for compile and run) for all this!




> On Wed, Oct 1, 2008 at 11:50 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> On Thu, Oct 2, 2008 at 12:28 AM, didier deshommes <dfdeshom at gmail.com> wrote:
>>>
>>> Ok, I've got a patch and I've looked at the generated C code, compiled
>>> the module and verified the behavior of the function. How do I write
>>> tests for cython? Do I just edit tests/run/dict.pyx and look at the
>>> generated code?
>>
>> More or less, yes. Write a 'set.pyx' file like this:
>>
>> __doc__ = u"""
>>>>> test_set()
>> True
>> """
>>
>> def test_set():
>>    cdef set s1 = set()
>>    cdef set s2 = set()
>>    # then do stuff with s1 and s2, like s1.add(1)
>>    # finally return True
>>    return True
>>
>>
>> Of course, I do not know if this can be readily incorporated in Cython
>> test suite, provided that this code is going to fail  in the case of a
>> Python 2.3 runtime.
>>
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dfdeshom at gmail.com  Thu Oct  2 22:25:48 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Thu, 2 Oct 2008 16:25:48 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
Message-ID: <4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>

On Thu, Oct 2, 2008 at 2:40 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Please, clarify my something: At wich stage the syntax error is shown
> for Python 2.3? Do you mean the C compiler fails? Or is Cython
> failing?
>

Cython fails with a syntax error on "cdef set s". As an aside, if you
have this in py2.3:

"""
#py 2.3
cdef object s
s =set()
"""

Everything will compile fine but the module will fail to load when you
try to import it with this message:
"undefined symbol: PySet_Type". That's because Symtab.py assumes that
sets are available in py2.3 also, and includes it in `builtin_entries`
(a dict).Should we work around that or should we let users shoot
themselves in the foot?

>> I also had to modify  PyrexTypes.py to make
>> sure that PyAnySet_CheckExact() was being passed, instead of
>> PySet_CheckExact() (there are 2 ways to check for the type of a set:
>> PyAnySet_CheckExact and PyFrozenSet_CheckExact). Reviews welcome!
>
> Your patch seems fine, but I believe the fix for 'type_test_code()'
> method is wrong for the case of 'frozenset', 'frozenset'.capitalize()
> -> 'Frozenset', and you need 'FrozenSet', right? I would add these two
> lines
>
> elif type == 'Frozenset'
>    type = 'FrozenSet'

Right.

>
> As a final note, It would be great if you could also provide a test
> (pyx file for compile and run) for all this!

Ok, will do.

didier

From dagss at student.matnat.uio.no  Thu Oct  2 22:58:47 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 02 Oct 2008 22:58:47 +0200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>	<loom.20081001T210058-967@post.gmane.org>	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
Message-ID: <48E53607.4080908@student.matnat.uio.no>

didier deshommes wrote:
> On Thu, Oct 2, 2008 at 2:40 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
>> Please, clarify my something: At wich stage the syntax error is shown
>> for Python 2.3? Do you mean the C compiler fails? Or is Cython
>> failing?
>>
> 
> Cython fails with a syntax error on "cdef set s". As an aside, if you
> have this in py2.3:
> 
> """
> #py 2.3
> cdef object s
> s =set()
> """
> 
> Everything will compile fine but the module will fail to load when you
> try to import it with this message:
> "undefined symbol: PySet_Type". That's because Symtab.py assumes that
> sets are available in py2.3 also, and includes it in `builtin_entries`
> (a dict).Should we work around that or should we let users shoot
> themselves in the foot?

My five cents: I'm -1 on making emulating later Python versions in 
earlier ones in most cases, and definitely this one. We are developing 
Cython for the present and future, and Python 2.3 is starting to belong 
to the past.

The Cython compiler should run under Python 2.3 and create Python 
2.3-compatible code, but that is different from backporting builtins 
like "set".

-- 
Dag Sverre

From ggellner at uoguelph.ca  Thu Oct  2 23:00:40 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Thu, 2 Oct 2008 17:00:40 -0400
Subject: [Cython] array assignment
Message-ID: <20081002210040.GA15290@encolpuis>

is there any way to assign an cdef C array at assignment time?

I want something like:

    cdef double a[4] = {0.5, 0.3, 0.1, 0.1}

as anything like this possible in Cython? Or do I need to do explicit
initialization.

Gabriel

From ggellner at uoguelph.ca  Thu Oct  2 23:03:06 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Thu, 2 Oct 2008 17:03:06 -0400
Subject: [Cython] array assignment
In-Reply-To: <20081002210040.GA15290@encolpuis>
References: <20081002210040.GA15290@encolpuis>
Message-ID: <20081002210306.GA15396@encolpuis>

On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
> is there any way to assign an cdef C array at assignment time?
> 
> I want something like:
> 
>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
> 
> as anything like this possible in Cython? Or do I need to do explicit
> initialization.
> 
And if it is explicit is there any way to not just do (can I do some kind of 
block assignment?)

    a[0] = 0.5
        ...
    a[3] = 0.1

Looking for magic :-)

Gabriel

From dalcinl at gmail.com  Thu Oct  2 23:41:51 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 18:41:51 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
Message-ID: <e7ba66e40810021441l6def9df8w9a635d8f7e40b173@mail.gmail.com>

On Thu, Oct 2, 2008 at 5:25 PM, didier deshommes <dfdeshom at gmail.com> wrote:
>
> Cython fails with a syntax error on "cdef set s". As an aside, if you
> have this in py2.3:
>
> """
> #py 2.3
> cdef object s
> s =set()
> """

OK, that's a bit strange, but should be related to the fact that
Cython access the builtin module (I remember a post from Dag about
this issue). IMHO, a 'cdef set s' should not fail with a syntax error.

> Everything will compile fine but the module will fail to load when you
> try to import it with this message:
> "undefined symbol: PySet_Type". That's because Symtab.py assumes that
> sets are available in py2.3 also, and includes it in `builtin_entries`
> (a dict).Should we work around that or should we let users shoot
> themselves in the foot?

This is expected and we are fine with that.

>>> I also had to modify  PyrexTypes.py to make
>>> sure that PyAnySet_CheckExact() was being passed, instead of
>>> PySet_CheckExact() (there are 2 ways to check for the type of a set:
>>> PyAnySet_CheckExact and PyFrozenSet_CheckExact). Reviews welcome!
>>
>> Your patch seems fine, but I believe the fix for 'type_test_code()'
>> method is wrong for the case of 'frozenset', 'frozenset'.capitalize()
>> -> 'Frozenset', and you need 'FrozenSet', right? I would add these two
>> lines
>>
>> elif type == 'Frozenset'
>>    type = 'FrozenSet'
>
> Right.
>
>>
>> As a final note, It would be great if you could also provide a test
>> (pyx file for compile and run) for all this!
>
> Ok, will do.
>

OK, send the final patch (and possibly the test cases) when you finish it.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Oct  2 23:45:57 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 2 Oct 2008 18:45:57 -0300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E53607.4080908@student.matnat.uio.no>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
	<48E53607.4080908@student.matnat.uio.no>
Message-ID: <e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>

On Thu, Oct 2, 2008 at 5:58 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> My five cents: I'm -1 on making emulating later Python versions in
> earlier ones in most cases, and definitely this one. We are developing
> Cython for the present and future, and Python 2.3 is starting to belong
> to the past.

Me to, -1 on this...

> The Cython compiler should run under Python 2.3 and create Python
> 2.3-compatible code, but that is different from backporting builtins
> like "set".

However, Cython-the-compiler should not fail with syntax error if you
do 'cdef set s', right? This indicates that builtin module is being
used in the Cython codebase, and that is a potential source of
problem. In sort, I believe if you run Cython with python2.3, the
generated source should be valid Py3 C code, even if something like
'set' is being used. I'm missing something here?



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dfdeshom at gmail.com  Fri Oct  3 00:00:17 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Thu, 2 Oct 2008 18:00:17 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
	<48E53607.4080908@student.matnat.uio.no>
	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>
Message-ID: <4db014670810021500p49da5c61w8e54d0378997d699@mail.gmail.com>

On Thu, Oct 2, 2008 at 5:45 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> However, Cython-the-compiler should not fail with syntax error if you
> do 'cdef set s', right? This indicates that builtin module is being
> used in the Cython codebase, and that is a potential source of
> problem. In sort, I believe if you run Cython with python2.3, the
> generated source should be valid Py3 C code, even if something like
> 'set' is being used. I'm missing something here?

To be clear the "syntax error on cdef set s" was introduced by *my*
code changes: I just removed set and frozenset whenever
hasattr(types, 'GetSetDescriptorType') was False (in Builtin.py). This
can be readily corrected and will not break anything. Of course if you
 import this with py2.3, you'll get a missing symbol error. Sorry if
there was any confusion about this.

didier

>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From greg.ewing at canterbury.ac.nz  Fri Oct  3 03:37:14 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Oct 2008 14:37:14 +1300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810020611m1a0f380elfcec472b287c465a@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<48E43493.4040009@canterbury.ac.nz>
	<e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>
	<48E448D2.1080506@canterbury.ac.nz>
	<e7ba66e40810020611m1a0f380elfcec472b287c465a@mail.gmail.com>
Message-ID: <48E5774A.3090704@canterbury.ac.nz>

Lisandro Dalcin wrote:
> On Thu, Oct 2, 2008 at 1:06 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> But "r" is conceptually
>> more correct, and it may become important in a future
>> version of Pyrex.
> 
> OK, good to know! Robert, should we fix this in Cython?

It occurred to me while investigating this that there's
a slight difference between the optimised version of
list.append et al and the Python version: the optimised
version will return 0 if you use it in a context where
the return value is used, vs. None for the Python
version.

Not sure if anyone will care enough about that to be
worth fixing it, though.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Fri Oct  3 03:47:37 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Oct 2008 14:47:37 +1300
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
Message-ID: <48E579B9.8000408@canterbury.ac.nz>

didier deshommes wrote:

> Cython fails with a syntax error on "cdef set s".

The reason for the syntax error is that the parser doesn't
suspect that 'set' might be a type name.

Not sure about Cython, but in Pyrex the parser's list of
possible type names is seeded from the tables in Builtin.py.
If 'set' is declared there as something of type 'type',
that should fix the syntax error.

(The need for the parser to know about type names is a
bit hacky, but C's screwy declaration syntax makes it
hard to avoid when using an LL(1) parser.)

-- 
Greg

From dfdeshom at gmail.com  Fri Oct  3 06:56:19 2008
From: dfdeshom at gmail.com (didier deshommes)
Date: Fri, 3 Oct 2008 00:56:19 -0400
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810021441l6def9df8w9a635d8f7e40b173@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
	<e7ba66e40810021441l6def9df8w9a635d8f7e40b173@mail.gmail.com>
Message-ID: <4db014670810022156m4194e44elc8e8166bb95a6397@mail.gmail.com>

On Thu, Oct 2, 2008 at 5:41 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> OK, send the final patch (and possibly the test cases) when you finish it.

Ok, here is the final patch, with some test cases.

didier
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: set.txt
Url: http://codespeak.net/pipermail/cython-dev/attachments/20081003/bfbbccf8/attachment.txt 

From ondrej at certik.cz  Fri Oct  3 07:31:44 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 3 Oct 2008 07:31:44 +0200
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <200810021846.14875.pagx13@yahoo.gr>
References: <200810021846.14875.pagx13@yahoo.gr>
Message-ID: <85b5c3130810022231l1dfebd29ga6f9370aa0da122e@mail.gmail.com>

On Thu, Oct 2, 2008 at 5:46 PM, P M <pagx13 at yahoo.gr> wrote:
> Hello, I am the original reporter of the bug with description "Does not accept
> Unicode docstrings" in the Debian BTS. I tried using the encoding: UTF-8
> header and using the u'' prefix, but the problem still persists. I'll try to
> post more information with this email.
>
> $ locale
> LANG=el_GR.UTF-8
> LC_CTYPE="el_GR.UTF-8"
> LC_NUMERIC="el_GR.UTF-8"
> LC_TIME="el_GR.UTF-8"
> LC_COLLATE="el_GR.UTF-8"
> LC_MONETARY="el_GR.UTF-8"
> LC_MESSAGES="el_GR.UTF-8"
> LC_PAPER="el_GR.UTF-8"
> LC_NAME="el_GR.UTF-8"
> LC_ADDRESS="el_GR.UTF-8"
> LC_TELEPHONE="el_GR.UTF-8"
> LC_MEASUREMENT="el_GR.UTF-8"
> LC_IDENTIFICATION="el_GR.UTF-8"
> LC_ALL=
> $ head -n1 `which cython`
> #!/usr/bin/python
> $ python --version
> Python 2.5.2
> $ cython --version
> Cython version 0.9.8
> $ file bug.pyx
> bug.pyx: UTF-8 Unicode text
> $ cat bug.pyx
> # -*- encoding: UTF-8 -*-
> def hello():
>        u"""???? ???, ?????!"""
>        print 'Hello, world!'
> $ cython bug.pyx
> Traceback (most recent call last):
>  File "/usr/bin/cython", line 8, in <module>
>    main(command_line = 1)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 527,
> in main
>    result = compile(sources, options)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 505,
> in compile
>    return compile_multiple(source, options)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 472,
> in compile_multiple
>    result = context.compile(source, options)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py", line 327,
> in compile
>    tree.process_implementation(scope, options, result)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/ModuleNode.py", line
> 59, in process_implementation
>    self.generate_c_code(env, options, result)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/ModuleNode.py", line
> 243, in generate_c_code
>    self.body.generate_function_definitions(env, code, options.transforms)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Nodes.py", line 839,
> in generate_function_definitions
>    with_pymethdef = env.is_py_class_scope)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Nodes.py", line
> 1442, in generate_function_header
>    self.entry.doc))
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py", line 52,
> in putln
>    self.put(code)
>  File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py", line 69,
> in put
>    self._write(code)
> UnicodeEncodeError: 'ascii' codec can't encode characters in position 38-41:
> ordinal not in range(128)
> $ echo $?
> 1
> $
>
> I hope this helps :)

Thanks for the clarification.

Ondrej

From robertwb at math.washington.edu  Fri Oct  3 10:01:50 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 01:01:50 -0700
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E448D2.1080506@canterbury.ac.nz>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<e7ba66e40810011208p7bbb3431mb48a52d1e80e9ece@mail.gmail.com>
	<loom.20081001T193537-418@post.gmane.org>
	<e7ba66e40810011310wfdc578bu20b1e0c0045fe7b4@mail.gmail.com>
	<loom.20081001T210058-967@post.gmane.org>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<48E43493.4040009@canterbury.ac.nz>
	<e7ba66e40810012058q6053b102y236255c936b965b9@mail.gmail.com>
	<48E448D2.1080506@canterbury.ac.nz>
Message-ID: <B5903639-A3E6-4730-907B-EFEB3EE432CE@math.washington.edu>

On Oct 1, 2008, at 9:06 PM, Greg Ewing wrote:

> Lisandro Dalcin wrote:
>
>> Mmm.. Perhaps some black magic in Cython? "i" is being used, and the
>> generated code interprets this as a return flag for errors, with
>> checks like if(unlikely(__pyx_1 ==  -1))) ...
>
> On further investigation, it seems it doesn't make a
> difference in this situation. But "r" is conceptually
> more correct, and it may become important in a future
> version of Pyrex.

It seems an "r" means the returned result has no significance other  
than whether or not an error occurred, while an "i" is used to both  
flag an error and return a value. I suppose we should fix this  
(though it is rather low priority).

As you mentioned, it could be used to make the optimized append  
return None as well instead of 0 (if anyone cares to look at it) by  
having the c_returncode_type coerce to an object using None.

- Robert


From robertwb at math.washington.edu  Fri Oct  3 10:24:30 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 01:24:30 -0700
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>
	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>
	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>
	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>
	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>
	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>
	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>
	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>
	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>
	<48E53607.4080908@student.matnat.uio.no>
	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>
Message-ID: <4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>

On Oct 2, 2008, at 2:45 PM, Lisandro Dalcin wrote:

> On Thu, Oct 2, 2008 at 5:58 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> My five cents: I'm -1 on making emulating later Python versions in
>> earlier ones in most cases, and definitely this one. We are  
>> developing
>> Cython for the present and future, and Python 2.3 is starting to  
>> belong
>> to the past.
>
> Me to, -1 on this...

Some stuff (for example buffers, the with statement, and the b"byte  
string" syntax) I believe are worth backporting. Python 2.3 will  
continue to be a target for us (at least) as long as it is a target  
for NumPy.

Backporting builtins, however, I don't think is usually worth the  
effort. If one wishes to use more advanced builtins in ones sources,  
then one limits the platform it can run on. This is exactly like .py  
files.

>> The Cython compiler should run under Python 2.3 and create Python
>> 2.3-compatible code, but that is different from backporting builtins
>> like "set".
>
> However, Cython-the-compiler should not fail with syntax error if you
> do 'cdef set s', right? This indicates that builtin module is being
> used in the Cython codebase, and that is a potential source of
> problem. In sort, I believe if you run Cython with python2.3, the
> generated source should be valid Py3 C code, even if something like
> 'set' is being used. I'm missing something here?

Let X <= Y. If you run Cython with Python X, then it should compile  
against and run under Python Y. However, if you run Cython with  
Python Y, it may not compile against or run under Python X (depending  
on if you have used features introduced after X). Without backporting  
every new feature back to 2.3, this is the best we can hope to do.

Letting set be defined no matter what violates this principle. Thus  
if they have a Python 2.3 toolchain, they can (too easily) produce  
code that they cannot use. Furthermore, the error will be some  
obscure (to most people at least) message in the C compile. Much  
better to raise an error during the Cython run. That way if Cython  
succeeds, the C compile will succeed and the module will work (modulo  
any bugs of course).

In this case I think Didier's original patch was the right approach,  
though we will have to adjust the testing framework to account for  
it. I might suggest that testing

import __builtin__
if hasattr(__builtin__, 'set'):
     ....

is a bit more intuitive than the test you had before.

- Robert


From robertwb at math.washington.edu  Fri Oct  3 10:32:55 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 01:32:55 -0700
Subject: [Cython] Robert, please help!
In-Reply-To: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
Message-ID: <D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>

On Oct 1, 2008, at 6:42 PM, Lisandro Dalcin wrote:

> Robert, in this changeset
>
> changeset:   1184:515f94dc16f2
> user:        Robert Bradshaw <robertwb at math.washington.edu>
> date:        Mon Sep 29 19:08:45 2008 -0700
> summary:     Refactoring of type parsing
>
> you cannot parse C function typedef like this:
>
> cdef extern from *:
>     void (somefunc)(int)
>
> Please note that this declaration is not of the form "void
> (*somefuncptr)(int)", and AFAIK the former is valid C code.

Does it have the same meaning, or is there something more subtle  
going on? We don't accept all valid C code, for example

     int foo(void)

is allowed in C but not in Cython.

> I was looking at the changes you pushed, but I was unable to discover
> what's going on. So I really need your help here.

I needed to remove the dependancy the parser had on knowing all type  
names, so in some places it needs to look ahead to determine if it's  
looking at a type or an expression. If this is the only bug  
introduced, than I'm willing to accept that, but are there other  
statements that are now illegal?

- Robert


From robertwb at math.washington.edu  Fri Oct  3 10:36:25 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 01:36:25 -0700
Subject: [Cython] PATCH: fix pyximport installation as a package
In-Reply-To: <e7ba66e40810010652u77fe0376ud388b549a2ea00e@mail.gmail.com>
References: <e7ba66e40810010652u77fe0376ud388b549a2ea00e@mail.gmail.com>
Message-ID: <1A8FA402-AB2F-4F8A-A77A-310555E58250@math.washington.edu>

Looks good, thanks.

On Oct 1, 2008, at 6:52 AM, Lisandro Dalcin wrote:

> This should fix the installation of pyximport as a package. Please,
> review and apply.
>
>
>
> --  
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0) 
> 342-451.1594<pyximport.diff>__________________________________________ 
> _____
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Fri Oct  3 11:01:58 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 02:01:58 -0700
Subject: [Cython] resolving name conflict -- does not work for enums !?
In-Reply-To: <48E4E7DD.1040906@mineway.de>
References: <48E4E7DD.1040906@mineway.de>
Message-ID: <B227506C-CF43-4C76-BEBC-7A63FF47783E@math.washington.edu>

On Oct 2, 2008, at 8:25 AM, Uwe Schmitt wrote:

> Hi,
>
> in order to resolve a name conflict concerning enums I followed the  
> description from
> http://docs.cython.org/docs/sharing_declarations.html#using-cimport- 
> to-resolve-naming-conflicts
>
> Here comes my code:
>
> ----- minimal.pxd ------------------------
>
>     cdef extern from "minimal.h":
>
>         cdef enum test:
>             A
>
> ----- minimal.pyx -------------------------
>
>     cimport minimal
>
>     A = minimal.A
>
> -------------------------------------------
>
> which does not work. Cython says "Assignment to non-lvalue 'A'"
> If I modify it to "cdef int A = minimal.A" I get "'A' redeclared".

minimal.pxd and minimal.pyx have a special relationship to each  
other, specifically, minimal.pxd defines the interface of the things  
implemented in minimal.pyx (for use from other modules). The  
"imaginary" part of "imaginary module" is important, as once you have  
defined a .pyx file the module is no longer imaginary.

> So, how can I provide the user of my module the same enums
> as defined internally ???

Simply rename minimal.pxd to something else.

As a side note, for non-extern enums, you can write

cdef public enum:
     ...

and it will automatically export the enum variables to your (python- 
level) module scope. Unfortunately, it doesn't yet work for extern  
enums http://trac.cython.org/cython_trac/ticket/92

- Robert

From robertwb at math.washington.edu  Fri Oct  3 11:07:39 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 02:07:39 -0700
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <200810021846.14875.pagx13@yahoo.gr>
References: <200810021846.14875.pagx13@yahoo.gr>
Message-ID: <F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>

Thanks for the info. I'm not sure why this isn't working for you,  
could you attach the actual bug.pyx file (preferably zipped, to be  
sure that no mail servers mangle it at all)?

- Robert

On Oct 2, 2008, at 8:46 AM, P M wrote:

> Hello, I am the original reporter of the bug with description "Does  
> not accept
> Unicode docstrings" in the Debian BTS. I tried using the encoding:  
> UTF-8
> header and using the u'' prefix, but the problem still persists.  
> I'll try to
> post more information with this email.
>
> $ locale
> LANG=el_GR.UTF-8
> LC_CTYPE="el_GR.UTF-8"
> LC_NUMERIC="el_GR.UTF-8"
> LC_TIME="el_GR.UTF-8"
> LC_COLLATE="el_GR.UTF-8"
> LC_MONETARY="el_GR.UTF-8"
> LC_MESSAGES="el_GR.UTF-8"
> LC_PAPER="el_GR.UTF-8"
> LC_NAME="el_GR.UTF-8"
> LC_ADDRESS="el_GR.UTF-8"
> LC_TELEPHONE="el_GR.UTF-8"
> LC_MEASUREMENT="el_GR.UTF-8"
> LC_IDENTIFICATION="el_GR.UTF-8"
> LC_ALL=
> $ head -n1 `which cython`
> #!/usr/bin/python
> $ python --version
> Python 2.5.2
> $ cython --version
> Cython version 0.9.8
> $ file bug.pyx
> bug.pyx: UTF-8 Unicode text
> $ cat bug.pyx
> # -*- encoding: UTF-8 -*-
> def hello():
>         u"""???? ???, ?????!"""
>         print 'Hello, world!'
> $ cython bug.pyx
> Traceback (most recent call last):
>   File "/usr/bin/cython", line 8, in <module>
>     main(command_line = 1)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py",  
> line 527,
> in main
>     result = compile(sources, options)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py",  
> line 505,
> in compile
>     return compile_multiple(source, options)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py",  
> line 472,
> in compile_multiple
>     result = context.compile(source, options)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Main.py",  
> line 327,
> in compile
>     tree.process_implementation(scope, options, result)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/ 
> ModuleNode.py", line
> 59, in process_implementation
>     self.generate_c_code(env, options, result)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/ 
> ModuleNode.py", line
> 243, in generate_c_code
>     self.body.generate_function_definitions(env, code,  
> options.transforms)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/ 
> Nodes.py", line 839,
> in generate_function_definitions
>     with_pymethdef = env.is_py_class_scope)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/ 
> Nodes.py", line
> 1442, in generate_function_header
>     self.entry.doc))
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py",  
> line 52,
> in putln
>     self.put(code)
>   File "/var/lib/python-support/python2.5/Cython/Compiler/Code.py",  
> line 69,
> in put
>     self._write(code)
> UnicodeEncodeError: 'ascii' codec can't encode characters in  
> position 38-41:
> ordinal not in range(128)
> $ echo $?
> 1
> $
>
> I hope this helps :)
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Fri Oct  3 11:14:14 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 02:14:14 -0700
Subject: [Cython] Asking for review: More result_code refactor etc.
In-Reply-To: <48E11D33.9050004@student.matnat.uio.no>
References: <48E11D33.9050004@student.matnat.uio.no>
Message-ID: <30D13E46-4CC3-44B6-8B39-7422D619B29E@math.washington.edu>

On Sep 29, 2008, at 11:23 AM, Dag Sverre Seljebotn wrote:

> Buffer access towards complex types (treating them as a struct of two
> floats) are now up in -dagss.
>
> In order to get there I had to do a couple of dangerous things  
> regarding
> result_code though. These patches are really small but someone with  
> more
> knowledge may have more or less confidence in them than I:

Sorry to take so long to answer this, but changes like this make me  
wary too. (I think they're *good* changes, they're just scary ones to  
try and pronounce safe).

> http://hg.cython.org/cython-dagss/rev/b2bcc32021fd
> http://hg.cython.org/cython-dagss/rev/a8a9e4c7d5bd

After looking at them (again), I say +1.

> Note that this should fix the CloneNode issue I talked about. (And all
> tests pass btw; yet this is the kind of thing that would cause
> horrendous failures in obscure cases which are likely not tested  
> for...)
>
> Also in -dagss I've introduces NewTempExprNode, which is a  
> transitionary
> class where the temp allocation is moved to code generation.  
> Ideally all
> ExprNodes should descend from this one instead (and then it should be
> folded back into ExprNode to complete the migration).

That sounds like a good idea.

> Some classes fail hard when they are changed to the new scheme. It  
> works
> for BinOpNode which now uses it. This should allow partial migration.
> (One must make sure a class can work with the new temp allocation  
> scheme
> before constructing them in a transform after analysis).

Good point.

- Robert


From robertwb at math.washington.edu  Fri Oct  3 11:26:09 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 02:26:09 -0700
Subject: [Cython] A small change in compiler directive implementation
In-Reply-To: <48D80266.6060701@student.matnat.uio.no>
References: <48D80266.6060701@student.matnat.uio.no>
Message-ID: <ED34485E-E88D-4F01-BB76-D3D519D22413@math.washington.edu>

On Sep 22, 2008, at 1:39 PM, Dag Sverre Seljebotn wrote:

> Just a note that I've changed some internals with compiler directives.
>
> Before each node would get an "options" attribute in a transform that
> ran pretty early. This was getting increasingly cumbersome to deal  
> with
> (as transforms needed to make sure options was copied over to new  
> nodes
> etc.). So now I've created a CompilerDirectivesNode that sit in the  
> tree
>   instead. Each recursive process then stores the directives these  
> sets
> in different places:
>
> - env.directives
> - code.globalstate.directives
> - For CythonTransform descendants, self.current_directives is
> automatically updated as long as you leave the default
> CompilerDirectivesNode visit alone.
>
> If anyone protests I'll take it upon me to back it out.

No, it looks good. I've even put it to use and it works nicely :).

- Robert



From robertwb at math.washington.edu  Fri Oct  3 11:26:44 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 02:26:44 -0700
Subject: [Cython] Issues porting to Cython
In-Reply-To: <1222142045.9872.19.camel@tachyon-laptop>
References: <1222121599.14956.21.camel@tachyon-laptop>
	<9D7C2A64-0D58-4B73-A36D-558A6E05C3B1@math.washington.edu>
	<1222142045.9872.19.camel@tachyon-laptop>
Message-ID: <836B3FDC-9453-4996-8E3F-D7AA4956644D@math.washington.edu>

On Sep 22, 2008, at 8:54 PM, Andrew Collette wrote:

>
>> No, this looks like a bug. http://trac.cython.org/cython_trac/ticket/
>> 73 For now, import your ctypedef types explicitly (functions and
>> classes should cimport fine).
>
> Thanks!  Since hdf5.pxd is 1154 lines and is purely external
> definitions, I'm getting around this by just including it in my
> module .pxd files.  This seems to work for now.

BTW, this is fixed in cython-devel.

- Robert


From dagss at student.matnat.uio.no  Fri Oct  3 12:16:40 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Oct 2008 12:16:40 +0200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>	<48E53607.4080908@student.matnat.uio.no>	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>
	<4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>
Message-ID: <48E5F108.50601@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 2, 2008, at 2:45 PM, Lisandro Dalcin wrote:
> 
>> On Thu, Oct 2, 2008 at 5:58 PM, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>> My five cents: I'm -1 on making emulating later Python versions in
>>> earlier ones in most cases, and definitely this one. We are  
>>> developing
>>> Cython for the present and future, and Python 2.3 is starting to  
>>> belong
>>> to the past.
>> Me to, -1 on this...
> 
> Some stuff (for example buffers, the with statement, and the b"byte  
> string" syntax) I believe are worth backporting. Python 2.3 will  
> continue to be a target for us (at least) as long as it is a target  
> for NumPy.

BTW: This changed a few days ago, when 1.2.0 was released: "Please note 
that NumPy 1.2.0 requires Python 2.4 or greater."

(I'm still +1 on Py2.3 compatability, just noting it.)

> Let X <= Y. If you run Cython with Python X, then it should compile  
> against and run under Python Y. However, if you run Cython with  
> Python Y, it may not compile against or run under Python X (depending  
> on if you have used features introduced after X). Without backporting  
> every new feature back to 2.3, this is the best we can hope to do.

I don't follow the logic here.

Please bear with me, this is important: One consequence is that there is 
currently no way to get access to Py3-only builtins from pyx files 
without using getattr (like "memoryview", unless it was added to 
Python2.6 last minute, I didn't fetch the release yet. But even if it 
was, I don't think there's any rule that all new Python 3.x builtins 
will be backported to 2.x?).

In my mind there's three different Python versions involved:

- A: The version of Python the user targets in the pyx file (when it 
comes to library and builtins; keeping the language part out of this).
- B: The version of Python that imports the compiled module. The user 
just has to make sure that this matches A in a reasonable fashion (the 
rules are non-trivial and not a simple >= or <=, as Py3 removes 
builtins, but Python lays these down for us).
- C: The version that runs Cython.

This discussion was about A and B -- that Cython won't allow users to 
code for Py2.4 and run under Py2.3.

You seem to be talking about C, which is something different. Cython 
just transforms code and could run on any Turing-machine, there's no 
natural reason it has to be connected with A or B.

We do currently require C >= B (which means that any Py3-only builtins 
are plain unavailable).

In my opinion, ideally C should be orthogonal to A and B because that 
seems to be less complicated from a release and bugfinding perspective 
(one less version to worry about in the mix) and it would fix the issue 
with accessing Py3-builtins for free, but I don't care very much.

> 
> Letting set be defined no matter what violates this principle. Thus  
> if they have a Python 2.3 toolchain, they can (too easily) produce  
> code that they cannot use. Furthermore, the error will be some  
> obscure (to most people at least) message in the C compile. Much  
> better to raise an error during the Cython run. That way if Cython  
> succeeds, the C compile will succeed and the module will work (modulo  
> any bugs of course).

You can guard against typing "zpi", but not (currently at least) against 
typing "os.path.exitst". I guess I tend to think of those as about the 
same thing -- i.e. accessing a builtin is just making use of the 
__builtin__ run-time module, it is just that we hacked some of them for 
speed purposes (as we in future may be able to do with os.path.exists 
too through the use of inline code in pxd files).

That was the opposing view. But I see that this is non-trivial, and as I 
said this is not an important matter to me, so I might as well end this 
mail with agreeing with you that what you present is the best course of 
action for now :-)

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Fri Oct  3 12:18:55 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Oct 2008 12:18:55 +0200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E5F108.50601@student.matnat.uio.no>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>	<48E53607.4080908@student.matnat.uio.no>	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>	<4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>
	<48E5F108.50601@student.matnat.uio.no>
Message-ID: <48E5F18F.20102@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Robert Bradshaw wrote:
>> Let X <= Y. If you run Cython with Python X, then it should compile  
>> against and run under Python Y. However, if you run Cython with  
>> Python Y, it may not compile against or run under Python X (depending  
>> on if you have used features introduced after X). Without backporting  
>> every new feature back to 2.3, this is the best we can hope to do.
> 
> I don't follow the logic here.
> 
> Please bear with me, this is important: One consequence is that there is 
> currently no way to get access to Py3-only builtins from pyx files 
> without using getattr (like "memoryview", unless it was added to 
> Python2.6 last minute, I didn't fetch the release yet. But even if it 
> was, I don't think there's any rule that all new Python 3.x builtins 
> will be backported to 2.x?).
> 
> In my mind there's three different Python versions involved:
> 
> - A: The version of Python the user targets in the pyx file (when it 
> comes to library and builtins; keeping the language part out of this).
> - B: The version of Python that imports the compiled module. The user 
> just has to make sure that this matches A in a reasonable fashion (the 
> rules are non-trivial and not a simple >= or <=, as Py3 removes 
> builtins, but Python lays these down for us).
> - C: The version that runs Cython.
> 
> This discussion was about A and B -- that Cython won't allow users to 
> code for Py2.4 and run under Py2.3.
> 
> You seem to be talking about C, which is something different. Cython 
> just transforms code and could run on any Turing-machine, there's no 
> natural reason it has to be connected with A or B.
> 
> We do currently require C >= B (which means that any Py3-only builtins 
> are plain unavailable).
> 
> In my opinion, ideally C should be orthogonal to A and B because that 
> seems to be less complicated from a release and bugfinding perspective 
> (one less version to worry about in the mix) and it would fix the issue 
> with accessing Py3-builtins for free, but I don't care very much.

To be more concrete, the problems you mention could be fixed with a 
simple compiler flag (or directive) to Cython: --target-platform=3.0 or 
similar, to control the builtins. But, "there's no lack of good ideas".

-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Oct  3 12:16:14 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 03:16:14 -0700
Subject: [Cython] array assignment
In-Reply-To: <20081002210306.GA15396@encolpuis>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
Message-ID: <33262D7C-13AF-401D-BF67-3B4E9B9A948C@math.washington.edu>

On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:

> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>> is there any way to assign an cdef C array at assignment time?
>>
>> I want something like:
>>
>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>
>> as anything like this possible in Cython? Or do I need to do explicit
>> initialization.
>>
> And if it is explicit is there any way to not just do (can I do  
> some kind of
> block assignment?)
>
>     a[0] = 0.5
>         ...
>     a[3] = 0.1
>
> Looking for magic :-)

Nothing yet, I put up http://trac.cython.org/cython_trac/ticket/93

Also, I would like to be able to use (literal) tuples/dicts to assign  
to structs (and vice-versa). Would non-literal tuples/dicts be going  
too far?

- Robert


From dagss at student.matnat.uio.no  Fri Oct  3 12:35:49 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Oct 2008 12:35:49 +0200
Subject: [Cython] array assignment
In-Reply-To: <33262D7C-13AF-401D-BF67-3B4E9B9A948C@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>
	<33262D7C-13AF-401D-BF67-3B4E9B9A948C@math.washington.edu>
Message-ID: <48E5F585.1040007@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:
> 
>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>> is there any way to assign an cdef C array at assignment time?
>>>
>>> I want something like:
>>>
>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>>
>>> as anything like this possible in Cython? Or do I need to do explicit
>>> initialization.
>>>
>> And if it is explicit is there any way to not just do (can I do  
>> some kind of
>> block assignment?)
>>
>>     a[0] = 0.5
>>         ...
>>     a[3] = 0.1
>>
>> Looking for magic :-)
> 
> Nothing yet, I put up http://trac.cython.org/cython_trac/ticket/93
> 
> Also, I would like to be able to use (literal) tuples/dicts to assign  
> to structs (and vice-versa). Would non-literal tuples/dicts be going  
> too far?

I think a default "literal struct constructor" might be more fitting? 
And I see no reason it would need to be literal except for brevity of C 
code.

A problem is with mutability though, if it is done using C static 
allocation. I.e.:

def f(x):
     cdef int a[3] = [1,2,3]
     if x: a[2] = 1
     print a[2]

f(True) # prints 1...
f(False) # likely prints 1 too!

This is consistent with what you expect from C but very contrary to what 
Python users would expect.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Fri Oct  3 12:44:03 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Oct 2008 12:44:03 +0200
Subject: [Cython] array assignment
In-Reply-To: <48E5F585.1040007@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<33262D7C-13AF-401D-BF67-3B4E9B9A948C@math.washington.edu>
	<48E5F585.1040007@student.matnat.uio.no>
Message-ID: <48E5F773.2030200@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> A problem is with mutability though, if it is done using C static 
> allocation. I.e.:
> 
> def f(x):
>      cdef int a[3] = [1,2,3]
>      if x: a[2] = 1
>      print a[2]
> 
> f(True) # prints 1...
> f(False) # likely prints 1 too!
> 
> This is consistent with what you expect from C but very contrary to what 
> Python users would expect.

No this is apparently plain wrong. Sorry.


-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Oct  3 12:38:17 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 03:38:17 -0700
Subject: [Cython] array assignment
In-Reply-To: <48E5F585.1040007@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>
	<33262D7C-13AF-401D-BF67-3B4E9B9A948C@math.washington.edu>
	<48E5F585.1040007@student.matnat.uio.no>
Message-ID: <F42A2CDC-E1F3-49D0-B0E3-95D2D500F3C4@math.washington.edu>

On Oct 3, 2008, at 3:35 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:
>>
>>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>>> is there any way to assign an cdef C array at assignment time?
>>>>
>>>> I want something like:
>>>>
>>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>>>
>>>> as anything like this possible in Cython? Or do I need to do  
>>>> explicit
>>>> initialization.
>>>>
>>> And if it is explicit is there any way to not just do (can I do
>>> some kind of
>>> block assignment?)
>>>
>>>     a[0] = 0.5
>>>         ...
>>>     a[3] = 0.1
>>>
>>> Looking for magic :-)
>>
>> Nothing yet, I put up http://trac.cython.org/cython_trac/ticket/93
>>
>> Also, I would like to be able to use (literal) tuples/dicts to assign
>> to structs (and vice-versa). Would non-literal tuples/dicts be going
>> too far?
>
> I think a default "literal struct constructor" might be more fitting?
> And I see no reason it would need to be literal except for brevity  
> of C
> code.

The literalness may not be reflected in the code (though it could  
impact more than the brevity of the C code, as assignment one-by-one  
may involve literals in the assembly, as opposed to a memcpy to  
assign an array at once.

> A problem is with mutability though, if it is done using C static
> allocation. I.e.:
>
> def f(x):
>      cdef int a[3] = [1,2,3]
>      if x: a[2] = 1
>      print a[2]
>
> f(True) # prints 1...
> f(False) # likely prints 1 too!
>
> This is consistent with what you expect from C but very contrary to  
> what
> Python users would expect.

No, I believe allocation is done on the stack.

- Robert



From greg.ewing at canterbury.ac.nz  Fri Oct  3 12:46:51 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 03 Oct 2008 22:46:51 +1200
Subject: [Cython] resolving name conflict -- does not work for enums !?
In-Reply-To: <48E4E7DD.1040906@mineway.de>
References: <48E4E7DD.1040906@mineway.de>
Message-ID: <48E5F81B.2000600@canterbury.ac.nz>

Uwe Schmitt wrote:

>     ----- minimal.pxd ------------------------
> 
>         cdef extern from "minimal.h":
> 
>             cdef enum test:
>                 A
> 
>     ----- minimal.pyx -------------------------
> 
>         cimport minimal
> 
>         A = minimal.A
> 
>     -------------------------------------------

You shouldn't be cimporting minimal in minimal.pyx. The files
minimal.pxd and minimal.pyx are two parts of the *same* module,
i.e. minimal. There's only one namespace, and you're trying
to define A as both a C constant and a Python global in that
namespace.

You need to put the definition of enum test into a different
module namespace, e.g.

   # mindefs.pxd
   cdef enum test:
     A

   # minimal.pyx:
   cimport mindefs
   A = mindefs.A

Note that you don't need a mindefs.pyx in this case -- you're
only using mindefs as a compile-time namespace.

-- 
Greg

From robertwb at math.washington.edu  Fri Oct  3 13:06:54 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 04:06:54 -0700
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <48E5F18F.20102@student.matnat.uio.no>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>	<48E53607.4080908@student.matnat.uio.no>	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>	<4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>
	<48E5F108.50601@student.matnat.uio.no>
	<48E5F18F.20102@student.matnat.uio.no>
Message-ID: <3D47B019-2A7D-4743-B8E2-EE467DC9FA15@math.washington.edu>


On Oct 3, 2008, at 3:18 AM, Dag Sverre Seljebotn wrote:

> Dag Sverre Seljebotn wrote:
>> Robert Bradshaw wrote:
>>> Let X <= Y. If you run Cython with Python X, then it should compile
>>> against and run under Python Y. However, if you run Cython with
>>> Python Y, it may not compile against or run under Python X  
>>> (depending
>>> on if you have used features introduced after X). Without  
>>> backporting
>>> every new feature back to 2.3, this is the best we can hope to do.
>>
>> I don't follow the logic here.

Sorry I wasn't very clear, thanks for splitting things up more below.

>> Please bear with me, this is important: One consequence is that  
>> there is
>> currently no way to get access to Py3-only builtins from pyx files
>> without using getattr (like "memoryview", unless it was added to
>> Python2.6 last minute, I didn't fetch the release yet. But even if it
>> was, I don't think there's any rule that all new Python 3.x builtins
>> will be backported to 2.x?).
>>
>> In my mind there's three different Python versions involved:
>>
>> - A: The version of Python the user targets in the pyx file (when it
>> comes to library and builtins; keeping the language part out of  
>> this).
>> - B: The version of Python that imports the compiled module. The user
>> just has to make sure that this matches A in a reasonable fashion  
>> (the
>> rules are non-trivial and not a simple >= or <=, as Py3 removes
>> builtins, but Python lays these down for us).
>> - C: The version that runs Cython.
>>
>> This discussion was about A and B -- that Cython won't allow users to
>> code for Py2.4 and run under Py2.3.
>>
>> You seem to be talking about C, which is something different. Cython
>> just transforms code and could run on any Turing-machine, there's no
>> natural reason it has to be connected with A or B.

Theoretically, C is orthogonal. However, for a given user, it is  
likely that there is strong correlation between all of A, B, and C.

>> We do currently require C >= B (which means that any Py3-only  
>> builtins
>> are plain unavailable).

No, we require that C >= A. Often C = B, and obviously B >= A, so  
this is reasonable.

>> In my opinion, ideally C should be orthogonal to A and B because that
>> seems to be less complicated from a release and bugfinding  
>> perspective
>> (one less version to worry about in the mix)

On the other hand, violating C < A results in a nice cython-compile  
error, whereas violating B < A is either a (perhaps not easily hit)  
runtime error or a c-compile error. Which would you rather debug?  
It's also reassuring to say "if it compiles under C, it runs under C 
+" because A is often implicit rather than explicit.

>> and it would fix the issue
>> with accessing Py3-builtins for free, but I don't care very much.

Is this because Cython doesn't run under Py3? (I thought it did.) Or  
is there something I'm not seeing here?

> To be more concrete, the problems you mention could be fixed with a
> simple compiler flag (or directive) to Cython: --target- 
> platform=3.0 or
> similar, to control the builtins. But, "there's no lack of good  
> ideas".

I think inspecting __builtins__ at runtime is a more elegant (and  
safe) approach than manually keeping lists for every version. A  
possible exception is the builtins that disappeared in Py3, in which  
case I would lean towards "forwardporting" them by importing from the  
right place, as they are still there somewhere).

- Robert


From dagss at student.matnat.uio.no  Fri Oct  3 13:32:41 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 03 Oct 2008 13:32:41 +0200
Subject: [Cython] cython doesn't compile cdef'd variables of type set?
In-Reply-To: <3D47B019-2A7D-4743-B8E2-EE467DC9FA15@math.washington.edu>
References: <41d5d14b0810011129o393b4b11nbf64896833981cf6@mail.gmail.com>	<9BA7DDDB-6C85-451E-B8EF-6BDA4D283319@math.washington.edu>	<4db014670810011425l41fce606oba63fcebb9af1c08@mail.gmail.com>	<2A88D46F-77ED-4837-814D-DD8F70CC8237@math.washington.edu>	<4db014670810012028m580a0992u9d57eeaafede3300@mail.gmail.com>	<e7ba66e40810012050w6defa590o4c4bf82bcf3b2d81@mail.gmail.com>	<4db014670810021106w3c338851q85cf0ab6c435ba10@mail.gmail.com>	<e7ba66e40810021140sdf7cc77if718ff4a28d66ac0@mail.gmail.com>	<4db014670810021325p2c76d192pad964314950fe7c0@mail.gmail.com>	<48E53607.4080908@student.matnat.uio.no>	<e7ba66e40810021445t102bfacbhfb08fcf43d52e672@mail.gmail.com>	<4770B9DF-1E5F-41A5-BFE5-BE389A035696@math.washington.edu>	<48E5F108.50601@student.matnat.uio.no>	<48E5F18F.20102@student.matnat.uio.no>
	<3D47B019-2A7D-4743-B8E2-EE467DC9FA15@math.washington.edu>
Message-ID: <48E602D9.9030407@student.matnat.uio.no>

Robert Bradshaw wrote:
> Is this because Cython doesn't run under Py3? (I thought it did.) Or  
> is there something I'm not seeing here?

I haven't heard anything about Cython itself running under Py3. Stefan 
made runtests.py run under Py3, and then you need to first run it under 
Python2 with --no-cleanup, and then under Py3 with --no-cython (so that 
the C files generated using Py2 is compiled under Py3).

So this is where your scheme breaks down currently.

Perhaps this was fixed without me noticing it (if so I've broken it 
again in cython-devel I'm afraid, but that should be easy to fix).

If you want to fix this, then the matter disappears. OTOH I think 
supporting both Python 2 and Python 3 for running Cython could bog us 
down somewhat, at least it is not Guido's recommended porting strategy 
(which is staying with 2.x until you can make a total switch). Users 
will need to keep 2.x around on their systems for years to come anyway 
so I don't think it is something we should worry about at this stage if 
it means much work.

-- 
Dag Sverre

From dalcinl at gmail.com  Fri Oct  3 16:15:43 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 3 Oct 2008 11:15:43 -0300
Subject: [Cython] Robert, please help!
In-Reply-To: <D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
Message-ID: <e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>

On Fri, Oct 3, 2008 at 5:32 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 1, 2008, at 6:42 PM, Lisandro Dalcin wrote:
>
>> Robert, in this changeset
>>
>> changeset:   1184:515f94dc16f2
>> user:        Robert Bradshaw <robertwb at math.washington.edu>
>> date:        Mon Sep 29 19:08:45 2008 -0700
>> summary:     Refactoring of type parsing
>>
>> you cannot parse C function typedef like this:
>>
>> cdef extern from *:
>>     void (somefunc)(int)
>>
>> Please note that this declaration is not of the form "void
>> (*somefuncptr)(int)", and AFAIK the former is valid C code.
>
> Does it have the same meaning, or is there something more subtle
> going on?

Robert, AFAIK they do not have the same meaning. the void
(func1)(args) is function typedef and void (*func2)(args) is a funtion
POINTER typedef. In short, they are really different types. MPI has
many of those (non-pointer) typedef for some callback functions.

I really believe that this is a regression. It could potentially make
invalid some codes. In fact, this change already broke my codes.

> We don't accept all valid C code, for example
>
>     int foo(void)
>
> is allowed in C but not in Cython.

Long ago I discussed that case with Sfefan, and we agreed that there
was not point on supporting 'int foo(void)', as that C declaration was
in fact related to support ancient C and the different meaning of 'int
foo()', hopefully fixed in C++.

But in the fase of function vs. function pointer typedef's, the issues
are different. They really are different types.


>> I was looking at the changes you pushed, but I was unable to discover
>> what's going on. So I really need your help here.
>
> I needed to remove the dependancy the parser had on knowing all type
> names, so in some places it needs to look ahead to determine if it's
> looking at a type or an expression. If this is the only bug
> introduced, than I'm willing to accept that, but are there other
> statements that are now illegal?
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From stefan_ml at behnel.de  Fri Oct  3 17:51:46 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 03 Oct 2008 17:51:46 +0200
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
References: <200810021846.14875.pagx13@yahoo.gr>
	<F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
Message-ID: <48E63F92.9020002@behnel.de>

Hi,

Robert Bradshaw wrote:
> Thanks for the info. I'm not sure why this isn't working for you,  
> could you attach the actual bug.pyx file (preferably zipped, to be  
> sure that no mail servers mangle it at all)?

Also, you're using Cython 0.9.8. Could you try with the latest release? IIRC,
there were a couple of unicode related fixes after 0.9.8 was released.

Stefan


From pagx13 at yahoo.gr  Fri Oct  3 18:05:18 2008
From: pagx13 at yahoo.gr (P M)
Date: Fri, 3 Oct 2008 19:05:18 +0300
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
References: <200810021846.14875.pagx13@yahoo.gr>
	<F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
Message-ID: <200810031905.18194.pagx13@yahoo.gr>

OK, I'm sending the file as an attachment :)

On Friday 03 October 2008 12:07:39 Robert Bradshaw wrote:
> Thanks for the info. I'm not sure why this isn't working for you,
> could you attach the actual bug.pyx file (preferably zipped, to be
> sure that no mail servers mangle it at all)?
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bug.pyx.zip
Type: application/x-zip
Size: 217 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081003/a21062ab/attachment.bin 

From pagx13 at yahoo.gr  Fri Oct  3 18:27:09 2008
From: pagx13 at yahoo.gr (P M)
Date: Fri, 3 Oct 2008 19:27:09 +0300
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <48E63F92.9020002@behnel.de>
References: <200810021846.14875.pagx13@yahoo.gr>
	<F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
	<48E63F92.9020002@behnel.de>
Message-ID: <200810031927.09646.pagx13@yahoo.gr>

On Friday 03 October 2008 18:51:46 Stefan Behnel wrote:
> Also, you're using Cython 0.9.8. Could you try with the latest release?
> IIRC, there were a couple of unicode related fixes after 0.9.8 was
> released.

I tried with the latest upstream (0.9.8.1.1), and the problem seems to be 
solved, thanks! (I ran cython from within the distribution directory, instead 
of installing it, hope that makes no difference.) The string is translated 
into C code as a series of octal escapes instead of UTF-8 text though... and 
the comments that show the position inside the original file place "?" marks 
in place of UTF-8 characters. But I guess that's OK :)

From ondrej at certik.cz  Fri Oct  3 22:16:17 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 3 Oct 2008 22:16:17 +0200
Subject: [Cython] Does not accept Unicode docstrings
In-Reply-To: <200810031927.09646.pagx13@yahoo.gr>
References: <200810021846.14875.pagx13@yahoo.gr>
	<F7F42678-E0B1-405C-ADBB-EB26DEE72C41@math.washington.edu>
	<48E63F92.9020002@behnel.de> <200810031927.09646.pagx13@yahoo.gr>
Message-ID: <85b5c3130810031316l73329a04pdf0ace2b0f07fe4a@mail.gmail.com>

On Fri, Oct 3, 2008 at 6:27 PM, P M <pagx13 at yahoo.gr> wrote:
> On Friday 03 October 2008 18:51:46 Stefan Behnel wrote:
>> Also, you're using Cython 0.9.8. Could you try with the latest release?
>> IIRC, there were a couple of unicode related fixes after 0.9.8 was
>> released.
>
> I tried with the latest upstream (0.9.8.1.1), and the problem seems to be
> solved, thanks! (I ran cython from within the distribution directory, instead
> of installing it, hope that makes no difference.) The string is translated
> into C code as a series of octal escapes instead of UTF-8 text though... and
> the comments that show the position inside the original file place "?" marks
> in place of UTF-8 characters. But I guess that's OK :)

So just uploading a new upstream version should fix it. Seems like
there is a job for me. :)

Ondrej

From robertwb at math.washington.edu  Fri Oct  3 23:56:17 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 3 Oct 2008 14:56:17 -0700
Subject: [Cython] Robert, please help!
In-Reply-To: <e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
Message-ID: <AE445869-E728-4CF5-A2BD-2D2ABDA50BA6@math.washington.edu>

On Oct 3, 2008, at 7:15 AM, Lisandro Dalcin wrote:

> On Fri, Oct 3, 2008 at 5:32 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Oct 1, 2008, at 6:42 PM, Lisandro Dalcin wrote:
>>
>>> Robert, in this changeset
>>>
>>> changeset:   1184:515f94dc16f2
>>> user:        Robert Bradshaw <robertwb at math.washington.edu>
>>> date:        Mon Sep 29 19:08:45 2008 -0700
>>> summary:     Refactoring of type parsing
>>>
>>> you cannot parse C function typedef like this:
>>>
>>> cdef extern from *:
>>>     void (somefunc)(int)
>>>
>>> Please note that this declaration is not of the form "void
>>> (*somefuncptr)(int)", and AFAIK the former is valid C code.
>>
>> Does it have the same meaning, or is there something more subtle
>> going on?
>
> Robert, AFAIK they do not have the same meaning. the void
> (func1)(args) is function typedef and void (*func2)(args) is a funtion
> POINTER typedef. In short, they are really different types. MPI has
> many of those (non-pointer) typedef for some callback functions.

I understand that they are different types, I thought the one could  
be used wherever the other is needed though.

> I really believe that this is a regression. It could potentially make
> invalid some codes. In fact, this change already broke my codes.

OK, I'll look into this some more then. Being able to use "cimport *"  
to import types is why I had to make these changes.

>> We don't accept all valid C code, for example
>>
>>     int foo(void)
>>
>> is allowed in C but not in Cython.
>
> Long ago I discussed that case with Sfefan, and we agreed that there
> was not point on supporting 'int foo(void)', as that C declaration was
> in fact related to support ancient C and the different meaning of 'int
> foo()', hopefully fixed in C++.
>
> But in the fase of function vs. function pointer typedef's, the issues
> are different. They really are different types.
>
>
>>> I was looking at the changes you pushed, but I was unable to  
>>> discover
>>> what's going on. So I really need your help here.
>>
>> I needed to remove the dependancy the parser had on knowing all type
>> names, so in some places it needs to look ahead to determine if it's
>> looking at a type or an expression. If this is the only bug
>> introduced, than I'm willing to accept that, but are there other
>> statements that are now illegal?
>>
>> - Robert
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>
>
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From greg.ewing at canterbury.ac.nz  Sat Oct  4 01:04:31 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Oct 2008 11:04:31 +1200
Subject: [Cython] Robert, please help!
In-Reply-To: <e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
Message-ID: <48E6A4FF.1070900@canterbury.ac.nz>

Lisandro Dalcin wrote:

> Long ago I discussed that case with Sfefan, and we agreed that there
> was not point on supporting 'int foo(void)', as that C declaration was
> in fact related to support ancient C and the different meaning of 'int
> foo()', hopefully fixed in C++.

Yes, in C++, all function headers are prototypes, so
there's no need for f(void). It seems to be accepted
for backward compatibility with C, but Pyrex/Cython
doesn't have to be backward compatible with anything,
so there's no need to support it.

As for (f)(), I'm not sure why you would ever have to
write that instead of f(). However, there's a degenerate
case of that when you have an empty declarator, e.g.
in a cast, where ()() means a function, whereas just
() on its own is regarded as a syntax error by C
compilers.

I suspect this isn't important, though, because casting
something to a function (as opposed to a function pointer)
isn't something I can think of a use for.

Nevertheless, (f)() is valid C declaration syntax, and
Pyrex aims to allow any valid C declaration as far as
reasonably possible. So I'd recommend fixing this if
it won't be too difficult.

-- 
Greg

From dalcinl at gmail.com  Sat Oct  4 05:46:03 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 4 Oct 2008 00:46:03 -0300
Subject: [Cython] Robert, please help!
In-Reply-To: <48E6A4FF.1070900@canterbury.ac.nz>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
	<48E6A4FF.1070900@canterbury.ac.nz>
Message-ID: <e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>

On Fri, Oct 3, 2008 at 8:04 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>
>> Long ago I discussed that case with Sfefan, and we agreed that there
>> was not point on supporting 'int foo(void)', as that C declaration was
>> in fact related to support ancient C and the different meaning of 'int
>> foo()', hopefully fixed in C++.
>
> Yes, in C++, all function headers are prototypes, so
> there's no need for f(void). It seems to be accepted
> for backward compatibility with C, but Pyrex/Cython
> doesn't have to be backward compatible with anything,
> so there's no need to support it.
>
> As for (f)(), I'm not sure why you would ever have to
> write that instead of f(). However, there's a degenerate
> case of that when you have an empty declarator, e.g.
> in a cast, where ()() means a function, whereas just
> () on its own is regarded as a syntax error by C
> compilers.

Greg, I?m not a C lawyer, but as reference, the MPI standard uses
"typedef int (f)(args)" everywhere, for example:
<http://www.mpi-forum.org/docs/mpi21-report-bw/node147.htm>.

>
> I suspect this isn't important, though, because casting
> something to a function (as opposed to a function pointer)
> isn't something I can think of a use for.
>
> Nevertheless, (f)() is valid C declaration syntax, and
> Pyrex aims to allow any valid C declaration as far as
> reasonably possible. So I'd recommend fixing this if
> it won't be too difficult.
>
> --
> Greg
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Sat Oct  4 06:04:38 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 04 Oct 2008 16:04:38 +1200
Subject: [Cython] Robert, please help!
In-Reply-To: <e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
	<48E6A4FF.1070900@canterbury.ac.nz>
	<e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>
Message-ID: <48E6EB56.9090605@canterbury.ac.nz>

Lisandro Dalcin wrote:

> Greg, I?m not a C lawyer, but as reference, the MPI standard uses
> "typedef int (f)(args)" everywhere

I just tried an experiment, and gcc seems to acccept
either

   typedef int f(void);

or

   typedef int (f)(void);

-- 
Greg

From dalcinl at gmail.com  Sat Oct  4 06:29:22 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 4 Oct 2008 01:29:22 -0300
Subject: [Cython] Robert, please help!
In-Reply-To: <48E6EB56.9090605@canterbury.ac.nz>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
	<48E6A4FF.1070900@canterbury.ac.nz>
	<e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>
	<48E6EB56.9090605@canterbury.ac.nz>
Message-ID: <e7ba66e40810032129u8e75f6fy12063ada1c90042e@mail.gmail.com>

OK, at this link http://publications.gbdirect.co.uk/c_book/chapter8/typedef.html

I've found the following example:

/*
* Using typedef, declare 'func' to have type
* 'function taking two int arguments, returning int'
*/
typedef int func(int, int);
/* ERROR */
func func_name{ /*....*/ }
/* Correct. Returns pointer to a type 'func' */
func *func_name(){ /*....*/ }
/*
* Correct if functions could return functions,
* but C can't.
*/
func func_name(){ /*....*/ }


I'm fine with that. So I?ll try tomorrow if Robert?s modifications can
handle this form. Thank's for the point, Greg!


On Sat, Oct 4, 2008 at 1:04 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>
>> Greg, I?m not a C lawyer, but as reference, the MPI standard uses
>> "typedef int (f)(args)" everywhere
>
> I just tried an experiment, and gcc seems to acccept
> either
>
>   typedef int f(void);
>
> or
>
>   typedef int (f)(void);
>
> --
> Greg
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Sat Oct  4 13:42:20 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 4 Oct 2008 04:42:20 -0700
Subject: [Cython] Pure python mode
Message-ID: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>

I polished off some more of the pure python mode stuff tonight. I  
just pushed it to cython-devel. Now you can do stuff like:

> import cython  # you need a single file to be in your path to use  
> it in pure Python mode, provided (and installed) with Cython
>
> @cython.locals(x=int, y=cython.p_int)
> def foo(x):
>     y = cython.cast(cython.p_int, x)
>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>     if cython.compiled:
>         print "the compiler was run
>     else:
>         print "just being interpreted"
>
> x = cython.declare(int)
> xx = cython.declare(cython.p_int)
> xx = cython.address(x)
>
> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>
> a = cython.declare(MyStruct)
> a.x = 5
>
> T = cython.typedef(MyStruct)
> b = cython.declare(cython.pointer(T))
> b[0].x = 4


On top of this, you can write a .pxd file to accompany your .py file  
and it will coerce your .py file classes and def statements into cdef  
and cpdef versions.

It's still very new and probably highly experimental (though it only  
modifies the cython.* nodes, so it shouldn't cause any regressions).  
Let me know what you think.

- Robert


From dagss at student.matnat.uio.no  Sat Oct  4 14:45:00 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: 04 Oct 2008 14:45:00 +0200
Subject: [Cython] Pure python mode
Message-ID: <3305976321.875339@smtp.netcom.no>

Robert wrote:

> I polished off some more of the pure python mode stuff tonight. I  
>just pushed it to cython-devel. Now you can do stuff like:
>

Excellent! I have some comments to the syntax of course :-)

Do you see this as the way forward for all new Cython code, or just when one has interpreter compatability requirements? I suppose Sage will want to stay consistent with the current code, while for NumPy/numerics one could for instance rewrite the tutorial etc. to this syntax. It probably is more cumbersome for daily usage though...

>> import cython  # you need a single file to be in your path to use  
> it in pure Python mode, provided (and installed) with Cython
>
>> @cython.locals(x=int, y=cython.p_int)
> def foo(x):
>     y = cython.cast(cython.p_int, x)
>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>     if cython.compiled:
>         print "the compiler was run
>     else:
>         print "just being interpreted"
>
>> x = cython.declare(int)
> xx = cython.declare(cython.p_int)
> xx = cython.address(x)

Can you also do something like

from cython import cast

? It doesn't work for directives but I consider that a bug...

As for syntax, I have a problem with the assignment style, i.e. I'd rather see:

cython.declare(x=cython.int)

because it seems more magical, and so one isn't surprised by stuff like this:

x = cython.declare(cython.int)
x = "hello" # illegal? Huh?

On the brainstorming side, I was wondering if perhaps both casting and declaration could be done via constructor:

x = cython.int(3)

It has the same problem as your cython.declare though so I won't support that at this stage (but with inference it could work).

Another alternative:

with cython.locals(..): ...

The thing is, "with" sets up a natural scope block, and so it could be possible to declare variables e.g. inside a for loop with such a construct. A bit verbose though.

>
>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>
>> a = cython.declare(MyStruct)
> a.x = 5
>
>> T = cython.typedef(MyStruct)
> b = cython.declare(cython.pointer(T))
> b[0].x = 4

Nice that all of this is working!

Brainstorming again:

I feel that this alternative to typedefs may not be too much work, just mentioning it:

cdef class A: pass
B = A # both symbol ass. and typedef

@cython.locals(x=B) # type..
def f():
    x = B() # module var

i.e. B = A be both a variable assignment *and* a typedef. This would only be for top-level module assignments, so

if True:
    C = A
@cython.locals(x=C) #illegal!
def f():
    x = C() # legal 

furthermore,

myint = cython.int

would only trigger a typedef as int is not a variable at runtime.

Take it for what it is (a brainstorming); if the assignment style is changed to statement style for cython.declare and cython.typedef I'm 100 percent happy, and quite happy for all of this even if it stays like now.

Dag Sverre


From robertwb at math.washington.edu  Sat Oct  4 21:17:55 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 4 Oct 2008 12:17:55 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <3305976321.875339@smtp.netcom.no>
References: <3305976321.875339@smtp.netcom.no>
Message-ID: <B9ECD74B-B418-4CC2-AB05-C013BBB5E23E@math.washington.edu>

On Oct 4, 2008, at 5:45 AM, Dag Sverre Seljebotn wrote:

> Robert wrote:
>
>> I polished off some more of the pure python mode stuff tonight. I
>> just pushed it to cython-devel. Now you can do stuff like:
>>
>
> Excellent! I have some comments to the syntax of course :-)

Thanks. Input is good.

> Do you see this as the way forward for all new Cython code, or just  
> when one has interpreter compatability requirements? I suppose Sage  
> will want to stay consistent with the current code, while for NumPy/ 
> numerics one could for instance rewrite the tutorial etc. to this  
> syntax. It probably is more cumbersome for daily usage though...

Both, but primarily for interpreter compatibility. Stuff like SymPy  
is my main target for things like this, and it will facilitate even  
easier migration. It will also be useful for bootstrapping the Cython  
compiler. The current syntax certainly is less cumbersome when  
dealing with a lot of c-ish code, and I don't anticipate existing  
codebases to be changing (not Sage at least, we've got more  
interesting things to work on...).

>>> import cython  # you need a single file to be in your path to use
>> it in pure Python mode, provided (and installed) with Cython
>>
>>> @cython.locals(x=int, y=cython.p_int)
>> def foo(x):
>>     y = cython.cast(cython.p_int, x)
>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>     if cython.compiled:
>>         print "the compiler was run
>>     else:
>>         print "just being interpreted"
>>
>>> x = cython.declare(int)
>> xx = cython.declare(cython.p_int)
>> xx = cython.address(x)
>
> Can you also do something like
>
> from cython import cast
>
> ? It doesn't work for directives but I consider that a bug...

No, it doesn't yet, but it will. I see an easy way to implement that.

> As for syntax, I have a problem with the assignment style, i.e. I'd  
> rather see:
>
> cython.declare(x=cython.int)
>
> because it seems more magical, and so one isn't surprised by stuff  
> like this:
>
> x = cython.declare(cython.int)
> x = "hello" # illegal? Huh?

I could certainly see supporting that form as well. The advantage of  
the x = cython.declare(...) syntax is that it is a declaration in the  
interpreter as well, so you can do stuff like

x = cython.declare(some_struct)
init_struct(x)

Another thing that might be good to support is

x = cython.declare(cython.int, 3)

> On the brainstorming side, I was wondering if perhaps both casting  
> and declaration could be done via constructor:
>
> x = cython.int(3)
>
> It has the same problem as your cython.declare though so I won't  
> support that at this stage (but with inference it could work).

Hmm... I suppose currently when A is a (cython defined) cdef class, A 
(3) *always* returns an A, but this may not be the case in the  
future. Or would it only work for primitive types? I could certainly  
see this working more naturally once type inference is up and running  
(and the need for most declarations would hopefully go away).

> Another alternative:
>
> with cython.locals(..): ...
>
> The thing is, "with" sets up a natural scope block,

This is actually a disadvantage, as neither Cython nor Python have  
the idea of scope blocks (only locals and globals).

> and so it could be possible to declare variables e.g. inside a for  
> loop with such a construct. A bit verbose though.
>
>>
>>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>
>>> a = cython.declare(MyStruct)
>> a.x = 5
>>
>>> T = cython.typedef(MyStruct)
>> b = cython.declare(cython.pointer(T))
>> b[0].x = 4
>
> Nice that all of this is working!
>
> Brainstorming again:
>
> I feel that this alternative to typedefs may not be too much work,  
> just mentioning it:
>
> cdef class A: pass
> B = A # both symbol ass. and typedef
>
> @cython.locals(x=B) # type..
> def f():
>     x = B() # module var
>
> i.e. B = A be both a variable assignment *and* a typedef. This  
> would only be for top-level module assignments, so
>
> if True:
>     C = A
> @cython.locals(x=C) #illegal!
> def f():
>     x = C() # legal
>
> furthermore,
>
> myint = cython.int
>
> would only trigger a typedef as int is not a variable at runtime.

The assignment-on-typedef seems a bit too magical for me...  
especially once we support all types as first-level objects (which I  
hope we do). This would also make

x = A
...
x = B

fail for "mysterious" reasons when one of A or B are defined to be  
types. Note that I tried to make it such that every place typing  
information is used, a type is required.

> Take it for what it is (a brainstorming); if the assignment style  
> is changed to statement style for cython.declare and cython.typedef  
> I'm 100 percent happy, and quite happy for all of this even if it  
> stays like now.

Brainstorming is good, thanks for the feedback.

- Robert


From kevinar18 at hotmail.com  Sat Oct  4 22:03:12 2008
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Sat, 4 Oct 2008 16:03:12 -0400
Subject: [Cython] What to do with modules
Message-ID: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>


As I understand the Docs, you can't access c types, functions, or classes in a module.  For example, this won't work:


somemodule.pyx

cdef int a = 1
py_get():
    return a

somefile.pyx

import somemodule
cdef int b = somemodule.a
print somemodule.py_get()

or this:

somemodule.pyx

cdef extern from *:
    int a = 1
py_get():
    return a

somefile.pyx

cimport somemodule
cdef int b = somemodule.a



So what do you do with modules that mix Cython and Python?
_________________________________________________________________
See how Windows connects the people, information, and fun that are part of your life.
http://clk.atdmt.com/MRT/go/msnnkwxp1020093175mrt/direct/01/

From jasone at canonware.com  Sat Oct  4 23:36:57 2008
From: jasone at canonware.com (Jason Evans)
Date: Sat, 04 Oct 2008 14:36:57 -0700
Subject: [Cython] Namespace semantics differences from Python
Message-ID: <48E7E1F9.7000103@canonware.com>

I am in the process of converting a largish Python program to Cython. 
One thing my software does throughout is to create package-specific 
exception types, so that exception handler code can accurately catch 
only certain types of exceptions.  For example, one subpackage defines 
the following exception types:

----
import Crux

class Exception(Crux.Exception):
     pass

class ValueError(Exception, ValueError):
     def __init__(self, str):
         self._str = str

     def __str__(self):
         return self._str
----

As you can see, this code defines module-specific classes with the same 
names as the top-level Python exception types.  The Cython compiler 
complains with errors like:

/home/jasone/crux/crux/./src/Crux/CTMatrix.pyx:19:0: Assignment to 
non-lvalue 'Exception'

Is this a "feature" or a bug in Cython?

Thanks,
Jason

From jasone at canonware.com  Sun Oct  5 00:46:12 2008
From: jasone at canonware.com (Jason Evans)
Date: Sat, 04 Oct 2008 15:46:12 -0700
Subject: [Cython] Namespace semantics differences from Python
In-Reply-To: <48E7E1F9.7000103@canonware.com>
References: <48E7E1F9.7000103@canonware.com>
Message-ID: <48E7F234.2010800@canonware.com>

Jason Evans wrote:
> [...]
> 
> Is this a "feature" or a bug in Cython?

The answer appears to be "none of the above"; the bug was in my code. 
There was an inconsistency in the .pxd file that I neglected to check 
for before sending email.

Jason

From robertwb at math.washington.edu  Sun Oct  5 06:20:01 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 4 Oct 2008 21:20:01 -0700
Subject: [Cython] What to do with modules
In-Reply-To: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
References: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
Message-ID: <9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>

On Oct 4, 2008, at 1:03 PM, Kevin Ar18 wrote:

> As I understand the Docs, you can't access c types, functions, or  
> classes in a module.

I'm not quite sure I understand your question, but I'll try to answer...

Python stores its module-level in a big dict. The speed gains of  
Cython are usually made by avoiding this dict storage (as well as the  
wrapping everything as an object that is required to do so).

> For example, this won't work:
>
>
> somemodule.pyx
>
> cdef int a = 1
> py_get():
>     return a
>
> somefile.pyx
>
> import somemodule
> cdef int b = somemodule.a

This won't, as a is not a Python object but a c global.

> print somemodule.py_get()

This should.

>
> or this:
>
> somemodule.pyx
>
> cdef extern from *:
>     int a = 1
> py_get():
>     return a
>
> somefile.pyx
>
> cimport somemodule
> cdef int b = somemodule.a

No, that won't either.

> So what do you do with modules that mix Cython and Python?

cdef classes live in both the C and Python namespace. cpdef functions  
are accessible from both C and Python as well. In this case, you  
probably need to just write

a = 1

and deal with it being a Python object so you can get at it from  
Python (and it is still accessible from C). Everything accessible  
from Python is also accessible from Cython.

We probably should support cimporting cdef'd global variables  
(probably with the same trick as we do for functions) and it would be  
nice to be able to declare variables as "public" or "readonly" as one  
can do with class attributes (though the only way I could see how to  
do it is the hack at http://mail.python.org/pipermail/python-list/ 
2007-January/422578.html or making a subtype of the module type)

- Robert


From ondrej at certik.cz  Mon Oct  6 00:46:42 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 6 Oct 2008 00:46:42 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
Message-ID: <85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>

On Sat, Oct 4, 2008 at 1:42 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> I polished off some more of the pure python mode stuff tonight. I
> just pushed it to cython-devel. Now you can do stuff like:
>
>> import cython  # you need a single file to be in your path to use
>> it in pure Python mode, provided (and installed) with Cython
>>
>> @cython.locals(x=int, y=cython.p_int)
>> def foo(x):
>>     y = cython.cast(cython.p_int, x)
>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>     if cython.compiled:
>>         print "the compiler was run
>>     else:
>>         print "just being interpreted"
>>
>> x = cython.declare(int)
>> xx = cython.declare(cython.p_int)
>> xx = cython.address(x)
>>
>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>
>> a = cython.declare(MyStruct)
>> a.x = 5
>>
>> T = cython.typedef(MyStruct)
>> b = cython.declare(cython.pointer(T))
>> b[0].x = 4
>
>
> On top of this, you can write a .pxd file to accompany your .py file
> and it will coerce your .py file classes and def statements into cdef
> and cpdef versions.
>
> It's still very new and probably highly experimental (though it only
> modifies the cython.* nodes, so it shouldn't cause any regressions).
> Let me know what you think.

Thanks a lot for doing this!

I installed cython-devel and then I tried this little example:


--------
import Cython as cython

@cython.locals(x=int, y=cython.p_int)
def foo(x):
    y = cython.cast(cython.p_int, x)
    print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
    if cython.compiled:
        print "the compiler was run"
    else:
        print "just being interpreted"

foo(5)
-----------

When running in it in Python, it gives:

  File "t.py", line 12, in <module>
    foo(5)
  File "t.py", line 5, in foo
    y = cython.cast(cython.p_int, x)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
    return type(arg)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in __init__
    raise ValueError
ValueError


Am I using it incorrectly? Btw, when I compile it with Cython:

$ python -c "import t; t.foo(5)"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "t.py", line 12, in t (t.c:515)
    foo(5)
  File "t.py", line 5, in t.foo (t.c:265)
    y = cython.cast(cython.p_int, x)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
    return type(arg)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in __init__
    raise ValueError
ValueError


Looking at the code, I need to pass it Cython p_int integers, right? I
also tried:

x = cython.declare(int)
xx = cython.declare(cython.p_int)
xx = cython.address(x)
foo(xx)

But this also raises the same error as above. But otherwise this is
exactly the way to go, I am really looking forward to just having one
codebase for both compiled and interpreted stuff. Unfortunately, I got
again busy lately to implement this myself, so thanks again for the
work.

Ondrej

From uschmitt at mineway.de  Mon Oct  6 12:38:14 2008
From: uschmitt at mineway.de (Uwe Schmitt)
Date: Mon, 06 Oct 2008 12:38:14 +0200
Subject: [Cython] [mailinglist] Re: resolving name conflict -- does not
 work for enums !?
In-Reply-To: <48E5F81B.2000600@canterbury.ac.nz>
References: <48E4E7DD.1040906@mineway.de> <48E5F81B.2000600@canterbury.ac.nz>
Message-ID: <48E9EA96.2040809@mineway.de>

Greg Ewing schrieb:
> You shouldn't be cimporting minimal in minimal.pyx. The files
> minimal.pxd and minimal.pyx are two parts of the *same* module,
> i.e. minimal. There's only one namespace, and you're trying
> to define A as both a C constant and a Python global in that
> namespace.
>
> You need to put the definition of enum test into a different
> module namespace, e.g.
>
>    # mindefs.pxd
>    cdef enum test:
>      A
>
>    # minimal.pyx:
>    cimport mindefs
>    A = mindefs.A
>
> Note that you don't need a mindefs.pyx in this case -- you're
> only using mindefs as a compile-time namespace.
>   

Thanks, that works now.
Greetings, Uwe

-- 
Dr. rer. nat. Uwe Schmitt
F&E Mathematik
 
mineway GmbH
Science Park 2
D-66123 Saarbr?cken
 
Telefon: +49 (0)681 8390 5334
Telefax: +49 (0)681 830 4376
 
uschmitt at mineway.de
www.mineway.de
 
Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer
Amtsgericht Saarbr?cken HRB 12339



From uschmitt at mineway.de  Mon Oct  6 18:03:44 2008
From: uschmitt at mineway.de (Uwe Schmitt)
Date: Mon, 06 Oct 2008 18:03:44 +0200
Subject: [Cython] wrapping c code, classmethod question
Message-ID: <48EA36E0.1010209@mineway.de>

Hi,

I'm trying to wrap an existing C entension as follows:

    cdef struct A:
        double * values
        int n


    cdef class Test:
        cdef A a

        @classmethod
        def buildFromData(cls, filename):

            cdef A a # = foofoo(filename)

            obj = cls()
            obj.a = a
            return obj

Running Cython the "obj.a=a" results in """Cannot convert 'A' to Python 
object""".

So: how can I instantiate extension classes from classmethods ?

Greetings, Uwe


-- 
Dr. rer. nat. Uwe Schmitt
F&E Mathematik
 
mineway GmbH
Science Park 2
D-66123 Saarbr?cken
 
Telefon: +49 (0)681 8390 5334
Telefax: +49 (0)681 830 4376
 
uschmitt at mineway.de
www.mineway.de
 
Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer
Amtsgericht Saarbr?cken HRB 12339


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081006/e1c0ff8e/attachment.htm 

From dagss at student.matnat.uio.no  Mon Oct  6 18:16:04 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 6 Oct 2008 18:16:04 +0200 (CEST)
Subject: [Cython] wrapping c code, classmethod question
In-Reply-To: <48EA36E0.1010209@mineway.de>
References: <48EA36E0.1010209@mineway.de>
Message-ID: <33372.88.90.248.62.1223309764.squirrel@webmail.uio.no>

Uwe Schmitt wrote:
> Hi,
>
> I'm trying to wrap an existing C entension as follows:
>
>     cdef struct A:
>         double * values
>         int n
>
>
>     cdef class Test:
>         cdef A a
>
>         @classmethod
>         def buildFromData(cls, filename):
>
>             cdef A a # = foofoo(filename)
>
>             obj = cls()
>             obj.a = a
>             return obj
>
> Running Cython the "obj.a=a" results in """Cannot convert 'A' to Python
> object""".

Because "obj.a" is an attribute of the Python object "obj", while a is a C
struct...

I.e. you need to type "cdef Test obj = cls()" instead, so that it is known
compile-time that "obj" will be an instance of Test and thus have a typed
"a" field.

(I don't know myself whether classmethod works as intended for cdef
classes either, but this should be one obstacle less...)

Dag Sverre



From robert.kern at gmail.com  Mon Oct  6 18:47:27 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 11:47:27 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
Message-ID: <gcdfev$8p3$1@ger.gmane.org>

Robert Bradshaw wrote:
> I polished off some more of the pure python mode stuff tonight. I  
> just pushed it to cython-devel. Now you can do stuff like:
> 
>> import cython  # you need a single file to be in your path to use  
>> it in pure Python mode, provided (and installed) with Cython

Hmm, does this cause a problem on systems with case-insensitive files? cython.py 
and Cython/ won't actually conflict on the filesystem, but I suspect the import 
mechanism may get confused.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From robertwb at math.washington.edu  Mon Oct  6 19:04:44 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 10:04:44 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
Message-ID: <A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>

On Oct 5, 2008, at 3:46 PM, Ondrej Certik wrote:

> On Sat, Oct 4, 2008 at 1:42 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> I polished off some more of the pure python mode stuff tonight. I
>> just pushed it to cython-devel. Now you can do stuff like:
>>
>>> import cython  # you need a single file to be in your path to use
>>> it in pure Python mode, provided (and installed) with Cython
>>>
>>> @cython.locals(x=int, y=cython.p_int)
>>> def foo(x):
>>>     y = cython.cast(cython.p_int, x)
>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>     if cython.compiled:
>>>         print "the compiler was run
>>>     else:
>>>         print "just being interpreted"
>>>
>>> x = cython.declare(int)
>>> xx = cython.declare(cython.p_int)
>>> xx = cython.address(x)
>>>
>>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>>
>>> a = cython.declare(MyStruct)
>>> a.x = 5
>>>
>>> T = cython.typedef(MyStruct)
>>> b = cython.declare(cython.pointer(T))
>>> b[0].x = 4
>>
>>
>> On top of this, you can write a .pxd file to accompany your .py file
>> and it will coerce your .py file classes and def statements into cdef
>> and cpdef versions.
>>
>> It's still very new and probably highly experimental (though it only
>> modifies the cython.* nodes, so it shouldn't cause any regressions).
>> Let me know what you think.
>
> Thanks a lot for doing this!

You're welcome.

> I installed cython-devel and then I tried this little example:
>
>
> --------
> import Cython as cython
>
> @cython.locals(x=int, y=cython.p_int)
> def foo(x):
>     y = cython.cast(cython.p_int, x)
>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>     if cython.compiled:
>         print "the compiler was run"
>     else:
>         print "just being interpreted"
>
> foo(5)
> -----------
>
> When running in it in Python, it gives:
>
>   File "t.py", line 12, in <module>
>     foo(5)
>   File "t.py", line 5, in foo
>     y = cython.cast(cython.p_int, x)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>     return type(arg)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in  
> __init__
>     raise ValueError
> ValueError
>
>
> Am I using it incorrectly?

No, it's just not letting you cast an int into an p_int. I suppose it  
should? It just doesn't make sense to emulate this in Python.

> Btw, when I compile it with Cython:
>
> $ python -c "import t; t.foo(5)"
> Traceback (most recent call last):
>   File "<string>", line 1, in <module>
>   File "t.py", line 12, in t (t.c:515)
>     foo(5)
>   File "t.py", line 5, in t.foo (t.c:265)
>     y = cython.cast(cython.p_int, x)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>     return type(arg)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in  
> __init__
>     raise ValueError
> ValueError

You need to import cython, not import Cython as cython.

> Looking at the code, I need to pass it Cython p_int integers, right? I
> also tried:
>
> x = cython.declare(int)
> xx = cython.declare(cython.p_int)
> xx = cython.address(x)
> foo(xx)
>
> But this also raises the same error as above.

This should work, Again, I think it's because you imported Cython  
rather than cython.

> But otherwise this is
> exactly the way to go, I am really looking forward to just having one
> codebase for both compiled and interpreted stuff. Unfortunately, I got
> again busy lately to implement this myself, so thanks again for the
> work.
>
> Ondrej
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Mon Oct  6 19:06:24 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 10:06:24 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <gcdfev$8p3$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
Message-ID: <BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>

On Oct 6, 2008, at 9:47 AM, Robert Kern wrote:

> Robert Bradshaw wrote:
>> I polished off some more of the pure python mode stuff tonight. I
>> just pushed it to cython-devel. Now you can do stuff like:
>>
>>> import cython  # you need a single file to be in your path to use
>>> it in pure Python mode, provided (and installed) with Cython
>
> Hmm, does this cause a problem on systems with case-insensitive  
> files? cython.py
> and Cython/ won't actually conflict on the filesystem, but I  
> suspect the import
> mechanism may get confused.

Currently all the emulating stuff is in Cython/Shadow.py. Both Cython/ 
__init__.py and cython.py import it, thus it should work on both case- 
sensitive and case-insensitive filesystems.

- Robert




From robert.kern at gmail.com  Mon Oct  6 19:18:40 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 12:18:40 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
Message-ID: <gcdh9g$hkh$1@ger.gmane.org>

Robert Bradshaw wrote:
> On Oct 6, 2008, at 9:47 AM, Robert Kern wrote:
> 
>> Robert Bradshaw wrote:
>>> I polished off some more of the pure python mode stuff tonight. I
>>> just pushed it to cython-devel. Now you can do stuff like:
>>>
>>>> import cython  # you need a single file to be in your path to use
>>>> it in pure Python mode, provided (and installed) with Cython
>> Hmm, does this cause a problem on systems with case-insensitive  
>> files? cython.py
>> and Cython/ won't actually conflict on the filesystem, but I  
>> suspect the import
>> mechanism may get confused.
> 
> Currently all the emulating stuff is in Cython/Shadow.py. Both Cython/ 
> __init__.py and cython.py import it, thus it should work on both case- 
> sensitive and case-insensitive filesystems.

Cython/Shadow.py is not what I'm concerned about. It's the presence of both 
cython.py and Cython/__init__.py. I'm wondering if the statement "import cython" 
might actually import Cython/__init__.py on some case-insensitive file system.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ondrej at certik.cz  Mon Oct  6 19:25:53 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 6 Oct 2008 19:25:53 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
Message-ID: <85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>

On Mon, Oct 6, 2008 at 7:04 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 5, 2008, at 3:46 PM, Ondrej Certik wrote:
>
>> On Sat, Oct 4, 2008 at 1:42 PM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>> I polished off some more of the pure python mode stuff tonight. I
>>> just pushed it to cython-devel. Now you can do stuff like:
>>>
>>>> import cython  # you need a single file to be in your path to use
>>>> it in pure Python mode, provided (and installed) with Cython
>>>>
>>>> @cython.locals(x=int, y=cython.p_int)
>>>> def foo(x):
>>>>     y = cython.cast(cython.p_int, x)
>>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>>     if cython.compiled:
>>>>         print "the compiler was run
>>>>     else:
>>>>         print "just being interpreted"
>>>>
>>>> x = cython.declare(int)
>>>> xx = cython.declare(cython.p_int)
>>>> xx = cython.address(x)
>>>>
>>>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>>>
>>>> a = cython.declare(MyStruct)
>>>> a.x = 5
>>>>
>>>> T = cython.typedef(MyStruct)
>>>> b = cython.declare(cython.pointer(T))
>>>> b[0].x = 4
>>>
>>>
>>> On top of this, you can write a .pxd file to accompany your .py file
>>> and it will coerce your .py file classes and def statements into cdef
>>> and cpdef versions.
>>>
>>> It's still very new and probably highly experimental (though it only
>>> modifies the cython.* nodes, so it shouldn't cause any regressions).
>>> Let me know what you think.
>>
>> Thanks a lot for doing this!
>
> You're welcome.
>
>> I installed cython-devel and then I tried this little example:
>>
>>
>> --------
>> import Cython as cython
>>
>> @cython.locals(x=int, y=cython.p_int)
>> def foo(x):
>>     y = cython.cast(cython.p_int, x)
>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>     if cython.compiled:
>>         print "the compiler was run"
>>     else:
>>         print "just being interpreted"
>>
>> foo(5)
>> -----------
>>
>> When running in it in Python, it gives:
>>
>>   File "t.py", line 12, in <module>
>>     foo(5)
>>   File "t.py", line 5, in foo
>>     y = cython.cast(cython.p_int, x)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>>     return type(arg)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>> __init__
>>     raise ValueError
>> ValueError
>>
>>
>> Am I using it incorrectly?
>
> No, it's just not letting you cast an int into an p_int. I suppose it
> should? It just doesn't make sense to emulate this in Python.

No, you are right, I was just trying to get it run.

>
>> Btw, when I compile it with Cython:
>>
>> $ python -c "import t; t.foo(5)"
>> Traceback (most recent call last):
>>   File "<string>", line 1, in <module>
>>   File "t.py", line 12, in t (t.c:515)
>>     foo(5)
>>   File "t.py", line 5, in t.foo (t.c:265)
>>     y = cython.cast(cython.p_int, x)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>>     return type(arg)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>> __init__
>>     raise ValueError
>> ValueError
>
> You need to import cython, not import Cython as cython.
>
>> Looking at the code, I need to pass it Cython p_int integers, right? I
>> also tried:
>>
>> x = cython.declare(int)
>> xx = cython.declare(cython.p_int)
>> xx = cython.address(x)
>> foo(xx)
>>
>> But this also raises the same error as above.
>
> This should work, Again, I think it's because you imported Cython
> rather than cython.


I tried that too:

ondra at fuji:~/ext/cython-pure$ cat t.py
import cython

@cython.locals(x=int, y=cython.p_int)
def foo(x):
    y = cython.cast(cython.p_int, x)
    print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
    if cython.compiled:
        print "the compiler was run"
    else:
        print "just being interpreted"

#foo(5)

x = cython.declare(int)
xx = cython.declare(cython.p_int)
xx = cython.address(x)
foo(xx)
ondra at fuji:~/ext/cython-pure$ python t.py
Traceback (most recent call last):
  File "t.py", line 17, in <module>
    foo(xx)
  File "t.py", line 5, in foo
    y = cython.cast(cython.p_int, x)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
    return type(arg)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in __init__
    raise ValueError
ValueError




With the same output. I copied the cython.py from cython-devel, is
that correct? I am probably doing something stupid. The error is
really simple and it fails, but currently I am not sure what it is
supposed to do correctly.

Ondrej

From robert.kern at gmail.com  Mon Oct  6 19:27:14 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 12:27:14 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <gcdh9g$hkh$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org>
Message-ID: <gcdhpi$jsr$1@ger.gmane.org>

Robert Kern wrote:
> Robert Bradshaw wrote:
>> On Oct 6, 2008, at 9:47 AM, Robert Kern wrote:
>>
>>> Robert Bradshaw wrote:
>>>> I polished off some more of the pure python mode stuff tonight. I
>>>> just pushed it to cython-devel. Now you can do stuff like:
>>>>
>>>>> import cython  # you need a single file to be in your path to use
>>>>> it in pure Python mode, provided (and installed) with Cython
>>> Hmm, does this cause a problem on systems with case-insensitive  
>>> files? cython.py
>>> and Cython/ won't actually conflict on the filesystem, but I  
>>> suspect the import
>>> mechanism may get confused.
>> Currently all the emulating stuff is in Cython/Shadow.py. Both Cython/ 
>> __init__.py and cython.py import it, thus it should work on both case- 
>> sensitive and case-insensitive filesystems.
> 
> Cython/Shadow.py is not what I'm concerned about. It's the presence of both 
> cython.py and Cython/__init__.py. I'm wondering if the statement "import cython" 
> might actually import Cython/__init__.py on some case-insensitive file system.

Oh, never mind. I see what you're saying. But then why do we need cython.py if 
everything is provided by Cython/__init__.py?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From ondrej at certik.cz  Mon Oct  6 19:35:00 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 6 Oct 2008 19:35:00 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <gcdhpi$jsr$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <gcdhpi$jsr$1@ger.gmane.org>
Message-ID: <85b5c3130810061035j214ae166v8768723a0cf02cfd@mail.gmail.com>

On Mon, Oct 6, 2008 at 7:27 PM, Robert Kern <robert.kern at gmail.com> wrote:
> Robert Kern wrote:
>> Robert Bradshaw wrote:
>>> On Oct 6, 2008, at 9:47 AM, Robert Kern wrote:
>>>
>>>> Robert Bradshaw wrote:
>>>>> I polished off some more of the pure python mode stuff tonight. I
>>>>> just pushed it to cython-devel. Now you can do stuff like:
>>>>>
>>>>>> import cython  # you need a single file to be in your path to use
>>>>>> it in pure Python mode, provided (and installed) with Cython
>>>> Hmm, does this cause a problem on systems with case-insensitive
>>>> files? cython.py
>>>> and Cython/ won't actually conflict on the filesystem, but I
>>>> suspect the import
>>>> mechanism may get confused.
>>> Currently all the emulating stuff is in Cython/Shadow.py. Both Cython/
>>> __init__.py and cython.py import it, thus it should work on both case-
>>> sensitive and case-insensitive filesystems.
>>
>> Cython/Shadow.py is not what I'm concerned about. It's the presence of both
>> cython.py and Cython/__init__.py. I'm wondering if the statement "import cython"
>> might actually import Cython/__init__.py on some case-insensitive file system.
>
> Oh, never mind. I see what you're saying. But then why do we need cython.py if
> everything is provided by Cython/__init__.py?

Exactly. Also it behaves identically (fails) on my system.

Ondrej

From dagss at student.matnat.uio.no  Mon Oct  6 19:38:10 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 6 Oct 2008 19:38:10 +0200 (CEST)
Subject: [Cython] Pure python mode
In-Reply-To: <gcdhpi$jsr$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <gcdhpi$jsr$1@ger.gmane.org>
Message-ID: <32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>

> Robert Kern wrote:
> Oh, never mind. I see what you're saying. But then why do we need
> cython.py if
> everything is provided by Cython/__init__.py?
>

At least for me, Python cannot import it by the name "cython" if only
Cython/... is available. And I think we want "import cython" (not "import
Cython") to work, as lowercase is more consistent with common Python
naming schemes in general and just "feels more right" for such core
language functionality.

(Also the module name would otherwise need to change in a lot of places --
when compiling under Cython, as opposed to running in Python, "cython" is
a magic, hard-wired module and "Cython" will not work, at least currently
(and my opinion is it never should))

Dag Sverre


From robertwb at math.washington.edu  Mon Oct  6 19:43:24 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 10:43:24 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <gcdhpi$jsr$1@ger.gmane.org>
	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>
Message-ID: <0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>

On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:

>> Robert Kern wrote:
>> Oh, never mind. I see what you're saying. But then why do we need
>> cython.py if
>> everything is provided by Cython/__init__.py?
>>
>
> At least for me, Python cannot import it by the name "cython" if only
> Cython/... is available. And I think we want "import cython" (not  
> "import
> Cython") to work, as lowercase is more consistent with common Python
> naming schemes in general and just "feels more right" for such core
> language functionality.
>
> (Also the module name would otherwise need to change in a lot of  
> places --
> when compiling under Cython, as opposed to running in Python,  
> "cython" is
> a magic, hard-wired module and "Cython" will not work, at least  
> currently
> (and my opinion is it never should))


My thoughts exactly. I think the "cython compiler codebase" and the  
"cython magic module" as completely separate concepts. The only  
reason Cython/__init__.py does anything is to be a workaround for  
case insensitive filesystems.

>

- Robert


From robertwb at math.washington.edu  Mon Oct  6 19:47:15 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 10:47:15 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
Message-ID: <1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>

On Oct 6, 2008, at 10:25 AM, Ondrej Certik wrote:

> On Mon, Oct 6, 2008 at 7:04 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Oct 5, 2008, at 3:46 PM, Ondrej Certik wrote:
>>
>>> On Sat, Oct 4, 2008 at 1:42 PM, Robert Bradshaw
>>> <robertwb at math.washington.edu> wrote:
>>>> I polished off some more of the pure python mode stuff tonight. I
>>>> just pushed it to cython-devel. Now you can do stuff like:
>>>>
>>>>> import cython  # you need a single file to be in your path to use
>>>>> it in pure Python mode, provided (and installed) with Cython
>>>>>
>>>>> @cython.locals(x=int, y=cython.p_int)
>>>>> def foo(x):
>>>>>     y = cython.cast(cython.p_int, x)
>>>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>>>     if cython.compiled:
>>>>>         print "the compiler was run
>>>>>     else:
>>>>>         print "just being interpreted"
>>>>>
>>>>> x = cython.declare(int)
>>>>> xx = cython.declare(cython.p_int)
>>>>> xx = cython.address(x)
>>>>>
>>>>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>>>>
>>>>> a = cython.declare(MyStruct)
>>>>> a.x = 5
>>>>>
>>>>> T = cython.typedef(MyStruct)
>>>>> b = cython.declare(cython.pointer(T))
>>>>> b[0].x = 4
>>>>
>>>>
>>>> On top of this, you can write a .pxd file to accompany your .py  
>>>> file
>>>> and it will coerce your .py file classes and def statements into  
>>>> cdef
>>>> and cpdef versions.
>>>>
>>>> It's still very new and probably highly experimental (though it  
>>>> only
>>>> modifies the cython.* nodes, so it shouldn't cause any  
>>>> regressions).
>>>> Let me know what you think.
>>>
>>> Thanks a lot for doing this!
>>
>> You're welcome.
>>
>>> I installed cython-devel and then I tried this little example:
>>>
>>>
>>> --------
>>> import Cython as cython
>>>
>>> @cython.locals(x=int, y=cython.p_int)
>>> def foo(x):
>>>     y = cython.cast(cython.p_int, x)
>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>     if cython.compiled:
>>>         print "the compiler was run"
>>>     else:
>>>         print "just being interpreted"
>>>
>>> foo(5)
>>> -----------
>>>
>>> When running in it in Python, it gives:
>>>
>>>   File "t.py", line 12, in <module>
>>>     foo(5)
>>>   File "t.py", line 5, in foo
>>>     y = cython.cast(cython.p_int, x)
>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in  
>>> cast
>>>     return type(arg)
>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>>> __init__
>>>     raise ValueError
>>> ValueError
>>>
>>>
>>> Am I using it incorrectly?
>>
>> No, it's just not letting you cast an int into an p_int. I suppose it
>> should? It just doesn't make sense to emulate this in Python.
>
> No, you are right, I was just trying to get it run.
>
>>
>>> Btw, when I compile it with Cython:
>>>
>>> $ python -c "import t; t.foo(5)"
>>> Traceback (most recent call last):
>>>   File "<string>", line 1, in <module>
>>>   File "t.py", line 12, in t (t.c:515)
>>>     foo(5)
>>>   File "t.py", line 5, in t.foo (t.c:265)
>>>     y = cython.cast(cython.p_int, x)
>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in  
>>> cast
>>>     return type(arg)
>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>>> __init__
>>>     raise ValueError
>>> ValueError
>>
>> You need to import cython, not import Cython as cython.
>>
>>> Looking at the code, I need to pass it Cython p_int integers,  
>>> right? I
>>> also tried:
>>>
>>> x = cython.declare(int)
>>> xx = cython.declare(cython.p_int)
>>> xx = cython.address(x)
>>> foo(xx)
>>>
>>> But this also raises the same error as above.
>>
>> This should work, Again, I think it's because you imported Cython
>> rather than cython.
>
>
> I tried that too:
>
> ondra at fuji:~/ext/cython-pure$ cat t.py
> import cython
>
> @cython.locals(x=int, y=cython.p_int)
> def foo(x):
>     y = cython.cast(cython.p_int, x)
>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>     if cython.compiled:
>         print "the compiler was run"
>     else:
>         print "just being interpreted"
>
> #foo(5)
>
> x = cython.declare(int)
> xx = cython.declare(cython.p_int)
> xx = cython.address(x)
> foo(xx)
> ondra at fuji:~/ext/cython-pure$ python t.py
> Traceback (most recent call last):
>   File "t.py", line 17, in <module>
>     foo(xx)
>   File "t.py", line 5, in foo
>     y = cython.cast(cython.p_int, x)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>     return type(arg)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in  
> __init__
>     raise ValueError
> ValueError
>
>
> With the same output. I copied the cython.py from cython-devel, is
> that correct? I am probably doing something stupid. The error is
> really simple and it fails, but currently I am not sure what it is
> supposed to do correctly.

To clarify, this is failing for you both interpreted and compiled?  
Try changing the line

> y = cython.cast(cython.p_int, x)


to

> y = cython.address(x)


so it can make sense of it.

- Robert


From ondrej at certik.cz  Mon Oct  6 20:10:56 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 6 Oct 2008 20:10:56 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
Message-ID: <85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>

On Mon, Oct 6, 2008 at 7:47 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 6, 2008, at 10:25 AM, Ondrej Certik wrote:
>
>> On Mon, Oct 6, 2008 at 7:04 PM, Robert Bradshaw
>> <robertwb at math.washington.edu> wrote:
>>> On Oct 5, 2008, at 3:46 PM, Ondrej Certik wrote:
>>>
>>>> On Sat, Oct 4, 2008 at 1:42 PM, Robert Bradshaw
>>>> <robertwb at math.washington.edu> wrote:
>>>>> I polished off some more of the pure python mode stuff tonight. I
>>>>> just pushed it to cython-devel. Now you can do stuff like:
>>>>>
>>>>>> import cython  # you need a single file to be in your path to use
>>>>>> it in pure Python mode, provided (and installed) with Cython
>>>>>>
>>>>>> @cython.locals(x=int, y=cython.p_int)
>>>>>> def foo(x):
>>>>>>     y = cython.cast(cython.p_int, x)
>>>>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>>>>     if cython.compiled:
>>>>>>         print "the compiler was run
>>>>>>     else:
>>>>>>         print "just being interpreted"
>>>>>>
>>>>>> x = cython.declare(int)
>>>>>> xx = cython.declare(cython.p_int)
>>>>>> xx = cython.address(x)
>>>>>>
>>>>>> MyStruct = cython.struct(x=int, y=int, data=cython.pp_double)
>>>>>>
>>>>>> a = cython.declare(MyStruct)
>>>>>> a.x = 5
>>>>>>
>>>>>> T = cython.typedef(MyStruct)
>>>>>> b = cython.declare(cython.pointer(T))
>>>>>> b[0].x = 4
>>>>>
>>>>>
>>>>> On top of this, you can write a .pxd file to accompany your .py
>>>>> file
>>>>> and it will coerce your .py file classes and def statements into
>>>>> cdef
>>>>> and cpdef versions.
>>>>>
>>>>> It's still very new and probably highly experimental (though it
>>>>> only
>>>>> modifies the cython.* nodes, so it shouldn't cause any
>>>>> regressions).
>>>>> Let me know what you think.
>>>>
>>>> Thanks a lot for doing this!
>>>
>>> You're welcome.
>>>
>>>> I installed cython-devel and then I tried this little example:
>>>>
>>>>
>>>> --------
>>>> import Cython as cython
>>>>
>>>> @cython.locals(x=int, y=cython.p_int)
>>>> def foo(x):
>>>>     y = cython.cast(cython.p_int, x)
>>>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>>>     if cython.compiled:
>>>>         print "the compiler was run"
>>>>     else:
>>>>         print "just being interpreted"
>>>>
>>>> foo(5)
>>>> -----------
>>>>
>>>> When running in it in Python, it gives:
>>>>
>>>>   File "t.py", line 12, in <module>
>>>>     foo(5)
>>>>   File "t.py", line 5, in foo
>>>>     y = cython.cast(cython.p_int, x)
>>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in
>>>> cast
>>>>     return type(arg)
>>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>>>> __init__
>>>>     raise ValueError
>>>> ValueError
>>>>
>>>>
>>>> Am I using it incorrectly?
>>>
>>> No, it's just not letting you cast an int into an p_int. I suppose it
>>> should? It just doesn't make sense to emulate this in Python.
>>
>> No, you are right, I was just trying to get it run.
>>
>>>
>>>> Btw, when I compile it with Cython:
>>>>
>>>> $ python -c "import t; t.foo(5)"
>>>> Traceback (most recent call last):
>>>>   File "<string>", line 1, in <module>
>>>>   File "t.py", line 12, in t (t.c:515)
>>>>     foo(5)
>>>>   File "t.py", line 5, in t.foo (t.c:265)
>>>>     y = cython.cast(cython.p_int, x)
>>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in
>>>> cast
>>>>     return type(arg)
>>>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>>>> __init__
>>>>     raise ValueError
>>>> ValueError
>>>
>>> You need to import cython, not import Cython as cython.
>>>
>>>> Looking at the code, I need to pass it Cython p_int integers,
>>>> right? I
>>>> also tried:
>>>>
>>>> x = cython.declare(int)
>>>> xx = cython.declare(cython.p_int)
>>>> xx = cython.address(x)
>>>> foo(xx)
>>>>
>>>> But this also raises the same error as above.
>>>
>>> This should work, Again, I think it's because you imported Cython
>>> rather than cython.
>>
>>
>> I tried that too:
>>
>> ondra at fuji:~/ext/cython-pure$ cat t.py
>> import cython
>>
>> @cython.locals(x=int, y=cython.p_int)
>> def foo(x):
>>     y = cython.cast(cython.p_int, x)
>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>     if cython.compiled:
>>         print "the compiler was run"
>>     else:
>>         print "just being interpreted"
>>
>> #foo(5)
>>
>> x = cython.declare(int)
>> xx = cython.declare(cython.p_int)
>> xx = cython.address(x)
>> foo(xx)
>> ondra at fuji:~/ext/cython-pure$ python t.py
>> Traceback (most recent call last):
>>   File "t.py", line 17, in <module>
>>     foo(xx)
>>   File "t.py", line 5, in foo
>>     y = cython.cast(cython.p_int, x)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>>     return type(arg)
>>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in
>> __init__
>>     raise ValueError
>> ValueError
>>
>>
>> With the same output. I copied the cython.py from cython-devel, is
>> that correct? I am probably doing something stupid. The error is
>> really simple and it fails, but currently I am not sure what it is
>> supposed to do correctly.
>
> To clarify, this is failing for you both interpreted and compiled?

In fact, cython refuses to compile this one:

$ cython t.py

Error converting Pyrex file to C:
------------------------------------------------------------
...
#foo(5)

x = cython.declare(int)
xx = cython.declare(cython.p_int)
xx = cython.address(x)
foo(xx)
     ^
------------------------------------------------------------

But when I tried the original foo(5), the cython version failed too.

I mean, this is some artificial example. I was just trying to get it
running to see how it works.

> Try changing the line
>
>> y = cython.cast(cython.p_int, x)
>
>
> to
>
>> y = cython.address(x)
>
>
> so it can make sense of it.

Still the same problem:

Traceback (most recent call last):
  File "t.py", line 17, in <module>
    foo(xx)
  File "t.py", line 5, in foo
    y = cython.address(x)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 22, in address
    return pointer(type(arg))([arg])
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 49, in __init__
    self._items = [cast(self._basetype, a) for a in value]
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
    return type(arg)
  File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in __init__
    raise ValueError
ValueError


But I tried this simpler example:

import cython

@cython.locals(x=int)
def foo(x):
    return x+5

print foo(5)


And this works as expected both interpreted and compiled!  Very nice.
How will you implement something like:

"
def f(x):
    return g(x)

cdef int g(int x):
    return x+5
"

E.g. having the option to write pure C functions (and classes). Maybe
something like:

@cython.locals(g=int, x=int)
def g(x):
    return x+5

?

Ondrej

From robert.kern at gmail.com  Mon Oct  6 20:51:48 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 13:51:48 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>
	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>
	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>
Message-ID: <gcdmo4$6th$1@ger.gmane.org>

Robert Bradshaw wrote:
> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
> 
>>> Robert Kern wrote:
>>> Oh, never mind. I see what you're saying. But then why do we need
>>> cython.py if
>>> everything is provided by Cython/__init__.py?
>>>
>> At least for me, Python cannot import it by the name "cython" if only
>> Cython/... is available. And I think we want "import cython" (not  
>> "import
>> Cython") to work, as lowercase is more consistent with common Python
>> naming schemes in general and just "feels more right" for such core
>> language functionality.
>>
>> (Also the module name would otherwise need to change in a lot of  
>> places --
>> when compiling under Cython, as opposed to running in Python,  
>> "cython" is
>> a magic, hard-wired module and "Cython" will not work, at least  
>> currently
>> (and my opinion is it never should))
> 
> My thoughts exactly. I think the "cython compiler codebase" and the  
> "cython magic module" as completely separate concepts. The only  
> reason Cython/__init__.py does anything is to be a workaround for  
> case insensitive filesystems.

In that case, I would recommend not using "cython" and pick another name that 
won't cause problems on such systems.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From dagss at student.matnat.uio.no  Mon Oct  6 21:20:35 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 6 Oct 2008 21:20:35 +0200 (CEST)
Subject: [Cython] About subclassing the module type
Message-ID: <33186.88.90.248.62.1223320835.squirrel@webmail.uio.no>

(I've mentioned this idea before but it now has more actuality.)

I just read ticket 95. When it comes to subclassing the module type (if
this is possible and easy and doesn't cost too much performance etc etc)
then this has another advantage: We could make early-bound symbols
read-only.

I.e.:

mymod.pyx:

cpdef f(): pass

Python session:
P> import mymod
P> mymod.f = sin
Exception raised:
CythonError("The mymod.f symbol was bound at compile-time and should thus
not be overwritten at runtime")

The main purpose is pedagogical of course, but as we work towards washing
away the differences between "C-level" code and Python-level code, stuff
like this could help getting the point across to users not used to
early-binding...

Same thing applies for "cdef classes" and so on.

Dag Sverre



From kevinar18 at hotmail.com  Mon Oct  6 21:35:24 2008
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Mon, 6 Oct 2008 15:35:24 -0400
Subject: [Cython] What to do with modules
In-Reply-To: <9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>
References: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
	<9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>
Message-ID: <BLU110-W41D0912DA0573CB391BC43AA390@phx.gbl>



>> So what do you do with modules that mix Cython and Python?
> 
> cdef classes live in both the C and Python namespace. cpdef functions  
> are accessible from both C and Python as well. In this case, you  
> probably need to just write
> 
> a = 1
> 
> and deal with it being a Python object so you can get at it from  
> Python (and it is still accessible from C). Everything accessible  
> from Python is also accessible from Cython.
> 
> We probably should support cimporting cdef'd global variables  
> (probably with the same trick as we do for functions) and it would be  
> nice to be able to declare variables as "public" or "readonly" as one  
> can do with class attributes (though the only way I could see how to  
> do it is the hack at http://mail.python.org/pipermail/python-list/ 
> 2007-January/422578.html or making a subtype of the module type)
So I essentially need to merge several hundred lines of module code into one file.  :(
_________________________________________________________________
Want to do more with Windows Live? Learn ?10 hidden secrets? from Jamie.
http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008

From robertwb at math.washington.edu  Mon Oct  6 22:45:52 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 13:45:52 -0700
Subject: [Cython] What to do with modules
In-Reply-To: <BLU110-W41D0912DA0573CB391BC43AA390@phx.gbl>
References: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
	<9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>
	<BLU110-W41D0912DA0573CB391BC43AA390@phx.gbl>
Message-ID: <CC92CC57-9015-434F-A202-4A4F74C5F989@math.washington.edu>

On Oct 6, 2008, at 12:35 PM, Kevin Ar18 wrote:

>
>
>>> So what do you do with modules that mix Cython and Python?
>>
>> cdef classes live in both the C and Python namespace. cpdef functions
>> are accessible from both C and Python as well. In this case, you
>> probably need to just write
>>
>> a = 1
>>
>> and deal with it being a Python object so you can get at it from
>> Python (and it is still accessible from C). Everything accessible
>> from Python is also accessible from Cython.
>>
>> We probably should support cimporting cdef'd global variables
>> (probably with the same trick as we do for functions) and it would be
>> nice to be able to declare variables as "public" or "readonly" as one
>> can do with class attributes (though the only way I could see how to
>> do it is the hack at http://mail.python.org/pipermail/python-list/
>> 2007-January/422578.html or making a subtype of the module type)
> So I essentially need to merge several hundred lines of module code  
> into one file.  :(

Probably not, that depends on what you're trying to do. Sage has  
hundreds of .pyx and .py files that all work with each other.

- Robert



From robertwb at math.washington.edu  Mon Oct  6 22:53:12 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 13:53:12 -0700
Subject: [Cython] About subclassing the module type
In-Reply-To: <33186.88.90.248.62.1223320835.squirrel@webmail.uio.no>
References: <33186.88.90.248.62.1223320835.squirrel@webmail.uio.no>
Message-ID: <4C076177-0BF0-4C4A-A6CE-BF475A4CFEAB@math.washington.edu>

On Oct 6, 2008, at 12:20 PM, Dag Sverre Seljebotn wrote:

> (I've mentioned this idea before but it now has more actuality.)
>
> I just read ticket 95. When it comes to subclassing the module type  
> (if
> this is possible and easy and doesn't cost too much performance etc  
> etc)

I looked into this a touch, and think it'd be fairly easy and very  
little (if any) performance overhead. The only question is if there  
are thing that access/manipulate the module's __dict__ directly,  
which is of course discouraged.

> then this has another advantage: We could make early-bound symbols
> read-only.

I've been thinking *exactly* the same thing.

- Robert


From robertwb at math.washington.edu  Mon Oct  6 22:58:53 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 13:58:53 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <gcdmo4$6th$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>
	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>
	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>
	<gcdmo4$6th$1@ger.gmane.org>
Message-ID: <9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>

On Oct 6, 2008, at 11:51 AM, Robert Kern wrote:

> Robert Bradshaw wrote:
>> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
>>
>>>> Robert Kern wrote:
>>>> Oh, never mind. I see what you're saying. But then why do we need
>>>> cython.py if
>>>> everything is provided by Cython/__init__.py?
>>>>
>>> At least for me, Python cannot import it by the name "cython" if  
>>> only
>>> Cython/... is available. And I think we want "import cython" (not
>>> "import
>>> Cython") to work, as lowercase is more consistent with common Python
>>> naming schemes in general and just "feels more right" for such core
>>> language functionality.
>>>
>>> (Also the module name would otherwise need to change in a lot of
>>> places --
>>> when compiling under Cython, as opposed to running in Python,
>>> "cython" is
>>> a magic, hard-wired module and "Cython" will not work, at least
>>> currently
>>> (and my opinion is it never should))
>>
>> My thoughts exactly. I think the "cython compiler codebase" and the
>> "cython magic module" as completely separate concepts. The only
>> reason Cython/__init__.py does anything is to be a workaround for
>> case insensitive filesystems.
>
> In that case, I would recommend not using "cython" and pick another  
> name that
> won't cause problems on such systems.

But it doesn't cause a problem on such systems. Personally, I can't  
think of any name that would be more appropriate than "cython."

- Robert

From robertwb at math.washington.edu  Mon Oct  6 23:11:56 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 14:11:56 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
Message-ID: <78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>


>>
>> To clarify, this is failing for you both interpreted and compiled?
>
> In fact, cython refuses to compile this one:
>
> $ cython t.py
>
> Error converting Pyrex file to C:
> ------------------------------------------------------------
> ...
> #foo(5)
>
> x = cython.declare(int)
> xx = cython.declare(cython.p_int)
> xx = cython.address(x)
> foo(xx)
>      ^
> ------------------------------------------------------------
>
> But when I tried the original foo(5), the cython version failed too.

This will fail, because foo takes an int.

> I mean, this is some artificial example. I was just trying to get it
> running to see how it works.

Yeah, my example was totally artificial...

>
>> Try changing the line
>>
>>> y = cython.cast(cython.p_int, x)
>>
>>
>> to
>>
>>> y = cython.address(x)
>>
>>
>> so it can make sense of it.
>
> Still the same problem:
>
> Traceback (most recent call last):
>   File "t.py", line 17, in <module>
>     foo(xx)
>   File "t.py", line 5, in foo
>     y = cython.address(x)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 22, in  
> address
>     return pointer(type(arg))([arg])
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 49, in  
> __init__
>     self._items = [cast(self._basetype, a) for a in value]
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 14, in cast
>     return type(arg)
>   File "/home/ondra/lib/lib/python/Cython/Shadow.py", line 53, in  
> __init__
>     raise ValueError
> ValueError

This works for me:

import cython

@cython.locals(x=int, y=cython.p_int)
def foo(x):
     y = cython.address(x)
     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
     print cython.compiled

foo(34)

> But I tried this simpler example:
>
> import cython
>
> @cython.locals(x=int)
> def foo(x):
>     return x+5
>
> print foo(5)
>
>
> And this works as expected both interpreted and compiled!  Very nice.
> How will you implement something like:
>
> "
> def f(x):
>     return g(x)
>
> cdef int g(int x):
>     return x+5
> "
>
> E.g. having the option to write pure C functions (and classes). Maybe
> something like:
>
> @cython.locals(g=int, x=int)
> def g(x):
>     return x+5

Exactly how to do this with decorators (and/or automatically) is less  
clear (syntax-wise, and there are more technical issues about  
signatures matching, etc.). What you can do is write an  
accompanying .pxd file which will, of course, be ignored by Python,  
but can declare cdef classes/methods/etc for Cython. Then your class  
and def statements will be appropriately coerced when you compile.

- Robert



From dagss at student.matnat.uio.no  Mon Oct  6 23:39:22 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 6 Oct 2008 23:39:22 +0200 (CEST)
Subject: [Cython] Pure python mode
In-Reply-To: <78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
Message-ID: <33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>

Robert wrote:
...
>> E.g. having the option to write pure C functions (and classes). Maybe
>> something like:
>>
>> @cython.locals(g=int, x=int)
>> def g(x):
>>     return x+5
>
> Exactly how to do this with decorators (and/or automatically) is less
> clear (syntax-wise, and there are more technical issues about
> signatures matching, etc.). What you can do is write an
> accompanying .pxd file which will, of course, be ignored by Python,
> but can declare cdef classes/methods/etc for Cython. Then your class
> and def statements will be appropriately coerced when you compile.

Syntax-wise this might be an argument against the name "locals" (and
before a release is done there is still time to change it).

a) With another name, it could be possible to make the return value of the
function the first positional argument. (I.e. a name that means
"@cython.cpdef", rather than only affecting the locals as such, but is
more descriptive).

This is assuming the "cdef/cpdef/def" can be decided automatically
(something like make everything cpdef and for pointer argument/return
types either introduce ctypes coercion or a stub raising a runtime error
when invoked from Python).

b) I think it makes sense to eventually add support for Py3 argument
decorators as well:

@cython.locals
def g(x: cython.int):
    return x + 5

"locals" is a bad fit for this. Of course, one would select another name
for this than the "locals" we have, however that is one more decorator to
learn without any obvious advantages over a concept like "@cython.cpdef"
which could conceptually fit in both situations. One could also drop the
function decorator, but this is rather unfriendly if you use other
decorators in the same program:

def g(x: cython.int):
    return x + 5

def f(x: "The input number"):
    return x + 5

def modify_docstrings_on_module_functions():
    # must now strip cython types...

So I think there should be a decorator on a function when you decide to
Cythonize it. Then you could have chained argument decorators:

@cython.cpdef # or whatever
def g(x: (cython.int, "The input number")):
    return x + 5

and have the cython.cpdef decorator process the argument decorator tuple
and replace it with the second item, which can then be processed by Python
code.

Dag Sverre


From robertwb at math.washington.edu  Mon Oct  6 23:57:38 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 14:57:38 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
Message-ID: <47D1C67E-E763-474C-A6EC-10E9391A4DA0@math.washington.edu>

On Oct 6, 2008, at 2:39 PM, Dag Sverre Seljebotn wrote:

> Robert wrote:
> ...
>>> E.g. having the option to write pure C functions (and classes).  
>>> Maybe
>>> something like:
>>>
>>> @cython.locals(g=int, x=int)
>>> def g(x):
>>>     return x+5
>>
>> Exactly how to do this with decorators (and/or automatically) is less
>> clear (syntax-wise, and there are more technical issues about
>> signatures matching, etc.). What you can do is write an
>> accompanying .pxd file which will, of course, be ignored by Python,
>> but can declare cdef classes/methods/etc for Cython. Then your class
>> and def statements will be appropriately coerced when you compile.
>
> Syntax-wise this might be an argument against the name "locals" (and
> before a release is done there is still time to change it).

The point of locals is to declare the types of local variables,  
orthogonal to the question of whether or not it is a cdef/cpdef/ect.  
function.

> a) With another name, it could be possible to make the return value  
> of the
> function the first positional argument. (I.e. a name that means
> "@cython.cpdef", rather than only affecting the locals as such, but is
> more descriptive).
>
> This is assuming the "cdef/cpdef/def" can be decided automatically
> (something like make everything cpdef and for pointer argument/return
> types either introduce ctypes coercion or a stub raising a runtime  
> error
> when invoked from Python).
>
> b) I think it makes sense to eventually add support for Py3 argument
> decorators as well:
>
> @cython.locals
> def g(x: cython.int):
>     return x + 5

Yes, I certainly want to support this form as well, but it's only for  
Py3+ (or even if it's in 2.6, that's not old enough).

> "locals" is a bad fit for this. Of course, one would select another  
> name
> for this than the "locals" we have, however that is one more  
> decorator to
> learn without any obvious advantages over a concept like  
> "@cython.cpdef"
> which could conceptually fit in both situations.

cython.cpdef would be a bad fit for a def function(that has typed  
local variables). There should be a decorator for calling convention  
(def/cdef/cpdef) and a decorator for locals. Both can declare what  
the argument types are.

> One could also drop the
> function decorator, but this is rather unfriendly if you use other
> decorators in the same program:
>
> def g(x: cython.int):
>     return x + 5
>
> def f(x: "The input number"):
>     return x + 5
>
> def modify_docstrings_on_module_functions():
>     # must now strip cython types...
>
> So I think there should be a decorator on a function when you  
> decide to
> Cythonize it.

Yep.

> Then you could have chained argument decorators:

Yes.

>
> @cython.cpdef # or whatever
> def g(x: (cython.int, "The input number")):
>     return x + 5
>
> and have the cython.cpdef decorator process the argument decorator  
> tuple
> and replace it with the second item, which can then be processed by  
> Python
> code.


??

One issue with calling conventions is that one wants to be able to  
share them across modules/use them in classes (and subclasses), etc.  
Until we have automatic .pxd creation, this will be hard to do with  
decorators alone.

- Robert


From ondrej at certik.cz  Tue Oct  7 00:03:51 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 7 Oct 2008 00:03:51 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
Message-ID: <85b5c3130810061503k4615e4d9oa449b1f655b9b79c@mail.gmail.com>

> This works for me:
>
> import cython
>
> @cython.locals(x=int, y=cython.p_int)
> def foo(x):
>     y = cython.address(x)
>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>     print cython.compiled
>
> foo(34)

Wow, this works too for me (I was trying that xx case above). This is
what I get with pure python:

36
False

and cythonized:

42
True

> Exactly how to do this with decorators (and/or automatically) is less
> clear (syntax-wise, and there are more technical issues about
> signatures matching, etc.). What you can do is write an
> accompanying .pxd file which will, of course, be ignored by Python,
> but can declare cdef classes/methods/etc for Cython. Then your class
> and def statements will be appropriately coerced when you compile.

This brings me the idea, why not to just take the current syntax and
just generate a pure Python one from it? That way one also needs to
maintain just one codebase, only it will by the Cython one, but it
could always be used in Pure python.

For some reason I prefer the pure python syntax approach though.

Ondrej

From dalcinl at gmail.com  Tue Oct  7 00:41:01 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Oct 2008 19:41:01 -0300
Subject: [Cython] PATH: backport pyrex support for 'ctypedef public api'
	declarations
Message-ID: <e7ba66e40810061541v75be0573xfde5c192eee0b540@mail.gmail.com>

This patch backport pyrex support for 'ctypedef public api'
declarations. You also have a testcase (should go to 'tests/compile',
or perhaps should merged in some other testcase).


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctypedefpubapi.diff
Type: text/x-patch
Size: 1075 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081006/d788cb73/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ctypedefpubapi.pyx
Type: application/octet-stream
Size: 171 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081006/d788cb73/attachment.obj 

From robert.kern at gmail.com  Tue Oct  7 00:54:15 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 17:54:15 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>
	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>
Message-ID: <gce4un$ou1$1@ger.gmane.org>

Robert Bradshaw wrote:
> On Oct 6, 2008, at 11:51 AM, Robert Kern wrote:
> 
>> Robert Bradshaw wrote:
>>> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
>>>
>>>>> Robert Kern wrote:
>>>>> Oh, never mind. I see what you're saying. But then why do we need
>>>>> cython.py if
>>>>> everything is provided by Cython/__init__.py?
>>>>>
>>>> At least for me, Python cannot import it by the name "cython" if  
>>>> only
>>>> Cython/... is available. And I think we want "import cython" (not
>>>> "import
>>>> Cython") to work, as lowercase is more consistent with common Python
>>>> naming schemes in general and just "feels more right" for such core
>>>> language functionality.
>>>>
>>>> (Also the module name would otherwise need to change in a lot of
>>>> places --
>>>> when compiling under Cython, as opposed to running in Python,
>>>> "cython" is
>>>> a magic, hard-wired module and "Cython" will not work, at least
>>>> currently
>>>> (and my opinion is it never should))
>>> My thoughts exactly. I think the "cython compiler codebase" and the
>>> "cython magic module" as completely separate concepts. The only
>>> reason Cython/__init__.py does anything is to be a workaround for
>>> case insensitive filesystems.
>> In that case, I would recommend not using "cython" and pick another  
>> name that
>> won't cause problems on such systems.
> 
> But it doesn't cause a problem on such systems. Personally, I can't  
> think of any name that would be more appropriate than "cython."

I guess it will probably not cause many problems. It's just a code smell that I 
would avoid. You're repeating code. And making cython.py do double-duty as both 
a module in site-packages and a script in /usr/local/bin adds to the code smell.

Oh, and don't forget to add cython.py to py_modules=, otherwise I don't think it 
will get installed correctly.

While I'm looking at the setup.py script, why are the modules in pyximport being 
listed separately in py_modules= instead of being listed as a package in packages=?

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From robertwb at math.washington.edu  Tue Oct  7 01:21:48 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 16:21:48 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <gce4un$ou1$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>
	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>
	<gce4un$ou1$1@ger.gmane.org>
Message-ID: <01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>

On Oct 6, 2008, at 3:54 PM, Robert Kern wrote:

> Robert Bradshaw wrote:
>> On Oct 6, 2008, at 11:51 AM, Robert Kern wrote:
>>
>>> Robert Bradshaw wrote:
>>>> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
>>>>
>>>>>> Robert Kern wrote:
>>>>>> Oh, never mind. I see what you're saying. But then why do we need
>>>>>> cython.py if
>>>>>> everything is provided by Cython/__init__.py?
>>>>>>
>>>>> At least for me, Python cannot import it by the name "cython" if
>>>>> only
>>>>> Cython/... is available. And I think we want "import cython" (not
>>>>> "import
>>>>> Cython") to work, as lowercase is more consistent with common  
>>>>> Python
>>>>> naming schemes in general and just "feels more right" for such  
>>>>> core
>>>>> language functionality.
>>>>>
>>>>> (Also the module name would otherwise need to change in a lot of
>>>>> places --
>>>>> when compiling under Cython, as opposed to running in Python,
>>>>> "cython" is
>>>>> a magic, hard-wired module and "Cython" will not work, at least
>>>>> currently
>>>>> (and my opinion is it never should))
>>>> My thoughts exactly. I think the "cython compiler codebase" and the
>>>> "cython magic module" as completely separate concepts. The only
>>>> reason Cython/__init__.py does anything is to be a workaround for
>>>> case insensitive filesystems.
>>> In that case, I would recommend not using "cython" and pick another
>>> name that
>>> won't cause problems on such systems.
>>
>> But it doesn't cause a problem on such systems. Personally, I can't
>> think of any name that would be more appropriate than "cython."
>
> I guess it will probably not cause many problems. It's just a code  
> smell that I
> would avoid. You're repeating code. And making cython.py do double- 
> duty as both
> a module in site-packages and a script in /usr/local/bin adds to  
> the code smell.
>
> Oh, and don't forget to add cython.py to py_modules=, otherwise I  
> don't think it
> will get installed correctly.

For those who want to ship sources without the cython dependancy, I  
they would just copy Shadow.py and name in cython.py in their own  
directory. Maybe there's a cleaner way to do this without renaming  
either "cython" or "Cython." Or should we put it there for testing  
purposes?

> While I'm looking at the setup.py script, why are the modules in  
> pyximport being
> listed separately in py_modules= instead of being listed as a  
> package in packages=?

I'm not sure, other than that I wasn't able to get the "obvious" way  
to work right away. I'm not distutils guru though, so if there's a  
better way I'd welcome a patch.

- Robert


From robert.kern at gmail.com  Tue Oct  7 01:33:11 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 18:33:11 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>	<gce4un$ou1$1@ger.gmane.org>
	<01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>
Message-ID: <gce77o$v55$1@ger.gmane.org>

Robert Bradshaw wrote:
> On Oct 6, 2008, at 3:54 PM, Robert Kern wrote:
> 
>> Robert Bradshaw wrote:
>>> On Oct 6, 2008, at 11:51 AM, Robert Kern wrote:
>>>
>>>> Robert Bradshaw wrote:
>>>>> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
>>>>>
>>>>>>> Robert Kern wrote:
>>>>>>> Oh, never mind. I see what you're saying. But then why do we need
>>>>>>> cython.py if
>>>>>>> everything is provided by Cython/__init__.py?
>>>>>>>
>>>>>> At least for me, Python cannot import it by the name "cython" if
>>>>>> only
>>>>>> Cython/... is available. And I think we want "import cython" (not
>>>>>> "import
>>>>>> Cython") to work, as lowercase is more consistent with common  
>>>>>> Python
>>>>>> naming schemes in general and just "feels more right" for such  
>>>>>> core
>>>>>> language functionality.
>>>>>>
>>>>>> (Also the module name would otherwise need to change in a lot of
>>>>>> places --
>>>>>> when compiling under Cython, as opposed to running in Python,
>>>>>> "cython" is
>>>>>> a magic, hard-wired module and "Cython" will not work, at least
>>>>>> currently
>>>>>> (and my opinion is it never should))
>>>>> My thoughts exactly. I think the "cython compiler codebase" and the
>>>>> "cython magic module" as completely separate concepts. The only
>>>>> reason Cython/__init__.py does anything is to be a workaround for
>>>>> case insensitive filesystems.
>>>> In that case, I would recommend not using "cython" and pick another
>>>> name that
>>>> won't cause problems on such systems.
>>> But it doesn't cause a problem on such systems. Personally, I can't
>>> think of any name that would be more appropriate than "cython."
>> I guess it will probably not cause many problems. It's just a code  
>> smell that I
>> would avoid. You're repeating code. And making cython.py do double- 
>> duty as both
>> a module in site-packages and a script in /usr/local/bin adds to  
>> the code smell.
>>
>> Oh, and don't forget to add cython.py to py_modules=, otherwise I  
>> don't think it
>> will get installed correctly.
> 
> For those who want to ship sources without the cython dependancy, I  
> they would just copy Shadow.py and name in cython.py in their own  
> directory.

Ah. I see the intended workflow, now.

> Maybe there's a cleaner way to do this without renaming  
> either "cython" or "Cython." Or should we put it there for testing  
> purposes?

Personally, I'd just rename the thing to "cythonize" and avoid the whole mess. I 
don't see much profit in building multiple workarounds just to keep a name.

>> While I'm looking at the setup.py script, why are the modules in  
>> pyximport being
>> listed separately in py_modules= instead of being listed as a  
>> package in packages=?
> 
> I'm not sure, other than that I wasn't able to get the "obvious" way  
> to work right away. I'm not distutils guru though, so if there's a  
> better way I'd welcome a patch.

The attached should do.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: cython-pyximport.diff
Url: http://codespeak.net/pipermail/cython-dev/attachments/20081006/6ae1d1e0/attachment-0001.diff 

From robertwb at math.washington.edu  Tue Oct  7 01:34:50 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 16:34:50 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <85b5c3130810061503k4615e4d9oa449b1f655b9b79c@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<85b5c3130810061503k4615e4d9oa449b1f655b9b79c@mail.gmail.com>
Message-ID: <6DD6B726-68AC-4C41-BABA-74E2E672A714@math.washington.edu>

On Oct 6, 2008, at 3:03 PM, Ondrej Certik wrote:

>> This works for me:
>>
>> import cython
>>
>> @cython.locals(x=int, y=cython.p_int)
>> def foo(x):
>>     y = cython.address(x)
>>     print x + cython.sizeof(y) + cython.sizeof(cython.p_int)
>>     print cython.compiled
>>
>> foo(34)
>
> Wow, this works too for me (I was trying that xx case above). This is
> what I get with pure python:
>
> 36
> False
>
> and cythonized:
>
> 42
> True

Yeah, the cython.sizeof stub always returns 1. The only place I could  
see sizeof being useful is a fake "malloc," etc. that return wrapped  
lists.

For most code the interpreted and compiled should do exactly the same  
thing.

>> Exactly how to do this with decorators (and/or automatically) is less
>> clear (syntax-wise, and there are more technical issues about
>> signatures matching, etc.). What you can do is write an
>> accompanying .pxd file which will, of course, be ignored by Python,
>> but can declare cdef classes/methods/etc for Cython. Then your class
>> and def statements will be appropriately coerced when you compile.
>
> This brings me the idea, why not to just take the current syntax and
> just generate a pure Python one from it? That way one also needs to
> maintain just one codebase, only it will by the Cython one, but it
> could always be used in Pure python.
>
> For some reason I prefer the pure python syntax approach though.

Same here. I'd edit the .py files then force the edit-compile-run  
cycle for that mode as well. Most code already exists (or starts out)  
as pure Python, then gets "cythonoized" which also makes it easier to  
go this direction. I also anticipate being able to point cython at  
any .py file to gain (at least some) improvements. All of these  
reasons make it easier for the .py file to be the "original" rather  
than generated.

Writing a .py writer would be a nontrivial amount of work as  
well...certainly doable but there are more productive things that I  
would rather spend my time doing :).

- Robert

From robertwb at math.washington.edu  Tue Oct  7 01:37:50 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 16:37:50 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <gce77o$v55$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>	<gce4un$ou1$1@ger.gmane.org>
	<01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>
	<gce77o$v55$1@ger.gmane.org>
Message-ID: <F0B60574-535E-4324-9D08-54CE9FB58FEF@math.washington.edu>

On Oct 6, 2008, at 4:33 PM, Robert Kern wrote:

> Robert Bradshaw wrote:
>> On Oct 6, 2008, at 3:54 PM, Robert Kern wrote:
>>>
>>> Oh, and don't forget to add cython.py to py_modules=, otherwise  
>>> I  don't think it
>>> will get installed correctly.
>> For those who want to ship sources without the cython dependancy,  
>> I  they would just copy Shadow.py and name in cython.py in their  
>> own  directory.
>
> Ah. I see the intended workflow, now.
>
>> Maybe there's a cleaner way to do this without renaming  either  
>> "cython" or "Cython." Or should we put it there for testing   
>> purposes?
>
> Personally, I'd just rename the thing to "cythonize" and avoid the  
> whole mess. I don't see much profit in building multiple  
> workarounds just to keep a name.

It's a question of making the (internal) code a bit uglier to make  
the user interface nicer.

>>> While I'm looking at the setup.py script, why are the modules in   
>>> pyximport being
>>> listed separately in py_modules= instead of being listed as a   
>>> package in packages=?
>> I'm not sure, other than that I wasn't able to get the "obvious"  
>> way  to work right away. I'm not distutils guru though, so if  
>> there's a  better way I'd welcome a patch.
>
> The attached should do.

Thanks. Have you tried it with pyximport on a fresh Python install?  
(I think this is what I did first and had issues, but I would be  
happy to be proven wrong.)

>
> -- 
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a  
> harmless enigma
>  that is made terrible by our own mad attempt to interpret it as  
> though it had
>  an underlying truth."
>   -- Umberto Eco
> diff -r f4b39b81687b setup.py
> --- a/setup.py  Wed Sep 03 18:24:45 2008 +0200
> +++ b/setup.py  Mon Oct 06 18:32:29 2008 -0500
> @@ -97,10 +97,8 @@
>
>      'Cython.Tests',
>      'Cython.Compiler.Tests',
> +    'pyximport',
>      ],
> -
> -  # pyximport
> -  py_modules = ["pyximport/pyximport", "pyximport/pyxbuild"],
>
>    **setup_args
>    )_______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From ondrej at certik.cz  Tue Oct  7 01:44:58 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 7 Oct 2008 01:44:58 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <6DD6B726-68AC-4C41-BABA-74E2E672A714@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<85b5c3130810061503k4615e4d9oa449b1f655b9b79c@mail.gmail.com>
	<6DD6B726-68AC-4C41-BABA-74E2E672A714@math.washington.edu>
Message-ID: <85b5c3130810061644k9cd60cav8fe00513db3f728e@mail.gmail.com>

>> For some reason I prefer the pure python syntax approach though.
>
> Same here. I'd edit the .py files then force the edit-compile-run
> cycle for that mode as well. Most code already exists (or starts out)
> as pure Python, then gets "cythonoized" which also makes it easier to
> go this direction. I also anticipate being able to point cython at
> any .py file to gain (at least some) improvements. All of these
> reasons make it easier for the .py file to be the "original" rather
> than generated.

Yes, exactly, well written.

>
> Writing a .py writer would be a nontrivial amount of work as
> well...certainly doable but there are more productive things that I
> would rather spend my time doing :).

It's definitely better to concentrate on getting the pure python
syntax working and we'll be fine.

Ondrej

From dalcinl at gmail.com  Tue Oct  7 01:53:24 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 6 Oct 2008 20:53:24 -0300
Subject: [Cython] Pure python mode
In-Reply-To: <gce77o$v55$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <gcdhpi$jsr$1@ger.gmane.org>
	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>
	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>
	<gcdmo4$6th$1@ger.gmane.org>
	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>
	<gce4un$ou1$1@ger.gmane.org>
	<01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>
	<gce77o$v55$1@ger.gmane.org>
Message-ID: <e7ba66e40810061653k3a7212bas9b8ba4262fac962e@mail.gmail.com>

Robert, your change requires a bit more of work. Inside pyximport/ you
have a Setup.py file plus some other stuff, like a README. Perhaps
Setup.py could be removed, and the pyximport/README contents could be
added as a docstring inside pyximport/__init__.py

I believe the distutils stuff (for the sake of Cython build/install,
not talking about Cython.Distutils) needs some cleanups and
enhacements. I'll try to make a good patch for this.

On Mon, Oct 6, 2008 at 8:33 PM, Robert Kern <robert.kern at gmail.com> wrote:
> Robert Bradshaw wrote:
>>
>> On Oct 6, 2008, at 3:54 PM, Robert Kern wrote:
>>
>>> Robert Bradshaw wrote:
>>>>
>>>> On Oct 6, 2008, at 11:51 AM, Robert Kern wrote:
>>>>
>>>>> Robert Bradshaw wrote:
>>>>>>
>>>>>> On Oct 6, 2008, at 10:38 AM, Dag Sverre Seljebotn wrote:
>>>>>>
>>>>>>>> Robert Kern wrote:
>>>>>>>> Oh, never mind. I see what you're saying. But then why do we need
>>>>>>>> cython.py if
>>>>>>>> everything is provided by Cython/__init__.py?
>>>>>>>>
>>>>>>> At least for me, Python cannot import it by the name "cython" if
>>>>>>> only
>>>>>>> Cython/... is available. And I think we want "import cython" (not
>>>>>>> "import
>>>>>>> Cython") to work, as lowercase is more consistent with common  Python
>>>>>>> naming schemes in general and just "feels more right" for such  core
>>>>>>> language functionality.
>>>>>>>
>>>>>>> (Also the module name would otherwise need to change in a lot of
>>>>>>> places --
>>>>>>> when compiling under Cython, as opposed to running in Python,
>>>>>>> "cython" is
>>>>>>> a magic, hard-wired module and "Cython" will not work, at least
>>>>>>> currently
>>>>>>> (and my opinion is it never should))
>>>>>>
>>>>>> My thoughts exactly. I think the "cython compiler codebase" and the
>>>>>> "cython magic module" as completely separate concepts. The only
>>>>>> reason Cython/__init__.py does anything is to be a workaround for
>>>>>> case insensitive filesystems.
>>>>>
>>>>> In that case, I would recommend not using "cython" and pick another
>>>>> name that
>>>>> won't cause problems on such systems.
>>>>
>>>> But it doesn't cause a problem on such systems. Personally, I can't
>>>> think of any name that would be more appropriate than "cython."
>>>
>>> I guess it will probably not cause many problems. It's just a code  smell
>>> that I
>>> would avoid. You're repeating code. And making cython.py do double- duty
>>> as both
>>> a module in site-packages and a script in /usr/local/bin adds to  the
>>> code smell.
>>>
>>> Oh, and don't forget to add cython.py to py_modules=, otherwise I  don't
>>> think it
>>> will get installed correctly.
>>
>> For those who want to ship sources without the cython dependancy, I  they
>> would just copy Shadow.py and name in cython.py in their own  directory.
>
> Ah. I see the intended workflow, now.
>
>> Maybe there's a cleaner way to do this without renaming  either "cython"
>> or "Cython." Or should we put it there for testing  purposes?
>
> Personally, I'd just rename the thing to "cythonize" and avoid the whole
> mess. I don't see much profit in building multiple workarounds just to keep
> a name.
>
>>> While I'm looking at the setup.py script, why are the modules in
>>>  pyximport being
>>> listed separately in py_modules= instead of being listed as a  package in
>>> packages=?
>>
>> I'm not sure, other than that I wasn't able to get the "obvious" way  to
>> work right away. I'm not distutils guru though, so if there's a  better way
>> I'd welcome a patch.
>
> The attached should do.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>  -- Umberto Eco
>
> diff -r f4b39b81687b setup.py
> --- a/setup.py  Wed Sep 03 18:24:45 2008 +0200
> +++ b/setup.py  Mon Oct 06 18:32:29 2008 -0500
> @@ -97,10 +97,8 @@
>
>     'Cython.Tests',
>     'Cython.Compiler.Tests',
> +    'pyximport',
>     ],
> -
> -  # pyximport
> -  py_modules = ["pyximport/pyximport", "pyximport/pyxbuild"],
>
>   **setup_args
>   )
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robert.kern at gmail.com  Tue Oct  7 02:22:40 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 19:22:40 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <F0B60574-535E-4324-9D08-54CE9FB58FEF@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdfev$8p3$1@ger.gmane.org>	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>	<gce4un$ou1$1@ger.gmane.org>	<01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>	<gce77o$v55$1@ger.gmane.org>
	<F0B60574-535E-4324-9D08-54CE9FB58FEF@math.washington.edu>
Message-ID: <gcea4g$640$1@ger.gmane.org>

Robert Bradshaw wrote:
> On Oct 6, 2008, at 4:33 PM, Robert Kern wrote:
> 
>> Robert Bradshaw wrote:
>>> On Oct 6, 2008, at 3:54 PM, Robert Kern wrote:
>>>> Oh, and don't forget to add cython.py to py_modules=, otherwise  
>>>> I  don't think it
>>>> will get installed correctly.
>>> For those who want to ship sources without the cython dependancy,  
>>> I  they would just copy Shadow.py and name in cython.py in their  
>>> own  directory.
>> Ah. I see the intended workflow, now.
>>
>>> Maybe there's a cleaner way to do this without renaming  either  
>>> "cython" or "Cython." Or should we put it there for testing   
>>> purposes?
>> Personally, I'd just rename the thing to "cythonize" and avoid the  
>> whole mess. I don't see much profit in building multiple  
>> workarounds just to keep a name.
> 
> It's a question of making the (internal) code a bit uglier to make  
> the user interface nicer.

Personally, I would say that it's more than a bit. Playing with corner cases in 
the import mechanism is like playing with fire. But that's just my personal 
opinion, and you don't share it, so I'll drop it.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From robert.kern at gmail.com  Tue Oct  7 02:33:32 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 06 Oct 2008 19:33:32 -0500
Subject: [Cython] Pure python mode
In-Reply-To: <e7ba66e40810061653k3a7212bas9b8ba4262fac962e@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<gcdh9g$hkh$1@ger.gmane.org>
	<gcdhpi$jsr$1@ger.gmane.org>	<32879.88.90.248.62.1223314690.squirrel@webmail.uio.no>	<0566191B-D889-4F65-BF5D-357C6F4D5DC5@math.washington.edu>	<gcdmo4$6th$1@ger.gmane.org>	<9F93735C-3FCE-4877-AE69-FD6B16D0BF57@math.washington.edu>	<gce4un$ou1$1@ger.gmane.org>	<01F39223-419D-4A32-886C-23C4CB81917B@math.washington.edu>	<gce77o$v55$1@ger.gmane.org>
	<e7ba66e40810061653k3a7212bas9b8ba4262fac962e@mail.gmail.com>
Message-ID: <gceaot$82f$1@ger.gmane.org>

Lisandro Dalcin wrote:
> Robert, your change requires a bit more of work. Inside pyximport/ you
> have a Setup.py file plus some other stuff, like a README. Perhaps
> Setup.py could be removed, and the pyximport/README contents could be
> added as a docstring inside pyximport/__init__.py

The README doesn't get installed. The Setup.py does get installed, but is 
harmless. I've removed it, though, in the attached patch.

Cython builds, installs (both as an egg and as a regular install), and pyximport 
works according to my ad hoc testing (pyximport.install(), then importing and 
running some of the modules in tests/run/) with the attached patch.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: cython-pyximport.diff
Url: http://codespeak.net/pipermail/cython-dev/attachments/20081006/7d24a78f/attachment.diff 

From greg.ewing at canterbury.ac.nz  Tue Oct  7 02:41:08 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Oct 2008 13:41:08 +1300
Subject: [Cython] Pure python mode
In-Reply-To: <gcdh9g$hkh$1@ger.gmane.org>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org>
Message-ID: <48EAB024.3080208@canterbury.ac.nz>

Robert Kern wrote:

> I'm wondering if the statement "import cython" 
> might actually import Cython/__init__.py on some case-insensitive file system.

On MacOSX (case-insensitive) it seems to be able
to distinguish between them.

-- 
Greg


From kevinar18 at hotmail.com  Tue Oct  7 03:30:46 2008
From: kevinar18 at hotmail.com (Kevin Ar18)
Date: Mon, 6 Oct 2008 21:30:46 -0400
Subject: [Cython] What to do with modules
In-Reply-To: <CC92CC57-9015-434F-A202-4A4F74C5F989@math.washington.edu>
References: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
	<9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>
	<BLU110-W41D0912DA0573CB391BC43AA390@phx.gbl> 
	<CC92CC57-9015-434F-A202-4A4F74C5F989@math.washington.edu>
Message-ID: <BLU110-W32523F1B7571FBC3A63DFEAA380@phx.gbl>


>> So I essentially need to merge several hundred lines of module code  
>> into one file.  :(
> 
> Probably not, that depends on what you're trying to do. Sage has  
> hundreds of .pyx and .py files that all work with each other.


A summary:
I have a project split among several pyx files/modules.
I want to convert as many vars, functions, and classes to Cython as I can (this will leave a mix of Cython and Python).

I need to be able to import my Cython and Python mixed modules and then access the Cython vars, functions, and classes directly (as well as the Python ones).

I don't know how to do this unless I...
1. Keep Python vars, functions, and classes in the modules that I can call, since I can't call Cython stuff (this is not a good option, since I want to convert as much to Cython as I can)
2. merge all modules into one file
_________________________________________________________________
Want to do more with Windows Live? Learn ?10 hidden secrets? from Jamie.
http://windowslive.com/connect/post/jamiethomson.spaces.live.com-Blog-cns!550F681DAD532637!5295.entry?ocid=TXT_TAGLM_WL_domore_092008

From robertwb at math.washington.edu  Tue Oct  7 03:39:47 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 18:39:47 -0700
Subject: [Cython] What to do with modules
In-Reply-To: <BLU110-W32523F1B7571FBC3A63DFEAA380@phx.gbl>
References: <BLU110-W23FDA85037E7F2DE4730F1AA3F0@phx.gbl>
	<9E4B94B9-8591-4069-98A0-402957B5BF3A@math.washington.edu>
	<BLU110-W41D0912DA0573CB391BC43AA390@phx.gbl>
	<CC92CC57-9015-434F-A202-4A4F74C5F989@math.washington.edu>
	<BLU110-W32523F1B7571FBC3A63DFEAA380@phx.gbl>
Message-ID: <3590980F-0157-47D2-971E-72426A6167B9@math.washington.edu>

On Oct 6, 2008, at 6:30 PM, Kevin Ar18 wrote:

>
>>> So I essentially need to merge several hundred lines of module code
>>> into one file.  :(
>>
>> Probably not, that depends on what you're trying to do. Sage has
>> hundreds of .pyx and .py files that all work with each other.
>
>
> A summary:
> I have a project split among several pyx files/modules.
> I want to convert as many vars, functions, and classes to Cython as  
> I can (this will leave a mix of Cython and Python).
>
> I need to be able to import my Cython and Python mixed modules and  
> then access the Cython vars, functions, and classes directly (as  
> well as the Python ones).
>
> I don't know how to do this unless I...
> 1. Keep Python vars, functions, and classes in the modules that I  
> can call, since I can't call Cython stuff (this is not a good  
> option, since I want to convert as much to Cython as I can)
> 2. merge all modules into one file

I would take a look at http://docs.cython.org/docs/ 
sharing_declarations.html . The only thing that you cannot do  
(currently) is cimport cython variables (like "cdef int i") from one  
module to another. We would like to support this, but for the moment  
it is a very small overhead, and the import would only need to happen  
once (e.g. the first time you load the module) unless your code  
depends heavily on setting module level variables from other modules,  
this shouldn't be an issue. In fact, if this is the case, you could  
restructure the code to have a cdef class that holds all the  
(previously global) variables.

- Robert


From dalcinl at gmail.com  Tue Oct  7 05:20:08 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Tue, 7 Oct 2008 00:20:08 -0300
Subject: [Cython] Pure python mode
In-Reply-To: <48EAB024.3080208@canterbury.ac.nz>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <48EAB024.3080208@canterbury.ac.nz>
Message-ID: <e7ba66e40810062020yf8ae4fdrbf9dbbc2a06b1ca8@mail.gmail.com>

On Mon, Oct 6, 2008 at 9:41 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Robert Kern wrote:
>
>> I'm wondering if the statement "import cython"
>> might actually import Cython/__init__.py on some case-insensitive file system.
>
> On MacOSX (case-insensitive) it seems to be able
> to distinguish between them.
>

Perhaps __import__ black magic? Python/import.c is not something easy
to parse, but I bet packages take precedence over modules (or the
other way)... Does this make sense?

BTW, Greg, could you write an abc.py file and other ABC.py (OSX is
also case-preserving, right?), add some (different) stuff in each, and
tell all us what do you get doing 'import abc' and 'import ABC' ? Some
time ago
Brian Granger warned me about using an MPI.pxd file and other mpi.pxd,
because this could be really problematic on OSX.

As R. Kern commented in previous mail, the import machinery is not a
something we should be playing games or assuming 'standard' behavior.
And what would happen if Cython is shipped in something like a
'Portable Python' bundle living on USB stick with FAT ? And perhaps in
the future Guido will change his mind and make Python a
case-insensitive language... (just joking, but I remember people
asking for this in the past on Python-Dev. You know, "Real Programmers
use Fortran" ;-) )



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Tue Oct  7 06:10:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 07 Oct 2008 17:10:22 +1300
Subject: [Cython] Pure python mode
In-Reply-To: <e7ba66e40810062020yf8ae4fdrbf9dbbc2a06b1ca8@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <48EAB024.3080208@canterbury.ac.nz>
	<e7ba66e40810062020yf8ae4fdrbf9dbbc2a06b1ca8@mail.gmail.com>
Message-ID: <48EAE12E.40208@canterbury.ac.nz>

Lisandro Dalcin wrote:

> BTW, Greg, could you write an abc.py file and other ABC.py

You can't have both of those in the same directory at the
same time on a case-insensitive file system.

However, Python certainly takes notice of the capitalization
when importing, as it fails to find the module if you don't
use the same capitalization as exists in the file system.

My guess is that it's reading the directory itself and
comparing case-sensitively, rather than relying on the
OS to compare file names.

-- 
Greg

From robertwb at math.washington.edu  Tue Oct  7 06:45:52 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 6 Oct 2008 21:45:52 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <e7ba66e40810062020yf8ae4fdrbf9dbbc2a06b1ca8@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<gcdfev$8p3$1@ger.gmane.org>
	<BF55CF4C-C25C-4864-86EE-E956D5BAA0F7@math.washington.edu>
	<gcdh9g$hkh$1@ger.gmane.org> <48EAB024.3080208@canterbury.ac.nz>
	<e7ba66e40810062020yf8ae4fdrbf9dbbc2a06b1ca8@mail.gmail.com>
Message-ID: <63342B1F-CEDE-42CE-9026-F081B5EBAA5F@math.washington.edu>

On Oct 6, 2008, at 8:20 PM, Lisandro Dalcin wrote:

> On Mon, Oct 6, 2008 at 9:41 PM, Greg Ewing  
> <greg.ewing at canterbury.ac.nz> wrote:
>> Robert Kern wrote:
>>
>>> I'm wondering if the statement "import cython"
>>> might actually import Cython/__init__.py on some case-insensitive  
>>> file system.
>>
>> On MacOSX (case-insensitive) it seems to be able
>> to distinguish between them.

MacOSX HSF+ is case preserving, case intensive.

> Perhaps __import__ black magic? Python/import.c is not something easy
> to parse, but I bet packages take precedence over modules (or the
> other way)... Does this make sense?

Yes, that is the case.

> BTW, Greg, could you write an abc.py file and other ABC.py (OSX is
> also case-preserving, right?), add some (different) stuff in each, and
> tell all us what do you get doing 'import abc' and 'import ABC' ? Some
> time ago
> Brian Granger warned me about using an MPI.pxd file and other mpi.pxd,
> because this could be really problematic on OSX.
>
> As R. Kern commented in previous mail, the import machinery is not a
> something we should be playing games or assuming 'standard' behavior.
> And what would happen if Cython is shipped in something like a
> 'Portable Python' bundle living on USB stick with FAT ? And perhaps in
> the future Guido will change his mind and make Python a
> case-insensitive language... (just joking, but I remember people
> asking for this in the past on Python-Dev. You know, "Real Programmers
> use Fortran" ;-) )

The current way works no matter which it chooses first--as long as it  
can find cython.py or Cython (and Python's going to be horribly  
broken if either of those fail) then it just works.

- Robert



From stefan_ml at behnel.de  Wed Oct  8 09:04:47 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 08 Oct 2008 09:04:47 +0200
Subject: [Cython] cython-devel seems broken
Message-ID: <48EC5B8F.6080502@behnel.de>

Hi,

I get this on an "hg pull" from cython-devel:

	searching for changes
	adding changesets
	adding manifests
	adding file changes
	abort: unknown parent 90e366e55871!
	transaction abort!
	rollback completed

Is this fixable somehow?

Stefan

From robertwb at math.washington.edu  Wed Oct  8 09:10:18 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 8 Oct 2008 00:10:18 -0700
Subject: [Cython] cython-devel seems broken
In-Reply-To: <48EC5B8F.6080502@behnel.de>
References: <48EC5B8F.6080502@behnel.de>
Message-ID: <D8D08143-8CA8-41B4-BDE9-F93844F95F51@math.washington.edu>

Never seen that before... can you do hg outgoing or hg incoming?

On Oct 8, 2008, at 12:04 AM, Stefan Behnel wrote:

> Hi,
>
> I get this on an "hg pull" from cython-devel:
>
> 	searching for changes
> 	adding changesets
> 	adding manifests
> 	adding file changes
> 	abort: unknown parent 90e366e55871!
> 	transaction abort!
> 	rollback completed
>
> Is this fixable somehow?
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dagss at student.matnat.uio.no  Wed Oct  8 09:39:00 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: 08 Oct 2008 09:39:00 +0200
Subject: [Cython] cython-devel seems broken
Message-ID: <3306303563.315674@smtp.netcom.no>

Does it work to clone it?

I did the last push (as of yesterday evening at least), there was a couple of merges but nothing out of the ordinary...

I was able to clone it fresh from the repo yesterday.

You could also try to pull -dagss, which should contain the same stuff now (for diagnostics purposes).

Dag Sverre Seljebotn
-----Original Message-----
From: Stefan Behnel <stefan_ml at behnel.de>
Date: Wednesday, Oct 8, 2008 9:04 am
Subject: [Cython] cython-devel seems broken
To: Cython-dev <cython-dev at codespeak.net>Reply-To: cython-dev at codespeak.net

Hi,
>
>I get this on an "hg pull" from cython-devel:
>
>	searching for changes
>	adding changesets
>	adding manifests
>	adding file changes
>	abort: unknown parent 90e366e55871!
>	transaction abort!
>	rollback completed
>
>Is this fixable somehow?
>
>Stefan
>_______________________________________________
>Cython-dev mailing list
>Cython-dev at codespeak.net
>http://codespeak.net/mailman/listinfo/cython-dev
>


From dalcinl at gmail.com  Wed Oct  8 15:17:21 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 8 Oct 2008 10:17:21 -0300
Subject: [Cython] cython-devel seems broken
In-Reply-To: <3306303563.315674@smtp.netcom.no>
References: <3306303563.315674@smtp.netcom.no>
Message-ID: <e7ba66e40810080617u3c9b638ft64982bf40c0ecd10@mail.gmail.com>

>From my side, 'hg pull' a fresh 'hg clone' worked just fine.

On Wed, Oct 8, 2008 at 4:39 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Does it work to clone it?
>
> I did the last push (as of yesterday evening at least), there was a couple of merges but nothing out of the ordinary...
>
> I was able to clone it fresh from the repo yesterday.
>
> You could also try to pull -dagss, which should contain the same stuff now (for diagnostics purposes).
>
> Dag Sverre Seljebotn
> -----Original Message-----
> From: Stefan Behnel <stefan_ml at behnel.de>
> Date: Wednesday, Oct 8, 2008 9:04 am
> Subject: [Cython] cython-devel seems broken
> To: Cython-dev <cython-dev at codespeak.net>Reply-To: cython-dev at codespeak.net
>
> Hi,
>>
>>I get this on an "hg pull" from cython-devel:
>>
>>       searching for changes
>>       adding changesets
>>       adding manifests
>>       adding file changes
>>       abort: unknown parent 90e366e55871!
>>       transaction abort!
>>       rollback completed
>>
>>Is this fixable somehow?
>>
>>Stefan
>>_______________________________________________
>>Cython-dev mailing list
>>Cython-dev at codespeak.net
>>http://codespeak.net/mailman/listinfo/cython-dev
>>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Wed Oct  8 19:48:19 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 8 Oct 2008 14:48:19 -0300
Subject: [Cython] WARN: isinstace optimization disabled!
Message-ID: <e7ba66e40810081048t726fa994rf7a8d618ccee6c76@mail.gmail.com>

Robert, I believe you forgot to remove an 'if 0 ...' in
Cython/Compiler/Optimize.py . 'isinstance' optimization is then
disabled. Please review!

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From aaron.devore at gmail.com  Wed Oct  8 21:15:59 2008
From: aaron.devore at gmail.com (Aaron DeVore)
Date: Wed, 8 Oct 2008 12:15:59 -0700
Subject: [Cython] list.index(instance) by identity or __eq__?
Message-ID: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>

I'm writing a class/type that will be inserted into a list. After it
is inserted the developer should be able to call
a_list.index(my_type_instance) and get the my_type_instance index by
*identity*, not by a call to my_type.__eq__. I have a my_type.__eq__
function but it doesn't give the correct behavior. Which behavior does
Cython use?

Cheers!
Aaron

From dagss at student.matnat.uio.no  Wed Oct  8 21:33:43 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 08 Oct 2008 21:33:43 +0200
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
Message-ID: <48ED0B17.5060400@student.matnat.uio.no>

Aaron DeVore wrote:
> I'm writing a class/type that will be inserted into a list. After it
> is inserted the developer should be able to call
> a_list.index(my_type_instance) and get the my_type_instance index by
> *identity*, not by a call to my_type.__eq__. I have a my_type.__eq__
> function but it doesn't give the correct behavior. Which behavior does
> Cython use?

Cython lists are Python lists, so this should behave in the same way as 
in Python. If something is not working it would help with a code small 
example.

I.e. to do what you describe you should either not implement __eq__ at 
all, or have it return "self is other".

More details would help here (i.e. a working example of what you are 
trying to do and how it works in Python).

If you are trying to do something which cannot be done in Python then 
the answer is no, I suppose. You could consider writing a wrapper like this:

class IdWrapper:
     def __init__(self, x): self.x = x
     def __eq__(self, other): return self.x is other.x

and only insert objects through their wrapper...

-- 
Dag Sverre

From stefan_ml at behnel.de  Thu Oct  9 07:58:14 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 09 Oct 2008 07:58:14 +0200
Subject: [Cython] cython-devel seems broken
In-Reply-To: <e7ba66e40810080617u3c9b638ft64982bf40c0ecd10@mail.gmail.com>
References: <3306303563.315674@smtp.netcom.no>
	<e7ba66e40810080617u3c9b638ft64982bf40c0ecd10@mail.gmail.com>
Message-ID: <48ED9D76.4070307@behnel.de>


Lisandro Dalcin wrote:
>>From my side, 'hg pull' a fresh 'hg clone' worked just fine.

Ok, works for me, too. It's just that I'll have to rescue the non-hg-ed stuff
lying around in my working copy that way, so I hope this won't happen too often.

Thanks,

Stefan

From animator333 at yahoo.com  Thu Oct  9 18:48:03 2008
From: animator333 at yahoo.com (Prashant Saxena)
Date: Thu, 9 Oct 2008 22:18:03 +0530 (IST)
Subject: [Cython] pickle error when package is used
Message-ID: <595839.6948.qm@web94908.mail.in2.yahoo.com>

hi,

I did a simple test of storing object's instance using pickle and shelve.
In the attached archive you can run "test.py", pickle and shelve both are working fine in read and write mode.
Now change line no. 12 in "test.py" to:

from core.Node import Node

inside core directiory we have the same "Node.pyd".

Run "test.py" again, and here we get an error saying:
cPickle.PicklingError: Can't pickle <class 'Node.Node'>: it's not the same object as Node.Node

In case if you use "Node.pyc", (delete "Node.pyd") everything works fine.

Am I missing something here?

Regards

Prashant


      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081009/3d5dfb6d/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cython_test.zip
Type: application/zip
Size: 19873 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081009/3d5dfb6d/attachment-0001.zip 

From robertwb at math.washington.edu  Thu Oct  9 19:46:17 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 10:46:17 -0700
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
Message-ID: <649543CF-09FF-4371-9AB8-BD71C12627FB@math.washington.edu>

On Oct 8, 2008, at 12:15 PM, Aaron DeVore wrote:

> I'm writing a class/type that will be inserted into a list. After it
> is inserted the developer should be able to call
> a_list.index(my_type_instance) and get the my_type_instance index by
> *identity*, not by a call to my_type.__eq__. I have a my_type.__eq__
> function but it doesn't give the correct behavior. Which behavior does
> Cython use?

I think the issue you're running into is that Python extension  
classes use rich comparison rather than __eq__ and friends. See  
http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/ 
Manual/special_methods.html .

- Robert



From dalcinl at gmail.com  Thu Oct  9 21:55:32 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 9 Oct 2008 16:55:32 -0300
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
Message-ID: <e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>

On Wed, Oct 8, 2008 at 4:15 PM, Aaron DeVore <aaron.devore at gmail.com> wrote:
> I have a my_type.__eq__
> function but it doesn't give the correct behavior. Which behavior does
> Cython use?

Could you show us the way you implemented your __eq__ method?



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Thu Oct  9 21:59:41 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 12:59:41 -0700
Subject: [Cython] WARN: isinstace optimization disabled!
In-Reply-To: <e7ba66e40810081048t726fa994rf7a8d618ccee6c76@mail.gmail.com>
References: <e7ba66e40810081048t726fa994rf7a8d618ccee6c76@mail.gmail.com>
Message-ID: <34895C88-F448-4557-BF07-AFDDF7AA07D5@math.washington.edu>

Yes, this is an oversight on my part. As we have more optimizations,  
we need to find a way to test if they're being used (maybe a regex  
that must be present in the generated C?)

Thanks for catching this, I'll re-enable.

On Oct 8, 2008, at 10:48 AM, Lisandro Dalcin wrote:

> Robert, I believe you forgot to remove an 'if 0 ...' in
> Cython/Compiler/Optimize.py . 'isinstance' optimization is then
> disabled. Please review!
>
> -- 
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Thu Oct  9 22:49:12 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 13:49:12 -0700
Subject: [Cython] Pure python mode
In-Reply-To: <33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
Message-ID: <E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>

I'd like to release this soon. I've changed declare so it can work as  
below (i.e. declare(var=type, ..)). For now there's no "cpdef/ 
cdefclass/earlybind/??" decorator, we'll hammer that one out later.  
Any other comments before we commit to it?

- Robert


On Oct 6, 2008, at 2:39 PM, Dag Sverre Seljebotn wrote:

> Robert wrote:
> ...
>>> E.g. having the option to write pure C functions (and classes).  
>>> Maybe
>>> something like:
>>>
>>> @cython.locals(g=int, x=int)
>>> def g(x):
>>>     return x+5
>>
>> Exactly how to do this with decorators (and/or automatically) is less
>> clear (syntax-wise, and there are more technical issues about
>> signatures matching, etc.). What you can do is write an
>> accompanying .pxd file which will, of course, be ignored by Python,
>> but can declare cdef classes/methods/etc for Cython. Then your class
>> and def statements will be appropriately coerced when you compile.
>
> Syntax-wise this might be an argument against the name "locals" (and
> before a release is done there is still time to change it).
>
> a) With another name, it could be possible to make the return value  
> of the
> function the first positional argument. (I.e. a name that means
> "@cython.cpdef", rather than only affecting the locals as such, but is
> more descriptive).
>
> This is assuming the "cdef/cpdef/def" can be decided automatically
> (something like make everything cpdef and for pointer argument/return
> types either introduce ctypes coercion or a stub raising a runtime  
> error
> when invoked from Python).
>
> b) I think it makes sense to eventually add support for Py3 argument
> decorators as well:
>
> @cython.locals
> def g(x: cython.int):
>     return x + 5
>
> "locals" is a bad fit for this. Of course, one would select another  
> name
> for this than the "locals" we have, however that is one more  
> decorator to
> learn without any obvious advantages over a concept like  
> "@cython.cpdef"
> which could conceptually fit in both situations. One could also  
> drop the
> function decorator, but this is rather unfriendly if you use other
> decorators in the same program:
>
> def g(x: cython.int):
>     return x + 5
>
> def f(x: "The input number"):
>     return x + 5
>
> def modify_docstrings_on_module_functions():
>     # must now strip cython types...
>
> So I think there should be a decorator on a function when you  
> decide to
> Cythonize it. Then you could have chained argument decorators:
>
> @cython.cpdef # or whatever
> def g(x: (cython.int, "The input number")):
>     return x + 5
>
> and have the cython.cpdef decorator process the argument decorator  
> tuple
> and replace it with the second item, which can then be processed by  
> Python
> code.
>
> Dag Sverre
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dalcinl at gmail.com  Thu Oct  9 23:35:33 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 9 Oct 2008 18:35:33 -0300
Subject: [Cython] cython-devel broken again
Message-ID: <e7ba66e40810091435n604b78b8m615149b106bf8c54@mail.gmail.com>

Trying to install Cython from cython-devel generates the following error:

$ python setup.py install --home=/u/dalcinl
Compiling module Cython.Plex.Scanners ...
ERROR: Internal compiler error: Result forced on non-temp node
Extension module compilation failed, using plain Python implementation
running install
<and more output ...>

Please review...

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Thu Oct  9 23:57:52 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 09 Oct 2008 23:57:52 +0200
Subject: [Cython] Pure python mode
In-Reply-To: <E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
Message-ID: <48EE7E60.7040502@student.matnat.uio.no>

Robert Bradshaw wrote:
> I'd like to release this soon. I've changed declare so it can work as  
> below (i.e. declare(var=type, ..)). For now there's no "cpdef/ 
> cdefclass/earlybind/??" decorator, we'll hammer that one out later.  
> Any other comments before we commit to it?

Hmm.. Well done! :-)

Comments about release schedule though:

For NumPy complex numbers are now in place which was the big lacking 
feature (of course, that makes native complex float support the next big 
lacking feature for numerical use). Structs/record arrays are incredibly 
near the surface (less than a day, easily), OTOH I'm very pressed on 
time at the moment, so I suppose they should not block a release.

There definitely needs to be betas this time around too, the result_code 
refactoring is more dangerous than anything we did in summer. (In fact 
the issue Lisandro just posted may be related to this...)

BTW, are we now anywhere near keeping a feature sync with Pyrex? I'm 
wondering if we could perhaps think again about the versioning scheme -- 
  the version number is getting very long to refer to, and the link to 
Pyrex seems less important than it used to. (Obviously it needs to not 
mess with packaging systems, but both 0.10 and 0.9.9 could work, rather 
than 0.9.8.2).

-- 
Dag Sverre

From robertwb at math.washington.edu  Fri Oct 10 00:04:33 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 15:04:33 -0700
Subject: [Cython] Next release
In-Reply-To: <48EE7E60.7040502@student.matnat.uio.no>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>	<85b5c3130810051546h1d953077n3becd7231525448f@mail.gmail.com>	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
	<48EE7E60.7040502@student.matnat.uio.no>
Message-ID: <12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>

On Oct 9, 2008, at 2:57 PM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> I'd like to release this soon. I've changed declare so it can work as
>> below (i.e. declare(var=type, ..)). For now there's no "cpdef/
>> cdefclass/earlybind/??" decorator, we'll hammer that one out later.
>> Any other comments before we commit to it?
>
> Hmm.. Well done! :-)
>
> Comments about release schedule though:
>
> For NumPy complex numbers are now in place which was the big lacking
> feature

Yeah, I saw some commits that direction but haven't yet tried it out.  
This should be very nice.

> (of course, that makes native complex float support the next big
> lacking feature for numerical use).

Yep.

> Structs/record arrays are incredibly
> near the surface (less than a day, easily), OTOH I'm very pressed on
> time at the moment, so I suppose they should not block a release.

If you get them done, cool, otherwise we'll continue without them for  
now.

> There definitely needs to be betas this time around too, the  
> result_code
> refactoring is more dangerous than anything we did in summer. (In fact
> the issue Lisandro just posted may be related to this...)

Yep. Any suggestions other than just reporting to the mailing list  
and compiling sage/lxml/etc?

> BTW, are we now anywhere near keeping a feature sync with Pyrex?

I believe the only feature we are lacking is multiple file  
compilation, but I could be wrong. There are also possibly some bug  
fixes in the latest that we haven't caught yet.

> I'm wondering if we could perhaps think again about the versioning  
> scheme --
>   the version number is getting very long to refer to, and the link to
> Pyrex seems less important than it used to. (Obviously it needs to not
> mess with packaging systems, but both 0.10 and 0.9.9 could work,  
> rather
> than 0.9.8.2).

Yes, this is worth revisiting.

- Robert

From dalcinl at gmail.com  Fri Oct 10 00:12:04 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 9 Oct 2008 19:12:04 -0300
Subject: [Cython] Next release
In-Reply-To: <12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
	<48EE7E60.7040502@student.matnat.uio.no>
	<12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>
Message-ID: <e7ba66e40810091512x7f881f58mfebf21afd51a26b5@mail.gmail.com>

On Thu, Oct 9, 2008 at 7:04 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 9, 2008, at 2:57 PM, Dag Sverre Seljebotn wrote:
>> BTW, are we now anywhere near keeping a feature sync with Pyrex?
>
> I believe the only feature we are lacking is multiple file
> compilation, but I could be wrong. There are also possibly some bug
> fixes in the latest that we haven't caught yet.
>

BTW, Could someone take a look at the patch I've sent a couple of days
ago about a Pyrex feature backport?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Fri Oct 10 00:14:23 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 15:14:23 -0700
Subject: [Cython] cython-devel broken again
In-Reply-To: <e7ba66e40810091435n604b78b8m615149b106bf8c54@mail.gmail.com>
References: <e7ba66e40810091435n604b78b8m615149b106bf8c54@mail.gmail.com>
Message-ID: <59B15F71-A200-4287-A3C4-F4FA104B6A66@math.washington.edu>

On Oct 9, 2008, at 2:35 PM, Lisandro Dalcin wrote:

> Trying to install Cython from cython-devel generates the following  
> error:
>
> $ python setup.py install --home=/u/dalcinl
> Compiling module Cython.Plex.Scanners ...
> ERROR: Internal compiler error: Result forced on non-temp node
> Extension module compilation failed, using plain Python implementation
> running install
> <and more output ...>
>
> Please review...

Oh, I see what went wrong. Fixed.

- Robert



From robertwb at math.washington.edu  Fri Oct 10 00:15:35 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 15:15:35 -0700
Subject: [Cython] Robert, please help!
In-Reply-To: <e7ba66e40810032129u8e75f6fy12063ada1c90042e@mail.gmail.com>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
	<48E6A4FF.1070900@canterbury.ac.nz>
	<e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>
	<48E6EB56.9090605@canterbury.ac.nz>
	<e7ba66e40810032129u8e75f6fy12063ada1c90042e@mail.gmail.com>
Message-ID: <BEC8EDCC-47AA-4172-A0DD-C21AD75097A0@math.washington.edu>


On Oct 3, 2008, at 9:29 PM, Lisandro Dalcin wrote:

> OK, at this link http://publications.gbdirect.co.uk/c_book/chapter8/ 
> typedef.html
>
> I've found the following example:
>
> /*
> * Using typedef, declare 'func' to have type
> * 'function taking two int arguments, returning int'
> */
> typedef int func(int, int);
> /* ERROR */
> func func_name{ /*....*/ }
> /* Correct. Returns pointer to a type 'func' */
> func *func_name(){ /*....*/ }
> /*
> * Correct if functions could return functions,
> * but C can't.
> */
> func func_name(){ /*....*/ }
>
>
> I'm fine with that. So I?ll try tomorrow if Robert?s modifications can
> handle this form. Thank's for the point, Greg!
>

Lisandro,

What was the final resolution on this?

- Robert



From robertwb at math.washington.edu  Fri Oct 10 00:17:40 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 15:17:40 -0700
Subject: [Cython] Next release
In-Reply-To: <e7ba66e40810091512x7f881f58mfebf21afd51a26b5@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<A267C914-808C-45CD-B347-3C00AE9E943C@math.washington.edu>
	<85b5c3130810061025n696ea13xe81af4231eb04ff0@mail.gmail.com>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
	<48EE7E60.7040502@student.matnat.uio.no>
	<12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>
	<e7ba66e40810091512x7f881f58mfebf21afd51a26b5@mail.gmail.com>
Message-ID: <2A0D365C-106F-4F25-AC9D-29C2B9054151@math.washington.edu>

On Oct 9, 2008, at 3:12 PM, Lisandro Dalcin wrote:

> On Thu, Oct 9, 2008 at 7:04 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> On Oct 9, 2008, at 2:57 PM, Dag Sverre Seljebotn wrote:
>>> BTW, are we now anywhere near keeping a feature sync with Pyrex?
>>
>> I believe the only feature we are lacking is multiple file
>> compilation, but I could be wrong. There are also possibly some bug
>> fixes in the latest that we haven't caught yet.
>>
>
> BTW, Could someone take a look at the patch I've sent a couple of days
> ago about a Pyrex feature backport?

The typedef one, yes. Also there's the set optimization one that I'd  
like to get in as well (for 2.4+).

Do you have an account on trac? Then we can post these issues for the  
next release to make sure they get done (or consciously postponed,  
though I don't think that's the case with either of these).

- Robert



From dalcinl at gmail.com  Fri Oct 10 00:36:32 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 9 Oct 2008 19:36:32 -0300
Subject: [Cython] Next release
In-Reply-To: <2A0D365C-106F-4F25-AC9D-29C2B9054151@math.washington.edu>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
	<48EE7E60.7040502@student.matnat.uio.no>
	<12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>
	<e7ba66e40810091512x7f881f58mfebf21afd51a26b5@mail.gmail.com>
	<2A0D365C-106F-4F25-AC9D-29C2B9054151@math.washington.edu>
Message-ID: <e7ba66e40810091536p116434eeq8052889cd972327@mail.gmail.com>

On Thu, Oct 9, 2008 at 7:17 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
>> BTW, Could someone take a look at the patch I've sent a couple of days
>> ago about a Pyrex feature backport?
>
> The typedef one, yes. Also there's the set optimization one that I'd
> like to get in as well (for 2.4+).
>
> Do you have an account on trac? Then we can post these issues for the
> next release to make sure they get done (or consciously postponed,
> though I don't think that's the case with either of these).

No, I do not have a Trac account. How do I get one?

BTW, Is there any chance I could get commit permissions for
cython-devel? Do not hessitate to say NO... but that would make far
easier for me to fix trivial bugs (like the previous one related to
isinstance optimization) and push small patches (like the Pyrex
backport, with IMHO do not need to much review).



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Fri Oct 10 00:54:50 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 9 Oct 2008 19:54:50 -0300
Subject: [Cython] Robert, please help!
In-Reply-To: <BEC8EDCC-47AA-4172-A0DD-C21AD75097A0@math.washington.edu>
References: <e7ba66e40810011842o2402b25dk7da32557a1326df3@mail.gmail.com>
	<D2DEA562-72AC-4D54-98A5-9597005A256D@math.washington.edu>
	<e7ba66e40810030715t51546976rb384f615ad70c34@mail.gmail.com>
	<48E6A4FF.1070900@canterbury.ac.nz>
	<e7ba66e40810032046s40722170o79a522df527323fa@mail.gmail.com>
	<48E6EB56.9090605@canterbury.ac.nz>
	<e7ba66e40810032129u8e75f6fy12063ada1c90042e@mail.gmail.com>
	<BEC8EDCC-47AA-4172-A0DD-C21AD75097A0@math.washington.edu>
Message-ID: <e7ba66e40810091554m5292fcd9odcd3dbd8c0e1b295@mail.gmail.com>

You have attached three different patches fixing this in different
ways. But I really believe the #3 is the best, safest one.

You also have a small pyx file in orther you can easily spot the
geneated code and the behavior of patch #3. WARNING: You have to
generate C sources and test with

$ cython --cleanup 9 modglbinit.pyx
$ <gcc modglbinit,c -> modglbinit.so>
$ python -c 'import modglb'

This example is vile hackery, and SHOULD NOT be included in the
testsuite right now (unless the testsuite generates C sources with
--cleanup options?). Perhaps if --cleanup could be passed as a
directive, then it could go in.


On Thu, Oct 9, 2008 at 7:15 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
>
> On Oct 3, 2008, at 9:29 PM, Lisandro Dalcin wrote:
>
>> OK, at this link http://publications.gbdirect.co.uk/c_book/chapter8/
>> typedef.html
>>
>> I've found the following example:
>>
>> /*
>> * Using typedef, declare 'func' to have type
>> * 'function taking two int arguments, returning int'
>> */
>> typedef int func(int, int);
>> /* ERROR */
>> func func_name{ /*....*/ }
>> /* Correct. Returns pointer to a type 'func' */
>> func *func_name(){ /*....*/ }
>> /*
>> * Correct if functions could return functions,
>> * but C can't.
>> */
>> func func_name(){ /*....*/ }
>>
>>
>> I'm fine with that. So I?ll try tomorrow if Robert?s modifications can
>> handle this form. Thank's for the point, Greg!
>>
>
> Lisandro,
>
> What was the final resolution on this?
>
> - Robert
>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cleanup_option1.diff
Type: text/x-patch
Size: 1266 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081009/69863e9f/attachment.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cleanup_option2.diff
Type: text/x-patch
Size: 743 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081009/69863e9f/attachment-0001.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cleanup_option3.diff
Type: text/x-patch
Size: 1006 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081009/69863e9f/attachment-0002.bin 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: modglbinit.pyx
Type: application/octet-stream
Size: 601 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081009/69863e9f/attachment.obj 

From robertwb at math.washington.edu  Fri Oct 10 08:55:13 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 9 Oct 2008 23:55:13 -0700
Subject: [Cython] Next release
In-Reply-To: <e7ba66e40810091536p116434eeq8052889cd972327@mail.gmail.com>
References: <20CF4396-1D84-404F-9BCC-2A2CC986F259@math.washington.edu>
	<1648675C-9FE0-4D45-B9A9-0D70214D6CCC@math.washington.edu>
	<85b5c3130810061110i674c87cfx1fe427d8cde1ddb3@mail.gmail.com>
	<78CB683C-F680-469E-BB4C-F1C771CA4E12@math.washington.edu>
	<33197.88.90.248.62.1223329162.squirrel@webmail.uio.no>
	<E96C653C-E3CA-449D-B12C-613DDD8D5587@math.washington.edu>
	<48EE7E60.7040502@student.matnat.uio.no>
	<12A83390-287F-4830-A795-C1A72A97793C@math.washington.edu>
	<e7ba66e40810091512x7f881f58mfebf21afd51a26b5@mail.gmail.com>
	<2A0D365C-106F-4F25-AC9D-29C2B9054151@math.washington.edu>
	<e7ba66e40810091536p116434eeq8052889cd972327@mail.gmail.com>
Message-ID: <AEC3D4AB-986C-4D72-BE43-1A1A8715D08B@math.washington.edu>

On Oct 9, 2008, at 3:36 PM, Lisandro Dalcin wrote:

> On Thu, Oct 9, 2008 at 7:17 PM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>>> BTW, Could someone take a look at the patch I've sent a couple of  
>>> days
>>> ago about a Pyrex feature backport?
>>
>> The typedef one, yes. Also there's the set optimization one that I'd
>> like to get in as well (for 2.4+).
>>
>> Do you have an account on trac? Then we can post these issues for the
>> next release to make sure they get done (or consciously postponed,
>> though I don't think that's the case with either of these).
>
> No, I do not have a Trac account. How do I get one?

Just send me an .htpasswd file off-list (just trying to prevent spam  
here).

> BTW, Is there any chance I could get commit permissions for
> cython-devel? Do not hessitate to say NO... but that would make far
> easier for me to fix trivial bugs (like the previous one related to
> isinstance optimization) and push small patches (like the Pyrex
> backport, with IMHO do not need to much review).


Yes, I'd be glad to do that. I'll set that up for you (using the same  
passwords as above).

- Robert


From aaron.devore at gmail.com  Fri Oct 10 09:28:01 2008
From: aaron.devore at gmail.com (Aaron DeVore)
Date: Fri, 10 Oct 2008 00:28:01 -0700
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
	<e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>
Message-ID: <2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>

Tragically, I can't show my implementation of __eq__. However, I will
say this: I absolutely cannot change __eq__ to work with identities.

For a relatively short list (usually 0-20 items, sometimes up to 1000)
would this work and still be blindingly fast? "----" indicates
indentation.:

int index
for index from 0 <= index < len(a_list):
----if a_list[index] is my_type_instance:
--------break

Or would I need to do this to get correct behavior:
int index
for index from 0 <= index < len(a_list):
----if id(a_list[index]) == id(my_type_instance):
--------break

2008/10/9 Lisandro Dalcin <dalcinl at gmail.com>:
> On Wed, Oct 8, 2008 at 4:15 PM, Aaron DeVore <aaron.devore at gmail.com> wrote:
>> I have a my_type.__eq__
>> function but it doesn't give the correct behavior. Which behavior does
>> Cython use?
>
> Could you show us the way you implemented your __eq__ method?
>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From robertwb at math.washington.edu  Fri Oct 10 10:02:56 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Oct 2008 01:02:56 -0700
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
	<e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>
	<2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>
Message-ID: <9264823B-EDAB-4BAD-8B6F-BC0F59ABC940@math.washington.edu>

On Oct 10, 2008, at 12:28 AM, Aaron DeVore wrote:

> Tragically, I can't show my implementation of __eq__. However, I will
> say this: I absolutely cannot change __eq__ to work with identities.
>
> For a relatively short list (usually 0-20 items, sometimes up to 1000)
> would this work and still be blindingly fast? "----" indicates
> indentation.:
>
> int index
> for index from 0 <= index < len(a_list):
> ----if a_list[index] is my_type_instance:
> --------break

This will compare on identity, if that's what you want then it should  
be quite fast.

> Or would I need to do this to get correct behavior:
> int index
> for index from 0 <= index < len(a_list):
> ----if id(a_list[index]) == id(my_type_instance):
> --------break

This is essentially how list.index() is implemented.

>
> 2008/10/9 Lisandro Dalcin <dalcinl at gmail.com>:
>> On Wed, Oct 8, 2008 at 4:15 PM, Aaron DeVore  
>> <aaron.devore at gmail.com> wrote:
>>> I have a my_type.__eq__
>>> function but it doesn't give the correct behavior. Which behavior  
>>> does
>>> Cython use?
>>
>> Could you show us the way you implemented your __eq__ method?
>>
>>
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From aaron.devore at gmail.com  Fri Oct 10 10:25:32 2008
From: aaron.devore at gmail.com (Aaron DeVore)
Date: Fri, 10 Oct 2008 01:25:32 -0700
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <9264823B-EDAB-4BAD-8B6F-BC0F59ABC940@math.washington.edu>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
	<e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>
	<2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>
	<9264823B-EDAB-4BAD-8B6F-BC0F59ABC940@math.washington.edu>
Message-ID: <2ead2fb0810100125of88811lb3a48c0c1b534861@mail.gmail.com>

I think I'll go for the first code example I gave. It looks like the
clearest version.

On Fri, Oct 10, 2008 at 1:02 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 10, 2008, at 12:28 AM, Aaron DeVore wrote:
>
>> Tragically, I can't show my implementation of __eq__. However, I will
>> say this: I absolutely cannot change __eq__ to work with identities.
>>
>> For a relatively short list (usually 0-20 items, sometimes up to 1000)
>> would this work and still be blindingly fast? "----" indicates
>> indentation.:
>>
>> int index
>> for index from 0 <= index < len(a_list):
>> ----if a_list[index] is my_type_instance:
>> --------break
>
> This will compare on identity, if that's what you want then it should
> be quite fast.
>
>> Or would I need to do this to get correct behavior:
>> int index
>> for index from 0 <= index < len(a_list):
>> ----if id(a_list[index]) == id(my_type_instance):
>> --------break
>
> This is essentially how list.index() is implemented.
>
>>
>> 2008/10/9 Lisandro Dalcin <dalcinl at gmail.com>:
>>> On Wed, Oct 8, 2008 at 4:15 PM, Aaron DeVore
>>> <aaron.devore at gmail.com> wrote:
>>>> I have a my_type.__eq__
>>>> function but it doesn't give the correct behavior. Which behavior
>>>> does
>>>> Cython use?
>>>
>>> Could you show us the way you implemented your __eq__ method?
>>>
>>>
>>>
>>> --
>>> Lisandro Dalc?n
>>> ---------------
>>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>>> Tel/Fax: +54-(0)342-451.1594
>>> _______________________________________________
>>> Cython-dev mailing list
>>> Cython-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/cython-dev
>>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From robertwb at math.washington.edu  Fri Oct 10 10:37:01 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Oct 2008 01:37:01 -0700
Subject: [Cython] array assignment
In-Reply-To: <20081002210306.GA15396@encolpuis>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
Message-ID: <EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>

On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:

> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>> is there any way to assign an cdef C array at assignment time?
>>
>> I want something like:
>>
>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>
>> as anything like this possible in Cython? Or do I need to do explicit
>> initialization.
>>
> And if it is explicit is there any way to not just do (can I do  
> some kind of
> block assignment?)
>
>     a[0] = 0.5
>         ...
>     a[3] = 0.1

This will now work

     cdef double *a = [0.5, 0.3, 0.1, 0.1]

Note that it is still allocated on the stack, but a pointer type is  
needed as arrays aren't lvalues.

- Robert


From dagss at student.matnat.uio.no  Fri Oct 10 10:40:59 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 10 Oct 2008 10:40:59 +0200
Subject: [Cython] array assignment
In-Reply-To: <EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
Message-ID: <48EF151B.8070005@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:
>
>   
>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>     
>>> is there any way to assign an cdef C array at assignment time?
>>>
>>> I want something like:
>>>
>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>>
>>> as anything like this possible in Cython? Or do I need to do explicit
>>> initialization.
>>>
>>>       
>> And if it is explicit is there any way to not just do (can I do  
>> some kind of
>> block assignment?)
>>
>>     a[0] = 0.5
>>         ...
>>     a[3] = 0.1
>>     
>
> This will now work
>
>      cdef double *a = [0.5, 0.3, 0.1, 0.1]
>
> Note that it is still allocated on the stack, but a pointer type is  
> needed as arrays aren't lvalues.
It looks a bit confusing...

I mean, for a traditional Cython user it looks like a temporary Python 
list is coerced, meaning that a will point to deallocated memory.

Ignoring that issue, what even happens if you reassign a to some other 
pointer? Or is that disallowed?

The initial syntax proposed avoids these problems. I understand that 
this was probably much easier, OTOH it might be better not to have a 
feature rather than having a confusing feature...

Dag Sverre

From robertwb at math.washington.edu  Fri Oct 10 10:49:02 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Oct 2008 01:49:02 -0700
Subject: [Cython] array assignment
In-Reply-To: <48EF151B.8070005@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
Message-ID: <FC19DD56-E469-49AB-AA59-50098CAA06AD@math.washington.edu>

On Oct 10, 2008, at 1:40 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:
>>
>>
>>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>>
>>>> is there any way to assign an cdef C array at assignment time?
>>>>
>>>> I want something like:
>>>>
>>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>>>
>>>> as anything like this possible in Cython? Or do I need to do  
>>>> explicit
>>>> initialization.
>>>>
>>>>
>>> And if it is explicit is there any way to not just do (can I do
>>> some kind of
>>> block assignment?)
>>>
>>>     a[0] = 0.5
>>>         ...
>>>     a[3] = 0.1
>>>
>>
>> This will now work
>>
>>      cdef double *a = [0.5, 0.3, 0.1, 0.1]
>>
>> Note that it is still allocated on the stack, but a pointer type is
>> needed as arrays aren't lvalues.
> It looks a bit confusing...
>
> I mean, for a traditional Cython user it looks like a temporary Python
> list is coerced, meaning that a will point to deallocated memory.

Yes, it may look like that, but that's not what happens. The  
traditional Cython user shouldn't have to think about allocation, so  
I don't think they'll be worried...

> Ignoring that issue, what even happens if you reassign a to some other
> pointer? Or is that disallowed?

This is allowed, and works as expected.

> The initial syntax proposed avoids these problems. I understand that
> this was probably much easier, OTOH it might be better not to have a
> feature rather than having a confusing feature...

Yes, I'd like to support the original syntax too, but I think this  
will be useful. I don't think it's too confusing.

- Robert



From stefan_ml at behnel.de  Fri Oct 10 11:33:05 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 10 Oct 2008 11:33:05 +0200 (CEST)
Subject: [Cython] array assignment
In-Reply-To: <EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
Message-ID: <62500.213.61.181.86.1223631185.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Oct 2, 2008, at 2:03 PM, Gabriel Gellner wrote:
>
>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>> is there any way to assign an cdef C array at assignment time?
>>>
>>> I want something like:
>>>
>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}

The RHS would be set literal syntax in Py3, which doesn't make sense here
at all.


>>> as anything like this possible in Cython? Or do I need to do explicit
>>> initialization.
>>>
>> And if it is explicit is there any way to not just do (can I do
>> some kind of block assignment?)
>>
>>     a[0] = 0.5
>>         ...
>>     a[3] = 0.1
>
> This will now work
>
>      cdef double *a = [0.5, 0.3, 0.1, 0.1]

Here, the RHS makes sense to me. Would

      cdef double *a = (0.5, 0.3, 0.1, 0.1)

also make sense to others? Or does this contradict 1WTDI? I wonder what
people think when they see both. I could imagine using a list RHS when I
consider changing the array in my function, while a tuple makes more sense
for immutable arrays. OTOH, I'm not sure users will bother with this
detail, they might just assume an assignment by value.

I assume this feature only works for initial assignments, right? Makes
perfect sense to me in the context of Cython.

I would personally prefer making the LHS

      cdef double a[] = [0.5, 0.3, 0.1, 0.1]

but I'm not close to using this anyway, so I'm not bothered with the
details. It would be nice if both worked, since pointers and array names
are pretty much equivalent in C anyway.

Just my two Euro-cents...

Stefan


From dagss at student.matnat.uio.no  Fri Oct 10 11:43:58 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Fri, 10 Oct 2008 11:43:58 +0200
Subject: [Cython] array assignment
In-Reply-To: <FC19DD56-E469-49AB-AA59-50098CAA06AD@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>
	<FC19DD56-E469-49AB-AA59-50098CAA06AD@math.washington.edu>
Message-ID: <48EF23DE.10205@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 10, 2008, at 1:40 AM, Dag Sverre Seljebotn wrote:
>
>   
>> The initial syntax proposed avoids these problems. I understand that
>> this was probably much easier, OTOH it might be better not to have a
>> feature rather than having a confusing feature...
>>     
>
> Yes, I'd like to support the original syntax too, but I think this  
> will be useful. I don't think it's too confusing.
>   
OK I'm convinced and +1. But then I'm leaning against ever supporting 
the original syntax, as that means less language complexity to worry 
about everywhere. But that is not the issue at stake here.

(BTW I was referring to the LHS only, I support Robert in using a list 
for the RHS.)

Dag Sverre

From ggellner at uoguelph.ca  Fri Oct 10 13:45:01 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Fri, 10 Oct 2008 07:45:01 -0400
Subject: [Cython] array assignment
In-Reply-To: <EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
Message-ID: <20081010114501.GA13472@encolpuis>

>> On Thu, Oct 02, 2008 at 05:00:40PM -0400, Gabriel Gellner wrote:
>>> is there any way to assign an cdef C array at assignment time?
>>>
>>> I want something like:
>>>
>>>     cdef double a[4] = {0.5, 0.3, 0.1, 0.1}
>>>
>>> as anything like this possible in Cython? Or do I need to do explicit
>>> initialization.
>>>
>> And if it is explicit is there any way to not just do (can I do some 
>> kind of
>> block assignment?)
>>
>>     a[0] = 0.5
>>         ...
>>     a[3] = 0.1
>
> This will now work
>
>     cdef double *a = [0.5, 0.3, 0.1, 0.1]
>
> Note that it is still allocated on the stack, but a pointer type is  
> needed as arrays aren't lvalues.
>
Thank you so much! You are awesome . . .

Gabriel

From ondrej at certik.cz  Fri Oct 10 15:15:58 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 10 Oct 2008 15:15:58 +0200
Subject: [Cython] docs.cython.org fix
Message-ID: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>

Hi,

please pull a link fix from here:

http://www.bitbucket.org/certik/cython-doc/

e.g.

hg pull http://www.bitbucket.org/certik/cython-doc/

Ondrej

From dalcinl at gmail.com  Fri Oct 10 15:24:57 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Fri, 10 Oct 2008 10:24:57 -0300
Subject: [Cython] list.index(instance) by identity or __eq__?
In-Reply-To: <2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>
References: <2ead2fb0810081215v34a482d8l5dd4a4d18875e481@mail.gmail.com>
	<e7ba66e40810091255n230b38eerc2bdb9b9bb5244f@mail.gmail.com>
	<2ead2fb0810100028u3b1ba737p3edd72845e8c1799@mail.gmail.com>
Message-ID: <e7ba66e40810100624w1b4d651cn8a9ebe8bf94e17ea@mail.gmail.com>

On Fri, Oct 10, 2008 at 4:28 AM, Aaron DeVore <aaron.devore at gmail.com> wrote:
> Tragically, I can't show my implementation of __eq__. However, I will
> say this: I absolutely cannot change __eq__ to work with identities.

Then have you tried to make __eq__ strict about types, and do something like:

cdef MyClass:
  def __richcmp__(self, other, int op):
     if not isinstance(self, MyClass): return NotImplemented
     if not isinstance(other, MyClass): return NotImplemented
     # and now impement the rest:
     if op == 2: # eq
         return True if condition else False
     elif op == 3: #ne
         # like above
     else:
          raise TypeError # only == and !=

>
> For a relatively short list (usually 0-20 items, sometimes up to 1000)
> would this work and still be blindingly fast? "----" indicates
> indentation.:
>
> int index
> for index from 0 <= index < len(a_list):
> ----if a_list[index] is my_type_instance:
> --------break
>
> Or would I need to do this to get correct behavior:
> int index
> for index from 0 <= index < len(a_list):
> ----if id(a_list[index]) == id(my_type_instance):
> --------break
>
> 2008/10/9 Lisandro Dalcin <dalcinl at gmail.com>:
>> On Wed, Oct 8, 2008 at 4:15 PM, Aaron DeVore <aaron.devore at gmail.com> wrote:
>>> I have a my_type.__eq__
>>> function but it doesn't give the correct behavior. Which behavior does
>>> Cython use?
>>
>> Could you show us the way you implemented your __eq__ method?
>>
>>
>>
>> --
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From ggellner at uoguelph.ca  Fri Oct 10 19:44:26 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Fri, 10 Oct 2008 13:44:26 -0400
Subject: [Cython] docs.cython.org fix
In-Reply-To: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>
References: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>
Message-ID: <20081010174426.GA14315@encolpuis>

On Fri, Oct 10, 2008 at 03:15:58PM +0200, Ondrej Certik wrote:
> Hi,
> 
> please pull a link fix from here:
> 
> http://www.bitbucket.org/certik/cython-doc/
> 
> e.g.
> 
> hg pull http://www.bitbucket.org/certik/cython-doc/
> 
Thanks I will do this tonight, finally having to learn to used mercurial . . .

Gabriel

From ondrej at certik.cz  Fri Oct 10 20:42:29 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 10 Oct 2008 20:42:29 +0200
Subject: [Cython] docs.cython.org fix
In-Reply-To: <20081010174426.GA14315@encolpuis>
References: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>
	<20081010174426.GA14315@encolpuis>
Message-ID: <85b5c3130810101142n783021d0nbcac9fa9f2fb598d@mail.gmail.com>

On Fri, Oct 10, 2008 at 7:44 PM, Gabriel Gellner <ggellner at uoguelph.ca> wrote:
> On Fri, Oct 10, 2008 at 03:15:58PM +0200, Ondrej Certik wrote:
>> Hi,
>>
>> please pull a link fix from here:
>>
>> http://www.bitbucket.org/certik/cython-doc/
>>
>> e.g.
>>
>> hg pull http://www.bitbucket.org/certik/cython-doc/
>>
> Thanks I will do this tonight, finally having to learn to used mercurial . . .

That's great, I am sure you'll not regret learning it. :)

Ondrej

From robertwb at math.washington.edu  Fri Oct 10 21:20:07 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Oct 2008 12:20:07 -0700
Subject: [Cython] docs.cython.org fix
In-Reply-To: <20081010174426.GA14315@encolpuis>
References: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>
	<20081010174426.GA14315@encolpuis>
Message-ID: <5A4FBCE3-53C3-4142-ABE3-66231C3676C1@math.washington.edu>

On Oct 10, 2008, at 10:44 AM, Gabriel Gellner wrote:

> On Fri, Oct 10, 2008 at 03:15:58PM +0200, Ondrej Certik wrote:
>> Hi,
>>
>> please pull a link fix from here:
>>
>> http://www.bitbucket.org/certik/cython-doc/
>>
>> e.g.
>>
>> hg pull http://www.bitbucket.org/certik/cython-doc/
>>
> Thanks I will do this tonight, finally having to learn to used  
> mercurial . . .

I actually already merged it and pushed to hg.cython.org/cython-docs

- Robert


From greg.ewing at canterbury.ac.nz  Sat Oct 11 08:20:13 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 11 Oct 2008 19:20:13 +1300
Subject: [Cython] array assignment
In-Reply-To: <48EF151B.8070005@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
Message-ID: <48F0459D.5020200@canterbury.ac.nz>

Dag Sverre Seljebotn wrote:
> Robert Bradshaw wrote:
> 
>>     cdef double *a = [0.5, 0.3, 0.1, 0.1]
>
>>Note that it is still allocated on the stack, but a pointer type is  
>>needed as arrays aren't lvalues.
> 
> It looks a bit confusing...

I think it looks confusing, too. Is 'a' a real pointer
here? I.e. can you assign a different pointer value to
it later?

If so, then it's not obvious where the memory for the
initial value is being allocated. A C person would
probably guess that it's either statically allocated
(by analogy with char *p = "literal") or heap allocated.
Allocating it on the stack is neither a C-like nor a
Python-like thing to do.

It would be better if it were written as

   cdef double a[] = [0.5, 0.3, 0.1, 0.1]

the same as a statically initialized array. The fact
that arrays aren't lvalues isn't relevant, because
this isn't an assignment, it's a specification of
an initial value.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Sat Oct 11 08:25:49 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 11 Oct 2008 19:25:49 +1300
Subject: [Cython] array assignment
In-Reply-To: <62500.213.61.181.86.1223631185.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<62500.213.61.181.86.1223631185.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <48F046ED.8020409@canterbury.ac.nz>

Stefan Behnel wrote:

> Here, the RHS makes sense to me. Would
> 
>       cdef double *a = (0.5, 0.3, 0.1, 0.1)
> 
> also make sense to others?

I think it would be reasonable to allow either list or
tuple notation to be used, but if there is only to be
one, it should probably be a list, on the grounds that
a C array is more of a homogeneous structure than a
heterogeneous one.

-- 
Greg

From robertwb at math.washington.edu  Sat Oct 11 08:54:16 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 10 Oct 2008 23:54:16 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F0459D.5020200@canterbury.ac.nz>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
	<48F0459D.5020200@canterbury.ac.nz>
Message-ID: <D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>


On Oct 10, 2008, at 11:20 PM, Greg Ewing wrote:

> Dag Sverre Seljebotn wrote:
>> Robert Bradshaw wrote:
>>
>>>     cdef double *a = [0.5, 0.3, 0.1, 0.1]
>>
>>> Note that it is still allocated on the stack, but a pointer type is
>>> needed as arrays aren't lvalues.
>>
>> It looks a bit confusing...
>
> I think it looks confusing, too. Is 'a' a real pointer
> here? I.e. can you assign a different pointer value to
> it later?

Yes.

> If so, then it's not obvious where the memory for the
> initial value is being allocated. A C person would
> probably guess that it's either statically allocated
> (by analogy with char *p = "literal") or heap allocated.
> Allocating it on the stack is neither a C-like nor a
> Python-like thing to do.

It's on the stack, just like literals. However, one can write

def foo(x, y):
     cdef double* a
     if x:
         a = [1,2,3,x]
     else:
         a = [1,2,3,y]

> It would be better if it were written as
>
>    cdef double a[] = [0.5, 0.3, 0.1, 0.1]
>
> the same as a statically initialized array. The fact
> that arrays aren't lvalues isn't relevant, because
> this isn't an assignment, it's a specification of
> an initial value.

Yes, I'd like to support this form as well, but it was a lot less  
"local" of a change and less flexible (see above). Also, I'd like to  
have a natural way to get memory-managed double*, etc. built into the  
language (using the allocate a string option). They would be very  
simple--contiguous one-dimensional arrays (anything more, just use  
NumPy), but would save people a lot of pain (especially those coming  
from the Python world who've never heard of malloc, memory leaks,  
segfaults ,etc.).

- Robert



From greg.ewing at canterbury.ac.nz  Sat Oct 11 09:24:20 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 11 Oct 2008 20:24:20 +1300
Subject: [Cython] array assignment
In-Reply-To: <D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
	<48F0459D.5020200@canterbury.ac.nz>
	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>
Message-ID: <48F054A4.8020300@canterbury.ac.nz>

Robert Bradshaw wrote:

> It's on the stack, just like literals.

What do you mean, "just like literals"? What literal
is ever allocated on the stack?

 > However, one can write
> 
> def foo(x, y):
>      cdef double* a
>      if x:
>          a = [1,2,3,x]
>      else:
>          a = [1,2,3,y]

Now I'm *really* confused. What the heck does that do?
Can you provide a C translation?

-- 
Greg

From robertwb at math.washington.edu  Sat Oct 11 14:06:13 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 05:06:13 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F054A4.8020300@canterbury.ac.nz>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
	<48F0459D.5020200@canterbury.ac.nz>
	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>
	<48F054A4.8020300@canterbury.ac.nz>
Message-ID: <086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>

On Oct 11, 2008, at 12:24 AM, Greg Ewing wrote:

> Robert Bradshaw wrote:
>
>> It's on the stack, just like literals.
>
> What do you mean, "just like literals"? What literal
> is ever allocated on the stack?

Array literals are allocated on the stack.

>
>> However, one can write
>>
>> def foo(x, y):
>>      cdef double* a
>>      if x:
>>          a = [1,2,3,x]
>>      else:
>>          a = [1,2,3,y]
>
> Now I'm *really* confused. What the heck does that do?
> Can you provide a C translation?


Sure. I'm ignoring the implicit conversions that happen for clarity...

double[4] tmp1;
double[4] tmp2;
if (x) {
     tmp1[0] = 1
     tmp1[1] = 2
     tmp1[2] = 3
     tmp1[3] = x
     a = tmp1
}
else {
     tmp2[0] = 1
     tmp2[1] = 2
     tmp2[2] = 3
     tmp2[3] = y
     a = tmp2
}

- Robert


From ben.aurel at gmail.com  Sat Oct 11 14:27:04 2008
From: ben.aurel at gmail.com (Ben Aurel)
Date: Sat, 11 Oct 2008 05:27:04 -0700
Subject: [Cython] Setup Errors: "error: Python.h: No such file or directory"
Message-ID: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>

hi
I try to build Cython on my ubuntu machine according to the README:
    $ python setup.py install

But I get a bunch of errors, starting with a message that says:
    ...
    /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
error: Python.h: No such file or directory
    ...
    (the first part of the installation log is appended at the end)


The Python.h apparently resides on the following directory on my system:

    /usr/local/python/include/python2.6/Python.h


How can I solve this setup errors? I think helping the installer to
find 'Python.h' is the first step. But how can I do that?

Any ideas?
thanks
ben

///
Setup log:
///
   1 Compiling module Cython.Plex.Scanners ...
   2 running install
   3 running build
   4 running build_py
   5 running build_ext
   6 building 'Cython.Plex.Scanners' extension
   7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c
/usr/local/src/Cython-0     .9.8.1.1/Cython/Plex/Scanners.c -o
build/temp.linux-i686-2.5/usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.o
   8 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
error: Python.h: No such file or directory
   9 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:5:26:
error: structmember.h: No such file or directory
  10 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:117:
error: expected specifier-qualifier-list before 'PyObject'
  11 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:135:
error: expected ')' before '*' token
  12 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:136:
error: expected '=', ',', ';', 'asm' or '__attribute__' before
'__pyx_PyInt_AsLon     gLong'
  13 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:137:
error: expected '=', ',', ';', 'asm' or '__attribute__' before
'__pyx_PyInt_AsUns     ignedLongLong'
  14 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:138:
error: expected ')' before '*' token
  15 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:143:
error: expected ')' before '*' token
  16 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:144:
error: expected ')' before '*' token
  17 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:145:
error: expected ')' before '*' token
  18 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:146:
error: expected ')' before '*' token
  19 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:147:
error: expected ')' before '*' token
  20 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:148:
error: expected ')' before '*' token
  21 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:149:
error: expected ')' before '*' token
  22 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:150:
error: expected ')' before '*' token
  23 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:151:
error: expected ')' before '*' token
  24 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:152:
error: expected ')' before '*' token
  25 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:153:
error: expected ')' before '*' token
  26 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:168:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  27 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:169:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  28 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:170:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  29 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:179:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  30 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:181:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  31 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:183:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  32 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:185:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token
  33 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:186:
error: expected ')' before '*' token
  34 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:188:
error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
token

From robertwb at math.washington.edu  Sat Oct 11 14:31:25 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 05:31:25 -0700
Subject: [Cython] Setup Errors: "error: Python.h: No such file or
	directory"
In-Reply-To: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
Message-ID: <9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>

You need the developer version of Python (i.e. the Python header  
files) to use or install Cython.

- Robert

On Oct 11, 2008, at 5:27 AM, Ben Aurel wrote:

> hi
> I try to build Cython on my ubuntu machine according to the README:
>     $ python setup.py install
>
> But I get a bunch of errors, starting with a message that says:
>     ...
>     /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
> error: Python.h: No such file or directory
>     ...
>     (the first part of the installation log is appended at the end)
>
>
> The Python.h apparently resides on the following directory on my  
> system:
>
>     /usr/local/python/include/python2.6/Python.h
>
>
> How can I solve this setup errors? I think helping the installer to
> find 'Python.h' is the first step. But how can I do that?
>
> Any ideas?
> thanks
> ben
>
> ///
> Setup log:
> ///
>    1 Compiling module Cython.Plex.Scanners ...
>    2 running install
>    3 running build
>    4 running build_py
>    5 running build_ext
>    6 building 'Cython.Plex.Scanners' extension
>    7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c
> /usr/local/src/Cython-0     .9.8.1.1/Cython/Plex/Scanners.c -o
> build/temp.linux-i686-2.5/usr/local/src/Cython-0.9.8.1.1/Cython/ 
> Plex/Scanners.o
>    8 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
> error: Python.h: No such file or directory
>    9 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:5:26:
> error: structmember.h: No such file or directory
>   10 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:117:
> error: expected specifier-qualifier-list before 'PyObject'
>   11 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:135:
> error: expected ')' before '*' token
>   12 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:136:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before
> '__pyx_PyInt_AsLon     gLong'
>   13 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:137:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before
> '__pyx_PyInt_AsUns     ignedLongLong'
>   14 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:138:
> error: expected ')' before '*' token
>   15 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:143:
> error: expected ')' before '*' token
>   16 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:144:
> error: expected ')' before '*' token
>   17 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:145:
> error: expected ')' before '*' token
>   18 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:146:
> error: expected ')' before '*' token
>   19 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:147:
> error: expected ')' before '*' token
>   20 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:148:
> error: expected ')' before '*' token
>   21 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:149:
> error: expected ')' before '*' token
>   22 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:150:
> error: expected ')' before '*' token
>   23 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:151:
> error: expected ')' before '*' token
>   24 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:152:
> error: expected ')' before '*' token
>   25 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:153:
> error: expected ')' before '*' token
>   26 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:168:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   27 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:169:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   28 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:170:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   29 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:179:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   30 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:181:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   31 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:183:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   32 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:185:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
>   33 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:186:
> error: expected ')' before '*' token
>   34 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:188:
> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
> token
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From dagss at student.matnat.uio.no  Sat Oct 11 15:55:47 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 11 Oct 2008 15:55:47 +0200
Subject: [Cython] array assignment
In-Reply-To: <086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>
	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
Message-ID: <48F0B063.9030401@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 11, 2008, at 12:24 AM, Greg Ewing wrote:
> 
>> Robert Bradshaw wrote:
>>
>>> It's on the stack, just like literals.
>> What do you mean, "just like literals"? What literal
>> is ever allocated on the stack?
> 
> Array literals are allocated on the stack.
> 
>>> However, one can write
>>>
>>> def foo(x, y):
>>>      cdef double* a
>>>      if x:
>>>          a = [1,2,3,x]
>>>      else:
>>>          a = [1,2,3,y]
>> Now I'm *really* confused. What the heck does that do?
>> Can you provide a C translation?
> 
> 
> Sure. I'm ignoring the implicit conversions that happen for clarity...
> 
> double[4] tmp1;
> double[4] tmp2;
> if (x) {
>      tmp1[0] = 1
>      tmp1[1] = 2
>      tmp1[2] = 3
>      tmp1[3] = x
>      a = tmp1
> }
> else {
>      tmp2[0] = 1
>      tmp2[1] = 2
>      tmp2[2] = 3
>      tmp2[3] = y
>      a = tmp2
> }
> 

I can see that this has many practical usecases etc., still I'm back to 
worrying. If the following concern is valid then I'm -1 on this for the 
coming release, it should at least be postponed.

The reason I worry is that one can write (rather contrived) code like this:

cdef int* a
cdef int* b
for i in range(2):
     a = [1,1,1]
     if i == 0:
         b = a
         b[0] = 2

print b[0] # will print 1...

...which will be rather non-obvious if you do not know *exactly* how the 
allocation etc. works.

The reason this is so bad is because it seems like you are pointing "a" 
to *something else* the second time around the loop, and that "b" should 
continue pointing to the "old" array. (But there is only one array. 
Which is especially counter-intuitive given how this code would work in 
Python. Etc.)

To boil it down a bit, the problem is that

a = [1,1,1]
b = a
a = [1,1,1]

has entirely different semantics from

for i in range(2):
     a = [1,1,1]
     if i == 0: b = a

even though the first example looks exactly like just rolling out the 
latter. At least it is very un-Pythonic.

(I did not actually try the code above, but if it doesn't work then I'd 
counter that the mechanism is so magical that I cannot figure it out :-) )

What do I propose instead?
  a) That the reference-counted heap-allocated array stuff in the wiki 
is implemented first (with a list constructor). So the thing gets the 
semantics of a reference counted array, constructed again each time it 
is assigned (very similar to a Python list).

  b) That simple cases (where the pointer is never reassigned or copied, 
only written/read to/from by indexing) are then optimized to use stack 
allocation instead.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Sat Oct 11 16:12:20 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 11 Oct 2008 16:12:20 +0200
Subject: [Cython] array assignment
In-Reply-To: <48F0B063.9030401@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
	<48F0B063.9030401@student.matnat.uio.no>
Message-ID: <48F0B444.1050007@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:

> What do I propose instead?
>   a) That the reference-counted heap-allocated array stuff in the wiki 
> is implemented first (with a list constructor). So the thing gets the 
> semantics of a reference counted array, constructed again each time it 
> is assigned (very similar to a Python list).
> 
>   b) That simple cases (where the pointer is never reassigned or copied, 
> only written/read to/from by indexing) are then optimized to use stack 
> allocation instead.
> 

First and foremost the point now is to .

But, builing on this proposal, I propose that it is called

cdef list[int] a = [1, 2, 3]

and is not a subtype of list, but close enough:

a) If it is not optimized (by b) above) it is a full Python object and 
emulates most behaviour (but has much more efficient indexing). (If only 
efficient indexing is used then it is a C stack-allocated array.)

b) "a = x", where x is a Python list, automatically converts

c) "cdef list y = someobj" needs to have added auto-conversion from 
list[type]

This allows keeping the pointer types "clean" and without too much 
magic, and also fits well with the buffer syntax etc. Users wouldn't 
have to learn two different worlds for operating with C arrays and NumPy 
arrays.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Sat Oct 11 16:17:35 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 11 Oct 2008 16:17:35 +0200
Subject: [Cython] array assignment
In-Reply-To: <48F0B444.1050007@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>
	<48F0B444.1050007@student.matnat.uio.no>
Message-ID: <48F0B57F.7030209@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Dag Sverre Seljebotn wrote:
> 
>> What do I propose instead?
>>   a) That the reference-counted heap-allocated array stuff in the wiki 
>> is implemented first (with a list constructor). So the thing gets the 
>> semantics of a reference counted array, constructed again each time it 
>> is assigned (very similar to a Python list).
>>
>>   b) That simple cases (where the pointer is never reassigned or copied, 
>> only written/read to/from by indexing) are then optimized to use stack 
>> allocation instead.
>>
> 
> First and foremost the point now is to .

Sorry! I meant: The point now was to argue for postponing the decision 
for later and disable it in the upcoming release.

I'd settle for disabling any "literal assignment" to the pointer outside 
of the declaration though, then I'd be back to "not worrying". (One 
could also disable it only inside loops, but that is too ugly and 
inconsistent.)

-- 
Dag Sverre

From robertwb at math.washington.edu  Sat Oct 11 16:15:56 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 07:15:56 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F0B444.1050007@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
	<48F0B063.9030401@student.matnat.uio.no>
	<48F0B444.1050007@student.matnat.uio.no>
Message-ID: <4729C46F-D463-44B8-B423-64364998269E@math.washington.edu>

On Oct 11, 2008, at 7:12 AM, Dag Sverre Seljebotn wrote:

> Dag Sverre Seljebotn wrote:
>
>> What do I propose instead?
>>   a) That the reference-counted heap-allocated array stuff in the  
>> wiki
>> is implemented first (with a list constructor). So the thing gets the
>> semantics of a reference counted array, constructed again each  
>> time it
>> is assigned (very similar to a Python list).
>>
>>   b) That simple cases (where the pointer is never reassigned or  
>> copied,
>> only written/read to/from by indexing) are then optimized to use  
>> stack
>> allocation instead.
>>
>
> First and foremost the point now is to .
>
> But, builing on this proposal, I propose that it is called
>
> cdef list[int] a = [1, 2, 3]
>
> and is not a subtype of list, but close enough:
>
> a) If it is not optimized (by b) above) it is a full Python object and
> emulates most behaviour (but has much more efficient indexing). (If  
> only
> efficient indexing is used then it is a C stack-allocated array.)
>
> b) "a = x", where x is a Python list, automatically converts
>
> c) "cdef list y = someobj" needs to have added auto-conversion from
> list[type]
>
> This allows keeping the pointer types "clean" and without too much
> magic, and also fits well with the buffer syntax etc. Users wouldn't
> have to learn two different worlds for operating with C arrays and  
> NumPy
> arrays.

I like this proposal, though I'd propose a new keyword (e.g. array)  
because list seems a bit odd (though it could be good, just not sure  
yet.)

cdef array[int] a = [1, 2, 3]

The question is not whether or not this is a better, more Pythonic  
way to do things, but whether or not anyone is going to hage time to  
implement this anytime soon (let alone by the next release). Yes, the  
arrays as they are now have a few warts (like using them in loops)  
but I think it's too useful to disable altogether.

- Robert


From robertwb at math.washington.edu  Sat Oct 11 16:17:35 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 07:17:35 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F0B57F.7030209@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>
	<48F0B444.1050007@student.matnat.uio.no>
	<48F0B57F.7030209@student.matnat.uio.no>
Message-ID: <7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>

On Oct 11, 2008, at 7:17 AM, Dag Sverre Seljebotn wrote:

> Dag Sverre Seljebotn wrote:
>> Dag Sverre Seljebotn wrote:
>>
>>> What do I propose instead?
>>>   a) That the reference-counted heap-allocated array stuff in the  
>>> wiki
>>> is implemented first (with a list constructor). So the thing gets  
>>> the
>>> semantics of a reference counted array, constructed again each  
>>> time it
>>> is assigned (very similar to a Python list).
>>>
>>>   b) That simple cases (where the pointer is never reassigned or  
>>> copied,
>>> only written/read to/from by indexing) are then optimized to use  
>>> stack
>>> allocation instead.
>>>
>>
>> First and foremost the point now is to .
>
> Sorry! I meant: The point now was to argue for postponing the decision
> for later and disable it in the upcoming release.
>
> I'd settle for disabling any "literal assignment" to the pointer  
> outside
> of the declaration though, then I'd be back to "not worrying". (One
> could also disable it only inside loops, but that is too ugly and
> inconsistent.)

I'm OK with only allowing it outside of declarations (and even  
better, only to an array type), but don't know if I'll have time to  
do that anytime soon.

- Robert


From dagss at student.matnat.uio.no  Sat Oct 11 16:31:50 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 11 Oct 2008 16:31:50 +0200
Subject: [Cython] array assignment
In-Reply-To: <7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>	<48F0B444.1050007@student.matnat.uio.no>	<48F0B57F.7030209@student.matnat.uio.no>
	<7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>
Message-ID: <48F0B8D6.7010502@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 11, 2008, at 7:17 AM, Dag Sverre Seljebotn wrote:
> 
>> Dag Sverre Seljebotn wrote:
>>> Dag Sverre Seljebotn wrote:
>>>
>>>> What do I propose instead?
>>>>   a) That the reference-counted heap-allocated array stuff in the  
>>>> wiki
>>>> is implemented first (with a list constructor). So the thing gets  
>>>> the
>>>> semantics of a reference counted array, constructed again each  
>>>> time it
>>>> is assigned (very similar to a Python list).
>>>>
>>>>   b) That simple cases (where the pointer is never reassigned or  
>>>> copied,
>>>> only written/read to/from by indexing) are then optimized to use  
>>>> stack
>>>> allocation instead.
>>>>
>>> First and foremost the point now is to .
>> Sorry! I meant: The point now was to argue for postponing the decision
>> for later and disable it in the upcoming release.
>>
>> I'd settle for disabling any "literal assignment" to the pointer  
>> outside
>> of the declaration though, then I'd be back to "not worrying". (One
>> could also disable it only inside loops, but that is too ugly and
>> inconsistent.)
> 
> I'm OK with only allowing it outside of declarations (and even  
> better, only to an array type), but don't know if I'll have time to  
> do that anytime soon.

I don't understand? Do you mean the opposite, i.e only allowing it 
inside of declarations?

Anyway, the (final and realistic) proposal was:

cdef int* a = [1, 2, 3] # ok!

while

cdef int* a
a = [1, 2, 3] # not ok!

At least for this release (adding features is easier than removing).

This absolutely shouldn't be more than half an hour, it is just about 
throwing up a compiler error in the right transform. If you're +1 to 
this but don't have the time I could have a look during next week. Have 
you decided on a timeframe for the release?

-- 
Dag Sverre

From robertwb at math.washington.edu  Sat Oct 11 16:32:01 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 07:32:01 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F0B8D6.7010502@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>	<48F0B444.1050007@student.matnat.uio.no>	<48F0B57F.7030209@student.matnat.uio.no>
	<7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>
	<48F0B8D6.7010502@student.matnat.uio.no>
Message-ID: <5EE66792-5EE4-4963-90DB-58143DC01482@math.washington.edu>

On Oct 11, 2008, at 7:31 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Oct 11, 2008, at 7:17 AM, Dag Sverre Seljebotn wrote:
>>
>>> Dag Sverre Seljebotn wrote:
>>>> Dag Sverre Seljebotn wrote:
>>>>
>>>>> What do I propose instead?
>>>>>   a) That the reference-counted heap-allocated array stuff in the
>>>>> wiki
>>>>> is implemented first (with a list constructor). So the thing gets
>>>>> the
>>>>> semantics of a reference counted array, constructed again each
>>>>> time it
>>>>> is assigned (very similar to a Python list).
>>>>>
>>>>>   b) That simple cases (where the pointer is never reassigned or
>>>>> copied,
>>>>> only written/read to/from by indexing) are then optimized to use
>>>>> stack
>>>>> allocation instead.
>>>>>
>>>> First and foremost the point now is to .
>>> Sorry! I meant: The point now was to argue for postponing the  
>>> decision
>>> for later and disable it in the upcoming release.
>>>
>>> I'd settle for disabling any "literal assignment" to the pointer
>>> outside
>>> of the declaration though, then I'd be back to "not worrying". (One
>>> could also disable it only inside loops, but that is too ugly and
>>> inconsistent.)
>>
>> I'm OK with only allowing it outside of declarations (and even
>> better, only to an array type), but don't know if I'll have time to
>> do that anytime soon.
>
> I don't understand? Do you mean the opposite, i.e only allowing it
> inside of declarations?

Sorry, jet lag... I meant to say disallowing it outside declarations.

> Anyway, the (final and realistic) proposal was:
>
> cdef int* a = [1, 2, 3] # ok!
>
> while
>
> cdef int* a
> a = [1, 2, 3] # not ok!
>
> At least for this release (adding features is easier than removing).
>
> This absolutely shouldn't be more than half an hour, it is just about
> throwing up a compiler error in the right transform.

Now that you put it that way, it makes it sound much easier--I love  
transformations :). But I actually realize how much more restrictive  
it is, as you can't do

cdef foo(int n, int* a):
     ...

foo(5, [1,2,3,4,5])

which I think could be very useful. So I would maybe say disallow  
array assignment except in declarations, but the above would still work.

> If you're +1 to
> this but don't have the time I could have a look during next week.  
> Have
> you decided on a timeframe for the release?

I was thinking in a week or two. I haven't tried compile Sage for a  
bit, but if that goes smoothly then I'd want an release candidate out  
soon, allowing a week for people to test.

- Robert


From ben.aurel at gmail.com  Sat Oct 11 21:30:29 2008
From: ben.aurel at gmail.com (Ben Aurel)
Date: Sat, 11 Oct 2008 12:30:29 -0700
Subject: [Cython] Setup Errors: "error: Python.h: No such file or
	directory"
In-Reply-To: <9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
	<9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>
Message-ID: <74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>

 hi robert
I've installed python from source*, and like I've wrote the 'python.h'
header file is there** but I don't know to get the cython installer to
find the file

* http://www.python.org/ftp/python/2.6/Python-2.6.tgz
**  /usr/local/python/include/python2.6/Python.h




On Sat, Oct 11, 2008 at 5:31 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> You need the developer version of Python (i.e. the Python header
> files) to use or install Cython.
>
> - Robert
>
> On Oct 11, 2008, at 5:27 AM, Ben Aurel wrote:
>
>> hi
>> I try to build Cython on my ubuntu machine according to the README:
>>     $ python setup.py install
>>
>> But I get a bunch of errors, starting with a message that says:
>>     ...
>>     /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
>> error: Python.h: No such file or directory
>>     ...
>>     (the first part of the installation log is appended at the end)
>>
>>
>> The Python.h apparently resides on the following directory on my
>> system:
>>
>>     /usr/local/python/include/python2.6/Python.h
>>
>>
>> How can I solve this setup errors? I think helping the installer to
>> find 'Python.h' is the first step. But how can I do that?
>>
>> Any ideas?
>> thanks
>> ben
>>
>> ///
>> Setup log:
>> ///
>>    1 Compiling module Cython.Plex.Scanners ...
>>    2 running install
>>    3 running build
>>    4 running build_py
>>    5 running build_ext
>>    6 building 'Cython.Plex.Scanners' extension
>>    7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
>> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c
>> /usr/local/src/Cython-0     .9.8.1.1/Cython/Plex/Scanners.c -o
>> build/temp.linux-i686-2.5/usr/local/src/Cython-0.9.8.1.1/Cython/
>> Plex/Scanners.o
>>    8 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
>> error: Python.h: No such file or directory
>>    9 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:5:26:
>> error: structmember.h: No such file or directory
>>   10 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:117:
>> error: expected specifier-qualifier-list before 'PyObject'
>>   11 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:135:
>> error: expected ')' before '*' token
>>   12 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:136:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before
>> '__pyx_PyInt_AsLon     gLong'
>>   13 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:137:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before
>> '__pyx_PyInt_AsUns     ignedLongLong'
>>   14 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:138:
>> error: expected ')' before '*' token
>>   15 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:143:
>> error: expected ')' before '*' token
>>   16 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:144:
>> error: expected ')' before '*' token
>>   17 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:145:
>> error: expected ')' before '*' token
>>   18 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:146:
>> error: expected ')' before '*' token
>>   19 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:147:
>> error: expected ')' before '*' token
>>   20 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:148:
>> error: expected ')' before '*' token
>>   21 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:149:
>> error: expected ')' before '*' token
>>   22 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:150:
>> error: expected ')' before '*' token
>>   23 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:151:
>> error: expected ')' before '*' token
>>   24 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:152:
>> error: expected ')' before '*' token
>>   25 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:153:
>> error: expected ')' before '*' token
>>   26 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:168:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   27 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:169:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   28 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:170:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   29 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:179:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   30 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:181:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   31 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:183:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   32 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:185:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>>   33 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:186:
>> error: expected ')' before '*' token
>>   34 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:188:
>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>> token
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From robertwb at math.washington.edu  Sat Oct 11 21:38:47 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 12:38:47 -0700
Subject: [Cython] Setup Errors: "error: Python.h: No such file or
	directory"
In-Reply-To: <74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
	<9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>
	<74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>
Message-ID: <13C1828E-23D8-42D4-A205-FC126A3652E7@math.washington.edu>

Can you compile the C file

------------- foo.c ----------------
#include "Python.h"

-------------------------------------

If not, then you probably need to set some environment variables  
somewhere. I'm not expert on that (and there is a huge amount of  
variance depending on exactly what system/os you're running) but I  
believe that is most likely the problem.

- Robert


On Oct 11, 2008, at 12:30 PM, Ben Aurel wrote:

>  hi robert
> I've installed python from source*, and like I've wrote the 'python.h'
> header file is there** but I don't know to get the cython installer to
> find the file
>
> * http://www.python.org/ftp/python/2.6/Python-2.6.tgz
> **  /usr/local/python/include/python2.6/Python.h
>
>
>
>
> On Sat, Oct 11, 2008 at 5:31 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> You need the developer version of Python (i.e. the Python header
>> files) to use or install Cython.
>>
>> - Robert
>>
>> On Oct 11, 2008, at 5:27 AM, Ben Aurel wrote:
>>
>>> hi
>>> I try to build Cython on my ubuntu machine according to the README:
>>>     $ python setup.py install
>>>
>>> But I get a bunch of errors, starting with a message that says:
>>>     ...
>>>     /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
>>> error: Python.h: No such file or directory
>>>     ...
>>>     (the first part of the installation log is appended at the end)
>>>
>>>
>>> The Python.h apparently resides on the following directory on my
>>> system:
>>>
>>>     /usr/local/python/include/python2.6/Python.h
>>>
>>>
>>> How can I solve this setup errors? I think helping the installer to
>>> find 'Python.h' is the first step. But how can I do that?
>>>
>>> Any ideas?
>>> thanks
>>> ben
>>>
>>> ///
>>> Setup log:
>>> ///
>>>    1 Compiling module Cython.Plex.Scanners ...
>>>    2 running install
>>>    3 running build
>>>    4 running build_py
>>>    5 running build_ext
>>>    6 building 'Cython.Plex.Scanners' extension
>>>    7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
>>> -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c
>>> /usr/local/src/Cython-0     .9.8.1.1/Cython/Plex/Scanners.c -o
>>> build/temp.linux-i686-2.5/usr/local/src/Cython-0.9.8.1.1/Cython/
>>> Plex/Scanners.o
>>>    8 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
>>> error: Python.h: No such file or directory
>>>    9 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:5:26:
>>> error: structmember.h: No such file or directory
>>>   10 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:117:
>>> error: expected specifier-qualifier-list before 'PyObject'
>>>   11 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:135:
>>> error: expected ')' before '*' token
>>>   12 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:136:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before
>>> '__pyx_PyInt_AsLon     gLong'
>>>   13 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:137:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before
>>> '__pyx_PyInt_AsUns     ignedLongLong'
>>>   14 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:138:
>>> error: expected ')' before '*' token
>>>   15 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:143:
>>> error: expected ')' before '*' token
>>>   16 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:144:
>>> error: expected ')' before '*' token
>>>   17 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:145:
>>> error: expected ')' before '*' token
>>>   18 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:146:
>>> error: expected ')' before '*' token
>>>   19 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:147:
>>> error: expected ')' before '*' token
>>>   20 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:148:
>>> error: expected ')' before '*' token
>>>   21 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:149:
>>> error: expected ')' before '*' token
>>>   22 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:150:
>>> error: expected ')' before '*' token
>>>   23 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:151:
>>> error: expected ')' before '*' token
>>>   24 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:152:
>>> error: expected ')' before '*' token
>>>   25 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:153:
>>> error: expected ')' before '*' token
>>>   26 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:168:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   27 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:169:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   28 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:170:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   29 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:179:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   30 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:181:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   31 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:183:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   32 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:185:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>>   33 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:186:
>>> error: expected ')' before '*' token
>>>   34 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:188:
>>> error: expected '=', ',', ';', 'asm' or '__attribute__' before '*'
>>> token
>>> _______________________________________________
>>> Cython-dev mailing list
>>> Cython-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/cython-dev
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From ondrej at certik.cz  Sat Oct 11 21:55:45 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Sat, 11 Oct 2008 21:55:45 +0200
Subject: [Cython] Setup Errors: "error: Python.h: No such file or
	directory"
In-Reply-To: <13C1828E-23D8-42D4-A205-FC126A3652E7@math.washington.edu>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
	<9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>
	<74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>
	<13C1828E-23D8-42D4-A205-FC126A3652E7@math.washington.edu>
Message-ID: <85b5c3130810111255n527ec52bpb3e37dbf0e22a09c@mail.gmail.com>

On Sat, Oct 11, 2008 at 9:38 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> Can you compile the C file
>
> ------------- foo.c ----------------
> #include "Python.h"
>
> -------------------------------------
>
> If not, then you probably need to set some environment variables
> somewhere. I'm not expert on that (and there is a huge amount of
> variance depending on exactly what system/os you're running) but I
> believe that is most likely the problem.

Compile this by hand:

  7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c
/usr/local/src/Cython-0     .9.8.1.1/Cython/Plex/Scanners.c -o
build/temp.linux-i686-2.5/usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.o
  8 /usr/local/src/Cython-0.9.8.1.1/Cython/Plex/Scanners.c:4:20:
error: Python.h: No such file or directory


And you need to get it working, e.g. change the -I line to point to
your dir with your python2.6 installation. After you get this working,
you need to figure out how to fix the cython's setup.py file.

Ondrej

From michael.abshoff at googlemail.com  Sun Oct 12 00:53:41 2008
From: michael.abshoff at googlemail.com (Michael Abshoff)
Date: Sat, 11 Oct 2008 15:53:41 -0700
Subject: [Cython] Setup Errors: "error: Python.h: No such file
	or	directory"
In-Reply-To: <85b5c3130810111255n527ec52bpb3e37dbf0e22a09c@mail.gmail.com>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>	<9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>	<74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>	<13C1828E-23D8-42D4-A205-FC126A3652E7@math.washington.edu>
	<85b5c3130810111255n527ec52bpb3e37dbf0e22a09c@mail.gmail.com>
Message-ID: <48F12E75.7000501@gmail.com>

Ondrej Certik wrote:
> On Sat, Oct 11, 2008 at 9:38 PM, Robert Bradshaw

<SNIP>

> And you need to get it working, e.g. change the -I line to point to
> your dir with your python2.6 installation. After you get this working,
> you need to figure out how to fix the cython's setup.py file.

One other very likely source of trouble can be when you have more than 
one python binary installed and the wrong on in $PATH. Distutils is 
usually clever enough to find the right one.

> Ondrej

Cheers,

Michael

> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
> 


From greg.ewing at canterbury.ac.nz  Sun Oct 12 02:10:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 12 Oct 2008 13:10:22 +1300
Subject: [Cython] array assignment
In-Reply-To: <4729C46F-D463-44B8-B423-64364998269E@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
	<48F0459D.5020200@canterbury.ac.nz>
	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>
	<48F054A4.8020300@canterbury.ac.nz>
	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
	<48F0B063.9030401@student.matnat.uio.no>
	<48F0B444.1050007@student.matnat.uio.no>
	<4729C46F-D463-44B8-B423-64364998269E@math.washington.edu>
Message-ID: <48F1406E.9050703@canterbury.ac.nz>

Robert Bradshaw wrote:
> Yes, the  
> arrays as they are now have a few warts (like using them in loops)  
> but I think it's too useful to disable altogether.

You could get the same effect in a much less misleading
way using

    cdef int data[] = [1, 2, 3, 4]
    cdef int *a = data

Now it's perfectly clear that a is pointing to something
stack-allocated. EIBTI.

-- 
Greg

From robertwb at math.washington.edu  Sun Oct 12 03:01:29 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sat, 11 Oct 2008 18:01:29 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F1406E.9050703@canterbury.ac.nz>
References: <20081002210040.GA15290@encolpuis>
	<20081002210306.GA15396@encolpuis>
	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>
	<48EF151B.8070005@student.matnat.uio.no>
	<48F0459D.5020200@canterbury.ac.nz>
	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>
	<48F054A4.8020300@canterbury.ac.nz>
	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>
	<48F0B063.9030401@student.matnat.uio.no>
	<48F0B444.1050007@student.matnat.uio.no>
	<4729C46F-D463-44B8-B423-64364998269E@math.washington.edu>
	<48F1406E.9050703@canterbury.ac.nz>
Message-ID: <600C5066-96D9-49B9-A34F-4C819420F8BC@math.washington.edu>

On Oct 11, 2008, at 5:10 PM, Greg Ewing wrote:

> Robert Bradshaw wrote:
>> Yes, the
>> arrays as they are now have a few warts (like using them in loops)
>> but I think it's too useful to disable altogether.
>
> You could get the same effect in a much less misleading
> way using
>
>     cdef int data[] = [1, 2, 3, 4]
>     cdef int *a = data
>
> Now it's perfectly clear that a is pointing to something
> stack-allocated. EIBTI.

Yes, but the whole point was to be more concise (specifically, better  
than setting the elements one by one, but even this is somewhat  
redundant). The reason I did it the way I did was because

cdef foo = x

gets translated to

cdef foo
foo = x

and so disallowing assignments in one place, but not in another, and  
allowing assignments to array types just once, etc. was a whole lot  
more complicated that what I wanted to deal with at the time.  
Eventually we will get there.

- Robert


From greg.ewing at canterbury.ac.nz  Sun Oct 12 06:43:19 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 12 Oct 2008 17:43:19 +1300
Subject: [Cython] Setup Errors: "error: Python.h: No such file
	or	directory"
In-Reply-To: <74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>
References: <74a4f4670810110527l20565dbfneb6948a3d1a77cc7@mail.gmail.com>
	<9AF72710-A208-4906-83F4-B7F102A0AA45@math.washington.edu>
	<74a4f4670810111230g14b02e98v55423a068f1b0165@mail.gmail.com>
Message-ID: <48F18067.3010203@canterbury.ac.nz>

Ben Aurel wrote:

> **  /usr/local/python/include/python2.6/Python.h

But

>>>   7 gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
>>>-Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c

Which looks to me like you're getting a system-installed
Python 2.5 when you run 'python', instead of the Python 2.6
that you installed yourself.

Try this instead:

   python2.6 setup.py install

Also, if you want 'python' to refer to python2.6, you may
need to adjust a symlink somewhere or adjust your PATH.

-- 
Greg

From ondrej at certik.cz  Sun Oct 12 18:54:44 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Sun, 12 Oct 2008 18:54:44 +0200
Subject: [Cython] docs.cython.org fix
In-Reply-To: <5A4FBCE3-53C3-4142-ABE3-66231C3676C1@math.washington.edu>
References: <85b5c3130810100615k6df5a979q4b612548eeaceb53@mail.gmail.com>
	<20081010174426.GA14315@encolpuis>
	<5A4FBCE3-53C3-4142-ABE3-66231C3676C1@math.washington.edu>
Message-ID: <85b5c3130810120954wb327ff4rb494c1d8cc22b050@mail.gmail.com>

On Fri, Oct 10, 2008 at 9:20 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 10, 2008, at 10:44 AM, Gabriel Gellner wrote:
>
>> On Fri, Oct 10, 2008 at 03:15:58PM +0200, Ondrej Certik wrote:
>>> Hi,
>>>
>>> please pull a link fix from here:
>>>
>>> http://www.bitbucket.org/certik/cython-doc/
>>>
>>> e.g.
>>>
>>> hg pull http://www.bitbucket.org/certik/cython-doc/
>>>
>> Thanks I will do this tonight, finally having to learn to used
>> mercurial . . .
>
> I actually already merged it and pushed to hg.cython.org/cython-docs

Thanks. Could someone please also update the doc.cython.org?

Ondrej

From animator333 at yahoo.com  Mon Oct 13 10:19:39 2008
From: animator333 at yahoo.com (Prashant Saxena)
Date: Mon, 13 Oct 2008 13:49:39 +0530 (IST)
Subject: [Cython] pubsub module and cython
Message-ID: <66663.23468.qm@web94913.mail.in2.yahoo.com>

Hi,

I did a simple test of using pubsub module and cython. It seems cython is doing something wrong and not able to properly handle listener.
The code is zipped here.

1. Run "test.py" and everything is working fine.
2. you can compile "test.py" using "setup.bat" on windows. Compiled "test.pyd" is also available.
3. Run "demo.py". Here you have an error.

Regards

Prashant



      Unlimited freedom, unlimited storage. Get it now, on http://help.yahoo.com/l/in/yahoo/mail/yahoomail/tools/tools-08.html/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081013/028baa28/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pubsub_test.zip
Type: application/zip
Size: 17203 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081013/028baa28/attachment.zip 

From stefan_ml at behnel.de  Mon Oct 13 11:53:35 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 13 Oct 2008 11:53:35 +0200 (CEST)
Subject: [Cython] pubsub module and cython
In-Reply-To: <66663.23468.qm@web94913.mail.in2.yahoo.com>
References: <66663.23468.qm@web94913.mail.in2.yahoo.com>
Message-ID: <60556.213.61.181.86.1223891615.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Hi,

it's usually a good idea to provide some more more information than "doing
something wrong" and "an error" in your problem report, especially some
hints on what the error output is. Otherwise, you require everyone to
unpack your test code, compile it and run it, although the error
description might already give some (or even enough) hints on what goes
wrong and who might be able to help you.

Stefan

Prashant Saxena wrote:
> I did a simple test of using pubsub module and cython. It seems cython is
> doing something wrong and not able to properly handle listener.
> The code is zipped here.
>
> 1. Run "test.py" and everything is working fine.
> 2. you can compile "test.py" using "setup.bat" on windows. Compiled
> "test.pyd" is also available.
> 3. Run "demo.py". Here you have an error.
>
> Regards
>
> Prashant


From animator333 at yahoo.com  Mon Oct 13 13:11:13 2008
From: animator333 at yahoo.com (Prashant Saxena)
Date: Mon, 13 Oct 2008 16:41:13 +0530 (IST)
Subject: [Cython] pubsub module and cython
Message-ID: <665018.6990.qm@web94905.mail.in2.yahoo.com>

Ok, Here are the details

Os win 2k
python 2.5.2
cython  0.9.8.1.1

1. I am using pure python code to convert .py into .c and compile using mingw32.( no cdef stuff)
2. test.py is working fine until unless I am converting into c extension.
3. When using test.pyd, I am getting following error:

Traceback (most recent call last):
  File "demo.py", line 1, in <module>
    import test
  File "test.py", line 33, in test (test..c:831)
    app.MainLoop()
  File "test.py", line 18, in test.MyButtons..__init__ (test.c:413)
    
  File "C:\Python25\Lib\site-packages\pubsubcore\publisher.py", line 86, in subscribe
    topicName, desc=noDesc, usingCallable=listener)
  File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line 423, in _newTopicFromTemplate_
    allArgsDocs, required = topicArgsFromCallable(usingCallable)
  File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line 618, in topicArgsFromCallable
    argsInfo = getListenerArgs(_callable)
  File "C:\Python25\lib\site-packages\pubsubcore\callables.py", line 158, in getArgs
    (args, va, vkwa, defaultVals) = getargspec(func)
  File "C:\Python25\lib\inspect.py", line 742, in getargspec
    raise TypeError('arg is not a Python function')
TypeError: arg is not a Python function


In second last line of error it seems the listener(self.OnPubSub) at line no. 17 in test.py:
Publisher.subscribe(self.OnPubSub, "topic")
is changed into something else which "inspect.py" is not able to recognized as a function.
Although subscription method is almost same as wxPython events binding, which is getting compiled and giving no problems.

I gave a try to pydispatch module and wx.lib.pubsub module to do the same thing but none of them are working when test.py is compiled into .pyd.
All of the modules are showing same type of error at line no.17 where we are subscribing our listener. Still I am not able to fugure out where actually the problem is, is it in the modules or is it with cython?

Hope this helps.

Prashant




----- Original Message ----
From: Stefan Behnel <stefan_ml at behnel.de>
To: cython-dev at codespeak.net
Sent: Monday, 13 October, 2008 3:23:35 PM
Subject: Re: [Cython] pubsub module and cython

Hi,

it's usually a good idea to provide some more more information than "doing
something wrong" and "an error" in your problem report, especially some
hints on what the error output is. Otherwise, you require everyone to
unpack your test code, compile it and run it, although the error
description might already give some (or even enough) hints on what goes
wrong and who might be able to help you.

Stefan

Prashant Saxena wrote:
> I did a simple test of using pubsub module and cython. It seems cython is
> doing something wrong and not able to properly handle listener.
> The code is zipped here.
>
> 1. Run "test.py" and everything is working fine.
> 2. you can compile "test.py" using "setup.bat" on windows. Compiled
> "test.pyd" is also available.
> 3. Run "demo.py". Here you have an error.
>
> Regards
>
> Prashant

_______________________________________________
Cython-dev mailing list
Cython-dev at codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev



      Check out the all-new face of Yahoo! India. Go to http://in.yahoo.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081013/095d78b0/attachment.htm 

From robertwb at math.washington.edu  Mon Oct 13 13:36:20 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Mon, 13 Oct 2008 04:36:20 -0700
Subject: [Cython] pubsub module and cython
In-Reply-To: <665018.6990.qm@web94905.mail.in2.yahoo.com>
References: <665018.6990.qm@web94905.mail.in2.yahoo.com>
Message-ID: <F396AD8F-0BA6-439B-A98B-FB770652E4E6@math.washington.edu>

That is correct, arg is not a Python function, it is a compiled  
function. It looks like the listener interface can only accept pure  
Python functions.

You could look at trying to fix wx, but I would recommend writing  
your test.py as an ordinary Python module (it doesn't look like that  
part of the code will benefit from Cython much) and wrapping the  
speed-critical/library interface parts in Cython.

- Robert

On Oct 13, 2008, at 4:11 AM, Prashant Saxena wrote:

> Ok, Here are the details
>
> Os win 2k
> python 2.5.2
> cython 0.9.8.1.1
>
> 1. I am using pure python code to convert .py into .c and compile  
> using mingw32.( no cdef stuff)
> 2. test.py is working fine until unless I am converting into c  
> extension.
> 3. When using test.pyd, I am getting following error:
>
> Traceback (most recent call last):
>   File "demo.py", line 1, in <module>
>     import test
>   File "test.py", line 33, in test (test.c:831)
>     app.MainLoop()
>   File "test.py", line 18, in test.MyButtons.__init__ (test.c:413)
>
>   File "C:\Python25\Lib\site-packages\pubsubcore\publisher.py",  
> line 86, in subscribe
>     topicName, desc=noDesc, usingCallable=listener)
>   File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line  
> 423, in _newTopicFromTemplate_
>     allArgsDocs, required = topicArgsFromCallable(usingCallable)
>   File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line  
> 618, in topicArgsFromCallable
>     argsInfo = getListenerArgs(_callable)
>   File "C:\Python25\lib\site-packages\pubsubcore\callables.py",  
> line 158, in getArgs
>     (args, va, vkwa, defaultVals) = getargspec(func)
>   File "C:\Python25\lib\inspect.py", line 742, in getargspec
>     raise TypeError('arg is not a Python function')
> TypeError: arg is not a Python function
>
>
> In second last line of error it seems the listener(self.OnPubSub)  
> at line no. 17 in test.py:
> Publisher.subscribe(self.OnPubSub, "topic")
> is changed into something else which "inspect.py" is not able to  
> recognized as a function.
> Although subscription method is almost same as wxPython events  
> binding, which is getting compiled and giving no problems.
>
> I gave a try to pydispatch module and wx.lib.pubsub module to do  
> the same thing but none of them are working when test.py is  
> compiled into .pyd.
> All of the modules are showing same type of error at line no.17  
> where we are subscribing our listener. Still I am not able to  
> fugure out where actually the problem is, is it in the modules or  
> is it with cython?
>
> Hope this helps.
>
> Prashant
>
>
> ----- Original Message ----
> From: Stefan Behnel <stefan_ml at behnel.de>
> To: cython-dev at codespeak.net
> Sent: Monday, 13 October, 2008 3:23:35 PM
> Subject: Re: [Cython] pubsub module and cython
>
> Hi,
>
> it's usually a good idea to provide some more more information than  
> "doing
> something wrong" and "an error" in your problem report, especially  
> some
> hints on what the error output is. Otherwise, you require everyone to
> unpack your test code, compile it and run it, although the error
> description might already give some (or even enough) hints on what  
> goes
> wrong and who might be able to help you.
>
> Stefan
>
> Prashant Saxena wrote:
> > I did a simple test of using pubsub module and cython. It seems  
> cython is
> > doing something wrong and not able to properly handle listener.
> > The code is zipped here.
> >
> > 1. Run "test.py" and everything is working fine.
> > 2. you can compile "test.py" using "setup.bat" on windows. Compiled
> > "test.pyd" is also available.
> > 3. Run "demo.py". Here you have an error.
> >
> > Regards
> >
> > Prashant
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
> Add more friends to your messenger and enjoy! Invite them now.
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From ondrej at certik.cz  Mon Oct 13 15:10:07 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 13 Oct 2008 15:10:07 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
Message-ID: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>

Hi,

what is now the canonical way to convert between (python) numpy array and

int *a

or

double *a

?

I looked at the new petsc4py and it uses this method to convert from numpy to C:

cdef inline ndarray iarray_s(object ob, PetscInt* size, PetscScalar** data):
    ob = PyArray_FROM_OTF(ob, NPY_PETSC_SCALAR, NPY_IN_ARRAY)
    cdef ndarray ary = <ndarray> ob
    if size!=NULL: size[0] = <PetscInt>     PyArray_SIZE(ary)
    if data!=NULL: data[0] = <PetscScalar*> PyArray_DATA(ary)
    return ary

The function PyArray_FROM_OTF() is defined in /usr/include/numpy/ndarrayobject.h

And this method to convert from C to numpy:

cdef inline ndarray array_s(PetscInt size, const_PetscScalar* data):
    cdef npy_intp s = <npy_intp> size
    cdef ndarray ary = PyArray_EMPTY(1, &s, NPY_PETSC_SCALAR, 0)
    if data != NULL: memcpy(ary.cdata, data, size*sizeof(PetscScalar))
    return ary


Is this the best way to do things now? When I investigated this half a
year ago, the way to do this was this:

http://wiki.cython.org/WrappingNumpy

however Robert wrote there that now the way is this:

http://wiki.cython.org/tutorials/numpy

unfortunately at the latter page it is only explained how to access
indices of numpy arrays using the buffer interface, but not how to
convert them back and forth to C.

Maybe the canonical methods could be shipped with either numpy or
cython? I can provide a patch, once I learn how to do things properly.

Thanks,
Ondrej

From animator333 at yahoo.com  Mon Oct 13 15:29:30 2008
From: animator333 at yahoo.com (Prashant Saxena)
Date: Mon, 13 Oct 2008 18:59:30 +0530 (IST)
Subject: [Cython] pubsub module and cython
Message-ID: <22416.81801.qm@web94908.mail.in2.yahoo.com>

I would prefer to do something with publisher module. The example that was included with this topic is for to show the problem only.
I am using publisher in couple of files where I am about to change the code to get the maximum gain using cython(cdef, loops and other stuff). 
Now I have to find a solution for this problem first, later I'll jump for conversion.

Could you please suggest me some hacks that would make this thing work If I would do some changes in pubsub modules or anything?

Prashant



----- Original Message ----
From: Robert Bradshaw <robertwb at math.washington.edu>
To: cython-dev at codespeak.net
Sent: Monday, 13 October, 2008 5:06:20 PM
Subject: Re: [Cython] pubsub module and cython

That is correct, arg is not a Python function, it is a compiled  
function. It looks like the listener interface can only accept pure  
Python functions.

You could look at trying to fix wx, but I would recommend writing  
your test.py as an ordinary Python module (it doesn't look like that  
part of the code will benefit from Cython much) and wrapping the  
speed-critical/library interface parts in Cython.

- Robert

On Oct 13, 2008, at 4:11 AM, Prashant Saxena wrote:

> Ok, Here are the details
>
> Os win 2k
> python 2.5.2
> cython 0.9.8.1.1
>
> 1. I am using pure python code to convert .py into .c and compile  
> using mingw32.( no cdef stuff)
> 2. test.py is working fine until unless I am converting into c  
> extension.
> 3. When using test.pyd, I am getting following error:
>
> Traceback (most recent call last):
>   File "demo.py", line 1, in <module>
>     import test
>   File "test.py", line 33, in test (test.c:831)
>     app.MainLoop()
>   File "test.py", line 18, in test.MyButtons.__init__ (test.c:413)
>
>   File "C:\Python25\Lib\site-packages\pubsubcore\publisher.py",  
> line 86, in subscribe
>     topicName, desc=noDesc, usingCallable=listener)
>   File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line  
> 423, in _newTopicFromTemplate_
>     allArgsDocs, required = topicArgsFromCallable(usingCallable)
>   File "C:\Python25\Lib\site-packages\pubsubcore\topics.py", line  
> 618, in topicArgsFromCallable
>     argsInfo = getListenerArgs(_callable)
>   File "C:\Python25\lib\site-packages\pubsubcore\callables.py",  
> line 158, in getArgs
>     (args, va, vkwa, defaultVals) = getargspec(func)
>   File "C:\Python25\lib\inspect.py", line 742, in getargspec
>     raise TypeError('arg is not a Python function')
> TypeError: arg is not a Python function
>
>
> In second last line of error it seems the listener(self.OnPubSub)  
> at line no. 17 in test.py:
> Publisher.subscribe(self.OnPubSub, "topic")
> is changed into something else which "inspect.py" is not able to  
> recognized as a function.
> Although subscription method is almost same as wxPython events  
> binding, which is getting compiled and giving no problems.
>
> I gave a try to pydispatch module and wx.lib.pubsub module to do  
> the same thing but none of them are working when test.py is  
> compiled into .pyd.
> All of the modules are showing same type of error at line no.17  
> where we are subscribing our listener. Still I am not able to  
> fugure out where actually the problem is, is it in the modules or  
> is it with cython?
>
> Hope this helps.
>
> Prashant
>
>
> ----- Original Message ----
> From: Stefan Behnel <stefan_ml at behnel.de>
> To: cython-dev at codespeak.net
> Sent: Monday, 13 October, 2008 3:23:35 PM
> Subject: Re: [Cython] pubsub module and cython
>
> Hi,
>
> it's usually a good idea to provide some more more information than  
> "doing
> something wrong" and "an error" in your problem report, especially  
> some
> hints on what the error output is. Otherwise, you require everyone to
> unpack your test code, compile it and run it, although the error
> description might already give some (or even enough) hints on what  
> goes
> wrong and who might be able to help you.
>
> Stefan
>
> Prashant Saxena wrote:
> > I did a simple test of using pubsub module and cython. It seems  
> cython is
> > doing something wrong and not able to properly handle listener.
> > The code is zipped here.
> >
> > 1. Run "test.py" and everything is working fine.
> > 2. you can compile "test.py" using "setup.bat" on windows. Compiled
> > "test.pyd" is also available.
> > 3. Run "demo.py". Here you have an error.
> >
> > Regards
> >
> > Prashant
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
> Add more friends to your messenger and enjoy! Invite them now.
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev

_______________________________________________
Cython-dev mailing list
Cython-dev at codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev



      Be the first one to try the new Messenger 9 Beta! Go to http://in.messenger.yahoo.com/win/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081013/ededcf66/attachment-0001.htm 

From dagss at student.matnat.uio.no  Mon Oct 13 15:38:08 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 13 Oct 2008 15:38:08 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
Message-ID: <48F34F40.7040705@student.matnat.uio.no>

Ondrej Certik wrote:
> Hi,
>
> what is now the canonical way to convert between (python) numpy array and
>
> int *a
>
> or
>
> double *a
>
> ?
>   
Yes, this should probably be fixed/be made more available.

The simplest thing is something like the (untested) code below. It is 
certainly possible to do some C calls instead of Python calls here (like 
petsc4py does); however I tink this definitely belong to the 80% of the 
code that only takes 20% of the time; Cython seems to be more about 
optimizing the other part that tends to get repeated :-)

import numpy as np
arr = some generic numpy array
assert arr.dtype == np.float64 # or whatever
cdef ndarray contarr
if not arr.flags['C_CONTIGUOUS']:
    # Array is not contiguous, need to make contiguous copy
    contarr = arr.copy(order='C')
else:
    contarr = arr
# Get data pointer. Important that contarr is cdef-ed ndarray
cdef npy.float64_t* ptr = <npy.float64_t*>contarr.data
call_c_function(ptr)
# If the C function modifies the data, and the array was not contiguous,
# then the data of contarr must be copied into arr, using standard NumPy 
calls:
arr[:] = contarr[:]

Comment: The only "shady" part here is "contarr.data", which accesses 
implementation details of NumPy arrays. I'm guessing that this will 
never change, but I once planned to make a generic cython function 
"cython.buffer.bufptr" which would return the same pointer but it could 
be acquired through the buffer API.

Note that the code above does not make use of the buffer API, as it is 
written to work regardless of number of dimensions.

Dag Sverre

From ondrej at certik.cz  Mon Oct 13 17:32:51 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 13 Oct 2008 17:32:51 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <48F34F40.7040705@student.matnat.uio.no>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
	<48F34F40.7040705@student.matnat.uio.no>
Message-ID: <85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>

On Mon, Oct 13, 2008 at 3:38 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Ondrej Certik wrote:
>> Hi,
>>
>> what is now the canonical way to convert between (python) numpy array and
>>
>> int *a
>>
>> or
>>
>> double *a
>>
>> ?
>>
> Yes, this should probably be fixed/be made more available.
>
> The simplest thing is something like the (untested) code below. It is
> certainly possible to do some C calls instead of Python calls here (like
> petsc4py does); however I tink this definitely belong to the 80% of the
> code that only takes 20% of the time; Cython seems to be more about
> optimizing the other part that tends to get repeated :-)
>
> import numpy as np
> arr = some generic numpy array
> assert arr.dtype == np.float64 # or whatever
> cdef ndarray contarr
> if not arr.flags['C_CONTIGUOUS']:
>    # Array is not contiguous, need to make contiguous copy
>    contarr = arr.copy(order='C')
> else:
>    contarr = arr
> # Get data pointer. Important that contarr is cdef-ed ndarray
> cdef npy.float64_t* ptr = <npy.float64_t*>contarr.data
> call_c_function(ptr)
> # If the C function modifies the data, and the array was not contiguous,
> # then the data of contarr must be copied into arr, using standard NumPy
> calls:
> arr[:] = contarr[:]
>
> Comment: The only "shady" part here is "contarr.data", which accesses
> implementation details of NumPy arrays. I'm guessing that this will
> never change, but I once planned to make a generic cython function
> "cython.buffer.bufptr" which would return the same pointer but it could
> be acquired through the buffer API.
>
> Note that the code above does not make use of the buffer API, as it is
> written to work regardless of number of dimensions.

Currently I use these methods:

cdef inline ndarray array_d(int size, double *data):
    #cdef ndarray ary2 = PyArray_ZEROS(1, &size, 12, 0)
    cdef ndarray ary = zeros(size, dtype=float64)
    if data != NULL: memcpy(ary.data, data, size*sizeof(double))
    return ary

cdef inline int iarray_d(ndarray a, int *size, double **data) except -1:
    if a.dtype != float64:
        raise TypeError("The array must have the dtype=float64.")
    if size!=NULL: size[0] = a.dimensions[0]
    if data!=NULL: data[0] = <double *> (a.data)



If I uncomment the commented out line (i.e. to call numpy C/API
directly instead of through Python), I'll get a segfault. I haven't
figured out yet why. But nevertheless, the current version works.

Ondrej

From robert.kern at gmail.com  Mon Oct 13 18:02:59 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Mon, 13 Oct 2008 11:02:59 -0500
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <48F34F40.7040705@student.matnat.uio.no>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
	<48F34F40.7040705@student.matnat.uio.no>
Message-ID: <gcvrfj$ibi$1@ger.gmane.org>

Dag Sverre Seljebotn wrote:

> Comment: The only "shady" part here is "contarr.data", which accesses 
> implementation details of NumPy arrays. I'm guessing that this will 
> never change, but I once planned to make a generic cython function 
> "cython.buffer.bufptr" which would return the same pointer but it could 
> be acquired through the buffer API.

The canonical way to avoid accessing .data directly is to use the PyArray_DATA() 
macro (returns a void*). Similarly, dimensions and strides can be accessed via 
PyArray_DIMS() and PyArray_STRIDES().

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From dagss at student.matnat.uio.no  Mon Oct 13 18:11:20 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 13 Oct 2008 18:11:20 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>	<48F34F40.7040705@student.matnat.uio.no>
	<85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>
Message-ID: <48F37328.3030205@student.matnat.uio.no>

Ondrej Certik wrote:
> On Mon, Oct 13, 2008 at 3:38 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Ondrej Certik wrote:
>>> Hi,
>>>
>>> what is now the canonical way to convert between (python) numpy array and
>>>
>>> int *a
>>>
>>> or
>>>
>>> double *a
>>>
>>> ?
>>>
>> Yes, this should probably be fixed/be made more available.
>>
>> The simplest thing is something like the (untested) code below. It is
>> certainly possible to do some C calls instead of Python calls here (like
>> petsc4py does); however I tink this definitely belong to the 80% of the
>> code that only takes 20% of the time; Cython seems to be more about
>> optimizing the other part that tends to get repeated :-)
>>
>> import numpy as np
>> arr = some generic numpy array
>> assert arr.dtype == np.float64 # or whatever
>> cdef ndarray contarr
>> if not arr.flags['C_CONTIGUOUS']:
>>    # Array is not contiguous, need to make contiguous copy
>>    contarr = arr.copy(order='C')
>> else:
>>    contarr = arr
>> # Get data pointer. Important that contarr is cdef-ed ndarray
>> cdef npy.float64_t* ptr = <npy.float64_t*>contarr.data
>> call_c_function(ptr)
>> # If the C function modifies the data, and the array was not contiguous,
>> # then the data of contarr must be copied into arr, using standard NumPy
>> calls:
>> arr[:] = contarr[:]
>>
>> Comment: The only "shady" part here is "contarr.data", which accesses
>> implementation details of NumPy arrays. I'm guessing that this will
>> never change, but I once planned to make a generic cython function
>> "cython.buffer.bufptr" which would return the same pointer but it could
>> be acquired through the buffer API.
>>
>> Note that the code above does not make use of the buffer API, as it is
>> written to work regardless of number of dimensions.
> 
> Currently I use these methods:
> 
> cdef inline ndarray array_d(int size, double *data):
>     #cdef ndarray ary2 = PyArray_ZEROS(1, &size, 12, 0)
>     cdef ndarray ary = zeros(size, dtype=float64)
>     if data != NULL: memcpy(ary.data, data, size*sizeof(double))
>     return ary
> 
> cdef inline int iarray_d(ndarray a, int *size, double **data) except -1:
>     if a.dtype != float64:
>         raise TypeError("The array must have the dtype=float64.")
>     if size!=NULL: size[0] = a.dimensions[0]
>     if data!=NULL: data[0] = <double *> (a.data)
> 
> 
> 
> If I uncomment the commented out line (i.e. to call numpy C/API
> directly instead of through Python), I'll get a segfault. I haven't
> figured out yet why. But nevertheless, the current version works.

The issue you are running into is likely that the size passed to 
PyArray_ZEROS must be an "npy_intp" rather than "int", which makes it 
64-bit on 64-bit platforms. As you pass it by its address this is a 
rather crucial point (otherwise you end up trying to allocate a rather 
random number of gigabytes).

This actually came up earlier on this mailing list. It would be great if 
you could make a documentation patch against ndarrayobject.h and submit 
it on the NumPy mailing list since it obviously seems like a common 
problem. Patches for numpy.pxd (including PyArray_ZEROS etc. declared 
with the right pointer types :-) ) are also welcome, currently numpy.pxd 
is geared towards high-level Python use and excludes a lot of C-level 
functions (because of lack of time/laziness only, throwing them in there 
doesn't hurt).

HOWEVER, note that your iarray_d will only work if the array that is 
passed in is contiguous. So you should either a) insert an assertion of 
a.flags['C_CONTIGUOUS'] at the beginning, or b) fix it so that it works 
with all arrays (as by my example).

Any non-trivial NumPy usage is likely to end up with non-contiguous 
arrays, so if the "a" argument could come from the end-user somehow then 
choking on non-contiguous arrays is not going to work well.


-- 
Dag Sverre

From dagss at student.matnat.uio.no  Mon Oct 13 18:17:10 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 13 Oct 2008 18:17:10 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <gcvrfj$ibi$1@ger.gmane.org>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>	<48F34F40.7040705@student.matnat.uio.no>
	<gcvrfj$ibi$1@ger.gmane.org>
Message-ID: <48F37486.40304@student.matnat.uio.no>

Robert Kern wrote:
> Dag Sverre Seljebotn wrote:
> 
>> Comment: The only "shady" part here is "contarr.data", which accesses 
>> implementation details of NumPy arrays. I'm guessing that this will 
>> never change, but I once planned to make a generic cython function 
>> "cython.buffer.bufptr" which would return the same pointer but it could 
>> be acquired through the buffer API.
> 
> The canonical way to avoid accessing .data directly is to use the PyArray_DATA() 
> macro (returns a void*). Similarly, dimensions and strides can be accessed via 
> PyArray_DIMS() and PyArray_STRIDES().

Good point. (And I already do this for the buffer access in numpy.pxd, 
so the buffer access stuff is forwards-compatible with any change here.)

The reason I want cython.buffer.bufptr BTW is to have a common way of 
accessing similar pointers for all buffer exporters, not only NumPy arrays.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Mon Oct 13 18:33:06 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 13 Oct 2008 18:33:06 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <48F37328.3030205@student.matnat.uio.no>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>	<48F34F40.7040705@student.matnat.uio.no>	<85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>
	<48F37328.3030205@student.matnat.uio.no>
Message-ID: <48F37842.4070306@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Ondrej Certik wrote:
>>
>> cdef inline ndarray array_d(int size, double *data):
>>     #cdef ndarray ary2 = PyArray_ZEROS(1, &size, 12, 0)
>>     cdef ndarray ary = zeros(size, dtype=float64)
>>     if data != NULL: memcpy(ary.data, data, size*sizeof(double))
>>     return ary
>>
>> cdef inline int iarray_d(ndarray a, int *size, double **data) except -1:
>>     if a.dtype != float64:
>>         raise TypeError("The array must have the dtype=float64.")
>>     if size!=NULL: size[0] = a.dimensions[0]
>>     if data!=NULL: data[0] = <double *> (a.data)
>>

> HOWEVER, note that your iarray_d will only work if the array that is 
> passed in is contiguous. So you should either a) insert an assertion of 
> a.flags['C_CONTIGUOUS'] at the beginning, or b) fix it so that it works 
> with all arrays (as by my example).
> 
> Any non-trivial NumPy usage is likely to end up with non-contiguous 
> arrays, so if the "a" argument could come from the end-user somehow then 
> choking on non-contiguous arrays is not going to work well.
> 
> 

Note that this is non-trivial since any copy made must also not be 
deallocated, so that iarray_d would have to return a reference which 
must be held on to for the duration of using the buffer data.

If you need to accept ndarrays coming from the user, a better idiom is 
likely to take a callback function which will be called with the buffer 
as an argument, or similar -- returning the buffer through a double** 
(or np.float64_t**) is rather error-prone when it comes to reference 
counting.

-- 
Dag Sverre

From stefan_ml at behnel.de  Mon Oct 13 19:21:49 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Mon, 13 Oct 2008 19:21:49 +0200
Subject: [Cython] pubsub module and cython
In-Reply-To: <22416.81801.qm@web94908.mail.in2.yahoo.com>
References: <22416.81801.qm@web94908.mail.in2.yahoo.com>
Message-ID: <48F383AD.5070204@behnel.de>

Hi,

Prashant Saxena wrote:
> I would prefer to do something with publisher module. The example that was included with this topic is for to show the problem only.
> I am using publisher in couple of files where I am about to change the code to get the maximum gain using cython(cdef, loops and other stuff). 
> Now I have to find a solution for this problem first, later I'll jump for conversion.
> 
> Could you please suggest me some hacks that would make this thing work If I would do some changes in pubsub modules or anything?

As Robert suggested: try to find the parts of your code that are performance
critical and use Cython for them. And find the parts that require Python code
and implement them in plain Python. Cython allows you to mix both pretty
freely, so this will allow you to fine-tune the point of language switch as
you see fit.

Stefan

From animator333 at yahoo.com  Mon Oct 13 21:03:06 2008
From: animator333 at yahoo.com (Prashant Saxena)
Date: Tue, 14 Oct 2008 00:33:06 +0530 (IST)
Subject: [Cython] pubsub module and cython
Message-ID: <943288.16772.qm@web94902.mail.in2.yahoo.com>

After investigating a lot in pubsub module, I have written my own custom dispatcher(50 lines :-)) and it's working like a charm with cython.

cython rocks!!!

Thanks

Prashant



----- Original Message ----
From: Stefan Behnel <stefan_ml at behnel.de>
To: cython-dev at codespeak.net
Sent: Monday, 13 October, 2008 10:51:49 PM
Subject: Re: [Cython] pubsub module and cython

Hi,

Prashant Saxena wrote:
> I would prefer to do something with publisher module. The example that was included with this topic is for to show the problem only.
> I am using publisher in couple of files where I am about to change the code to get the maximum gain using cython(cdef, loops and other stuff). 
> Now I have to find a solution for this problem first, later I'll jump for conversion.
> 
> Could you please suggest me some hacks that would make this thing work If I would do some changes in pubsub modules or anything?

As Robert suggested: try to find the parts of your code that are performance
critical and use Cython for them. And find the parts that require Python code
and implement them in plain Python. Cython allows you to mix both pretty
freely, so this will allow you to fine-tune the point of language switch as
you see fit.

Stefan
_______________________________________________
Cython-dev mailing list
Cython-dev at codespeak.net
http://codespeak.net/mailman/listinfo/cython-dev



      Add more friends to your messenger and enjoy! Go to http://messenger.yahoo.com/invite/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/cython-dev/attachments/20081014/f01bcfdd/attachment-0001.htm 

From dagss at student.matnat.uio.no  Mon Oct 13 22:44:40 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Mon, 13 Oct 2008 22:44:40 +0200
Subject: [Cython] Literal struct syntax
Message-ID: <48F3B338.4060308@student.matnat.uio.no>

I'm sorry to bring this up as people would rather code and discuss other 
stuff, but, well, once things are released they tend to stick. So I'll 
just raise the concern and you do what you want to do...

My concern is about the dict struct construction syntax -- from 
"struct_conversion.pyx":

cdef Point p = {color: color, x: x, y: y}

That is, without string keys, but rather some magic keys. This feels 
very odd to a Python eye -- especially as the coercion the other way 
(from struct to dict) seems to have normal string keys.

For more brevity without typing all the " there's always the 
Point(color=...), syntax, and I feel being consistent with keeping it a 
normal dict is important because a later feature might be

cdef Point p
d = {"color": color, "x" : x, "y" : y}
p = d

and then these "literal dict vs. Python dict" kind of thing might be 
even more confusing.

-- 
Dag Sverre

From dagss at student.matnat.uio.no  Tue Oct 14 09:12:40 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Oct 2008 09:12:40 +0200
Subject: [Cython] array assignment
In-Reply-To: <5EE66792-5EE4-4963-90DB-58143DC01482@math.washington.edu>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>	<48F0B444.1050007@student.matnat.uio.no>	<48F0B57F.7030209@student.matnat.uio.no>	<7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>	<48F0B8D6.7010502@student.matnat.uio.no>
	<5EE66792-5EE4-4963-90DB-58143DC01482@math.washington.edu>
Message-ID: <48F44668.5040703@student.matnat.uio.no>

Robert Bradshaw wrote:
> On Oct 11, 2008, at 7:31 AM, Dag Sverre Seljebotn wrote:
>> This absolutely shouldn't be more than half an hour, it is just about
>> throwing up a compiler error in the right transform.
> 
> Now that you put it that way, it makes it sound much easier--I love  
> transformations :). But I actually realize how much more restrictive  
> it is, as you can't do
> 
> cdef foo(int n, int* a):
>      ...
> 
> foo(5, [1,2,3,4,5])
> 
> which I think could be very useful. So I would maybe say disallow  
> array assignment except in declarations, but the above would still work.

As you pointed out in another post this was more hairy than I expected 
because of cdef transformation.

But anyway a patch is up doing this now. It is not tied to declaration, 
but what can Cython prove is the first assignment, but that works just 
as well (or better) for this purpose! (And is the same currently).

-- 
Dag Sverre

From robertwb at math.washington.edu  Tue Oct 14 10:12:22 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Oct 2008 01:12:22 -0700
Subject: [Cython] array assignment
In-Reply-To: <48F44668.5040703@student.matnat.uio.no>
References: <20081002210040.GA15290@encolpuis>	<20081002210306.GA15396@encolpuis>	<EAF2E87A-FF8C-41D5-B25D-33BC46E381C9@math.washington.edu>	<48EF151B.8070005@student.matnat.uio.no>	<48F0459D.5020200@canterbury.ac.nz>	<D32E68BE-56A6-4EF1-927D-D5505EAD87EC@math.washington.edu>	<48F054A4.8020300@canterbury.ac.nz>	<086685DA-2F6A-4B59-AC1B-7A40F3CEC8E9@math.washington.edu>	<48F0B063.9030401@student.matnat.uio.no>	<48F0B444.1050007@student.matnat.uio.no>	<48F0B57F.7030209@student.matnat.uio.no>	<7C0C6049-2616-449F-9179-588CDAE32219@math.washington.edu>	<48F0B8D6.7010502@student.matnat.uio.no>
	<5EE66792-5EE4-4963-90DB-58143DC01482@math.washington.edu>
	<48F44668.5040703@student.matnat.uio.no>
Message-ID: <087D00CB-AFF0-4736-A0DC-2D9836CA428E@math.washington.edu>

On Oct 14, 2008, at 12:12 AM, Dag Sverre Seljebotn wrote:

> Robert Bradshaw wrote:
>> On Oct 11, 2008, at 7:31 AM, Dag Sverre Seljebotn wrote:
>>> This absolutely shouldn't be more than half an hour, it is just  
>>> about
>>> throwing up a compiler error in the right transform.
>>
>> Now that you put it that way, it makes it sound much easier--I love
>> transformations :). But I actually realize how much more restrictive
>> it is, as you can't do
>>
>> cdef foo(int n, int* a):
>>      ...
>>
>> foo(5, [1,2,3,4,5])
>>
>> which I think could be very useful. So I would maybe say disallow
>> array assignment except in declarations, but the above would still  
>> work.
>
> As you pointed out in another post this was more hairy than I expected
> because of cdef transformation.
>
> But anyway a patch is up doing this now. It is not tied to  
> declaration,
> but what can Cython prove is the first assignment, but that works just
> as well (or better) for this purpose! (And is the same currently).

Thanks. A bit more restrictive, but certainly harder for the user to  
invoke confusing behavior.

- Robert



From robertwb at math.washington.edu  Tue Oct 14 10:20:07 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 14 Oct 2008 01:20:07 -0700
Subject: [Cython] Literal struct syntax
In-Reply-To: <48F3B338.4060308@student.matnat.uio.no>
References: <48F3B338.4060308@student.matnat.uio.no>
Message-ID: <E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>

On Oct 13, 2008, at 1:44 PM, Dag Sverre Seljebotn wrote:

> I'm sorry to bring this up as people would rather code and discuss  
> other
> stuff, but, well, once things are released they tend to stick. So I'll
> just raise the concern and you do what you want to do...

No, it is good to bring this up now. In fact, I would encourage  
everyone to look at
http://hg.cython.org/cython-devel/file/5a7f83a639f7/tests/run/ 
struct_conversion.pyx and say what they think.

> My concern is about the dict struct construction syntax -- from
> "struct_conversion.pyx":
>
> cdef Point p = {color: color, x: x, y: y}
>
> That is, without string keys, but rather some magic keys. This feels
> very odd to a Python eye -- especially as the coercion the other way
> (from struct to dict) seems to have normal string keys.

As a (weak) counter argument, I found myself forgetting the quotes  
when testing, and it's not much more magic than the magic string kwds  
arguments keys.

> For more brevity without typing all the " there's always the
> Point(color=...), syntax, and I feel being consistent with keeping  
> it a
> normal dict is important because a later feature might be
>
> cdef Point p
> d = {"color": color, "x" : x, "y" : y}
> p = d

Yes, that's certainly in the plans.

> and then these "literal dict vs. Python dict" kind of thing might be
> even more confusing.

Point taken. I'm not too attached to this syntax--lets see if anyone  
else has feedback.

- Robert
  

From ondrej at certik.cz  Tue Oct 14 14:16:32 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 14 Oct 2008 14:16:32 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <48F37842.4070306@student.matnat.uio.no>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
	<48F34F40.7040705@student.matnat.uio.no>
	<85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>
	<48F37328.3030205@student.matnat.uio.no>
	<48F37842.4070306@student.matnat.uio.no>
Message-ID: <85b5c3130810140516h18756481t2ac8a9f56cdb6c84@mail.gmail.com>

On Mon, Oct 13, 2008 at 6:33 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Dag Sverre Seljebotn wrote:
>> Ondrej Certik wrote:
>>>
>>> cdef inline ndarray array_d(int size, double *data):
>>>     #cdef ndarray ary2 = PyArray_ZEROS(1, &size, 12, 0)
>>>     cdef ndarray ary = zeros(size, dtype=float64)
>>>     if data != NULL: memcpy(ary.data, data, size*sizeof(double))
>>>     return ary
>>>
>>> cdef inline int iarray_d(ndarray a, int *size, double **data) except -1:
>>>     if a.dtype != float64:
>>>         raise TypeError("The array must have the dtype=float64.")
>>>     if size!=NULL: size[0] = a.dimensions[0]
>>>     if data!=NULL: data[0] = <double *> (a.data)
>>>
>
>> HOWEVER, note that your iarray_d will only work if the array that is
>> passed in is contiguous. So you should either a) insert an assertion of
>> a.flags['C_CONTIGUOUS'] at the beginning, or b) fix it so that it works
>> with all arrays (as by my example).
>>
>> Any non-trivial NumPy usage is likely to end up with non-contiguous
>> arrays, so if the "a" argument could come from the end-user somehow then
>> choking on non-contiguous arrays is not going to work well.
>>
>>
>
> Note that this is non-trivial since any copy made must also not be
> deallocated, so that iarray_d would have to return a reference which
> must be held on to for the duration of using the buffer data.
>
> If you need to accept ndarrays coming from the user, a better idiom is
> likely to take a callback function which will be called with the buffer
> as an argument, or similar -- returning the buffer through a double**
> (or np.float64_t**) is rather error-prone when it comes to reference
> counting.

Yeah, I thought pretty much along the same lines.

Basically, I am fine if the function raises an exception on
noncontiguous arrays and/or copy the array automatically, so that's
not an issue.

The big problem is with the reference counting, i.e. who is
reposnsible for freeing the double** memory. Basically, the memory is
automatically freeed when the numpy array goes out of scope, right? So
basically, the rule is: either do something with the double** array
immediatelly (in C/C++) on the fly, or make a copy of it if you need
it longer. Another option would be to Py_IncRef/DecRef the numpy array
to make sure the C array is not recycled.


Ondrej

From dagss at student.matnat.uio.no  Tue Oct 14 14:28:56 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 14 Oct 2008 14:28:56 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <85b5c3130810140516h18756481t2ac8a9f56cdb6c84@mail.gmail.com>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>	<48F34F40.7040705@student.matnat.uio.no>	<85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>	<48F37328.3030205@student.matnat.uio.no>	<48F37842.4070306@student.matnat.uio.no>
	<85b5c3130810140516h18756481t2ac8a9f56cdb6c84@mail.gmail.com>
Message-ID: <48F49088.30709@student.matnat.uio.no>

Ondrej Certik wrote:
> On Mon, Oct 13, 2008 at 6:33 PM, Dag Sverre Seljebotn
>
>   
>> Note that this is non-trivial since any copy made must also not be
>> deallocated, so that iarray_d would have to return a reference which
>> must be held on to for the duration of using the buffer data.
>>
>> If you need to accept ndarrays coming from the user, a better idiom is
>> likely to take a callback function which will be called with the buffer
>> as an argument, or similar -- returning the buffer through a double**
>> (or np.float64_t**) is rather error-prone when it comes to reference
>> counting.
>>     
>
> Yeah, I thought pretty much along the same lines.
>
> Basically, I am fine if the function raises an exception on
> noncontiguous arrays and/or copy the array automatically, so that's
> not an issue.
>
> The big problem is with the reference counting, i.e. who is
> reposnsible for freeing the double** memory. Basically, the memory is
> automatically freeed when the numpy array goes out of scope, right? So
> basically, the rule is: either do something with the double** array
> immediatelly (in C/C++) on the fly, or make a copy of it if you need
> it longer. Another option would be to Py_IncRef/DecRef the numpy array
> to make sure the C array is not recycled.
>   
Pretty much. To hold a reference, simply assign it to a Cython "object" 
variable, don't bother manually reference counting it.

I think the general rule should be "keep the data in the numpy array for 
as long as possible". I must admit that I cannot imagine many situations 
where it would be good to store the double* for future reference rather 
than the entire Python array object. They're just pointers, it's not 
like it is "heavier" to store the ndarray :-)

I suppose one case is if you have a stateful C library like this:

clib.set_data(mydata)
clib.perform(...)
clib.release_data()

and you want to create a 1:1 wrapper in Cython. Yes, in that case you 
need to mirror the state info on the Cython-side and store a reference 
to the ndarray in the wrapper as well as feeding the pointer to C.

Dag Sverre

From ondrej at certik.cz  Tue Oct 14 14:44:51 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 14 Oct 2008 14:44:51 +0200
Subject: [Cython] canonical way of converting numpy arrays to C arrays
In-Reply-To: <48F49088.30709@student.matnat.uio.no>
References: <85b5c3130810130610j57bcdcd3r6adaa30b513b27cb@mail.gmail.com>
	<48F34F40.7040705@student.matnat.uio.no>
	<85b5c3130810130832k25d410ddl88a228d378b78e0@mail.gmail.com>
	<48F37328.3030205@student.matnat.uio.no>
	<48F37842.4070306@student.matnat.uio.no>
	<85b5c3130810140516h18756481t2ac8a9f56cdb6c84@mail.gmail.com>
	<48F49088.30709@student.matnat.uio.no>
Message-ID: <85b5c3130810140544h3888c2b3oe788d6b21e0d4fc0@mail.gmail.com>

On Tue, Oct 14, 2008 at 2:28 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Ondrej Certik wrote:
>> On Mon, Oct 13, 2008 at 6:33 PM, Dag Sverre Seljebotn
>>
>>
>>> Note that this is non-trivial since any copy made must also not be
>>> deallocated, so that iarray_d would have to return a reference which
>>> must be held on to for the duration of using the buffer data.
>>>
>>> If you need to accept ndarrays coming from the user, a better idiom is
>>> likely to take a callback function which will be called with the buffer
>>> as an argument, or similar -- returning the buffer through a double**
>>> (or np.float64_t**) is rather error-prone when it comes to reference
>>> counting.
>>>
>>
>> Yeah, I thought pretty much along the same lines.
>>
>> Basically, I am fine if the function raises an exception on
>> noncontiguous arrays and/or copy the array automatically, so that's
>> not an issue.
>>
>> The big problem is with the reference counting, i.e. who is
>> reposnsible for freeing the double** memory. Basically, the memory is
>> automatically freeed when the numpy array goes out of scope, right? So
>> basically, the rule is: either do something with the double** array
>> immediatelly (in C/C++) on the fly, or make a copy of it if you need
>> it longer. Another option would be to Py_IncRef/DecRef the numpy array
>> to make sure the C array is not recycled.
>>
> Pretty much. To hold a reference, simply assign it to a Cython "object"
> variable, don't bother manually reference counting it.

Nice trick, didn't occur to me.

>
> I think the general rule should be "keep the data in the numpy array for
> as long as possible". I must admit that I cannot imagine many situations
> where it would be good to store the double* for future reference rather
> than the entire Python array object. They're just pointers, it's not
> like it is "heavier" to store the ndarray :-)
>
> I suppose one case is if you have a stateful C library like this:
>
> clib.set_data(mydata)
> clib.perform(...)
> clib.release_data()
>
> and you want to create a 1:1 wrapper in Cython. Yes, in that case you
> need to mirror the state info on the Cython-side and store a reference
> to the ndarray in the wrapper as well as feeding the pointer to C.

This is exactly my case. I am writting a C++ code that operates over
double* arrays and I have 1:1 Cython wrapper for the classes. But a
nice trick is to hold the reference to the input array in the Cython
object in the Cython class --- that could do the job.

Ondrej

From aaron.devore at gmail.com  Tue Oct 14 22:06:03 2008
From: aaron.devore at gmail.com (Aaron DeVore)
Date: Tue, 14 Oct 2008 13:06:03 -0700
Subject: [Cython] Literal struct syntax
In-Reply-To: <E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
Message-ID: <2ead2fb0810141306v1ef83ff7ra71490ab5af1ea02@mail.gmail.com>

Time for my two cents. When I first saw the {color:color, x:x} syntax
it did seem somewhat jarring. After I thought about it a bit more it
makes more sense. Its advantages over a "string": value syntax:

1. It gives a clear visual distinction between a dict and a struct.
2. dict values are accessed via keys that are often strings (d["k"] ->
v) whereas structs are always referred to like attributes (p.name)
3. Having what appears to be string keys doesn't reflect that the keys
are not, in fact, strings.
4. Some other languages use the same syntax (e.g. objects in JavaScript)
5. It's clear that users are restricted to Python's identifier syntax.

The disadvantage:
1. In {color:color, x:x, y:y} the 'keys' look like identifiers.

I haven't read back through the archives to catch this entire
discussion but I have a suggestion. How about having a character
before the braces to visually indicate that it is a struct? Maybe
something like s{color: color}. That would give a clear visual cue,
not introduce a significant syntax change, and be similar to 1L, u"",
and r"".

Alternatively, the struct member names could be surrounded with a
character that isn't a quote. Just off the top of my head:
- A period/dot like {.color:color}. Pros: matches to the idea of
attribute/member access, fewer characters to type. Cons: Periods
sometimes aren't very visible at the beginning of a word, still looks
somewhat like a variable.
- {|color|: color}. Pros: distinct syntax, Cons: The { and | visually
blend together. The | looks too much like an I in some fonts. Doesn't
match up to any preexisting syntax concepts.
- {/color/: color}. Pros/Cons: Same as pipes, looks like regexp syntax
in Perl and other languages.
- {`color`: color}: Pros: Vaguely looks like a string literal, but
still is distinct. Very visible in email. Cons: Introduces new syntax.



-Aaron

From greg.ewing at canterbury.ac.nz  Wed Oct 15 01:09:50 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 15 Oct 2008 12:09:50 +1300
Subject: [Cython] Literal struct syntax
In-Reply-To: <E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
Message-ID: <48F526BE.7040107@canterbury.ac.nz>

Robert Bradshaw wrote:

> On Oct 13, 2008, at 1:44 PM, Dag Sverre Seljebotn wrote:
> 
>>cdef Point p = {color: color, x: x, y: y}
>>
>>That is, without string keys, but rather some magic keys.
> 
> As a (weak) counter argument, I found myself forgetting the quotes
> when testing, and it's not much more magic than the magic string kwds  
> arguments keys.

But it's inconsistent, because it suggests evaluating
an expression to get the key, as in a dict constructor.

A compromise might be

   cdef Point p = {color = color, x = x, y = y}

-- 
Greg

From stefan_ml at behnel.de  Wed Oct 15 08:44:05 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Oct 2008 08:44:05 +0200 (CEST)
Subject: [Cython] Literal struct syntax
In-Reply-To: <48F526BE.7040107@canterbury.ac.nz>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<48F526BE.7040107@canterbury.ac.nz>
Message-ID: <44461.213.61.181.86.1224053045.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Greg Ewing wrote:
> A compromise might be
>
>    cdef Point p = {color = color, x = x, y = y}

Yes, keywords were my first thought as well. They heavily reduce the
magic. I just wasn't sure if {} is the syntax of choice here, or if
(colour=5, x=y) would match my intuition better. It matches the new
constructor after all.

Another thought I had was that this should somehow match named tuples (new
in Py2.6), but there isn't a literal syntax for them, so this won't help
much. That makes keywords the next best choice, IMHO.

http://code.activestate.com/recipes/500261/
http://docs.python.org/library/collections.html#collections.namedtuple

Stefan


From stefan_ml at behnel.de  Wed Oct 15 09:03:24 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 15 Oct 2008 09:03:24 +0200 (CEST)
Subject: [Cython] Literal struct syntax
In-Reply-To: <E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
Message-ID: <32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Robert Bradshaw wrote:
> On Oct 13, 2008, at 1:44 PM, Dag Sverre Seljebotn wrote:
>> For more brevity without typing all the " there's always the
>> Point(color=...), syntax

Which, as I only noticed now, actually matches the way named tuples are used.


>> and I feel being consistent with keeping
>> it a normal dict is important because a later feature might be
>>
>> cdef Point p
>> d = {"color": color, "x" : x, "y" : y}
>> p = d
>
> Yes, that's certainly in the plans.

That certainly matches the opposite conversion way, which, as I see in the
test, you already implemented.

Given this goal, I wonder why the literals are needed in the first place.
Currently, you can just do this:

    p = Point(color=color, x=y)

and once dict unpacking is implemented, you can also do

    p = dict(color=color, x=y)

or

    p = {"color" : color, "x" : y}

where the dict creation could even be optimised away, at least in the
trivial case.

Adding another literal syntax to this, only for the sake of removing one
little name before the parentheses, seems unnecessary to me.

Apart from this little detail, I'm very happy with the general concepts
and I really like what you implemented here. This totally makes sense in
the context of Cython.

Stefan


From dalcinl at gmail.com  Wed Oct 15 18:24:47 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 13:24:47 -0300
Subject: [Cython] Dag, please review this patch
Message-ID: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>

Dag, I was working on a fix for tiket #90
<http://trac.cython.org/cython_trac/ticket/90>, but your latest commit
forced me to rework my patch. Could you please review the attached
patch (you also have it in trac!!)?

 All the tests pass with this patch, but perhaps I'm missing
something... Thanks in advance.

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fixnoneleaks.diff
Type: text/x-patch
Size: 854 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081015/41cfb709/attachment.bin 

From dagss at student.matnat.uio.no  Wed Oct 15 19:04:05 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Oct 2008 19:04:05 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
Message-ID: <48F62285.80309@student.matnat.uio.no>

Lisandro Dalcin wrote:
> Dag, I was working on a fix for tiket #90
> <http://trac.cython.org/cython_trac/ticket/90>, but your latest commit
> forced me to rework my patch. Could you please review the attached
> patch (you also have it in trac!!)?
> 
>  All the tests pass with this patch, but perhaps I'm missing
> something... Thanks in advance.

See <http://trac.cython.org/cython_trac/ticket/90>

"""
The patch should work -- however it seems a bit fragile.

I'm attaching an alternative version, which rather modifies how the 
initalization happens. This should also be very slightly faster.
"""

I don't dare to apply my patch though, a bit unsure if it could break 
something, so somebody please review my patch :-)


-- 
Dag Sverre

From dalcinl at gmail.com  Wed Oct 15 19:14:11 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 14:14:11 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F62285.80309@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<48F62285.80309@student.matnat.uio.no>
Message-ID: <e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>

Well, what I do not like about your patch is that the cdef module
globals are not initialized to 'None' in the module init function. And
this is a bit unsafe, Cython almost always assumes that PyObect*
pointers are not-NULL. I can imagine some code breaking with your
patch.

On Wed, Oct 15, 2008 at 2:04 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Lisandro Dalcin wrote:
>> Dag, I was working on a fix for tiket #90
>> <http://trac.cython.org/cython_trac/ticket/90>, but your latest commit
>> forced me to rework my patch. Could you please review the attached
>> patch (you also have it in trac!!)?
>>
>>  All the tests pass with this patch, but perhaps I'm missing
>> something... Thanks in advance.
>
> See <http://trac.cython.org/cython_trac/ticket/90>
>
> """
> The patch should work -- however it seems a bit fragile.
>
> I'm attaching an alternative version, which rather modifies how the
> initalization happens. This should also be very slightly faster.
> """
>
> I don't dare to apply my patch though, a bit unsure if it could break
> something, so somebody please review my patch :-)
>
>
> --
> Dag Sverre
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Wed Oct 15 19:56:18 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Oct 2008 19:56:18 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
Message-ID: <48F62EC2.6080907@student.matnat.uio.no>

Lisandro Dalcin wrote:
> Well, what I do not like about your patch is that the cdef module
> globals are not initialized to 'None' in the module init function. And
> this is a bit unsafe, Cython almost always assumes that PyObect*
> pointers are not-NULL. I can imagine some code breaking with your
> patch.
> 

Indeed, it is unsafe. Very much so, I just created a crashing testcase :-)

gtg, brb


-- 
Dag Sverre

From dagss at student.matnat.uio.no  Wed Oct 15 20:13:25 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Oct 2008 20:13:25 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F62EC2.6080907@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
Message-ID: <48F632C5.6080706@student.matnat.uio.no>

Dag Sverre Seljebotn wrote:
> Lisandro Dalcin wrote:
>> Well, what I do not like about your patch is that the cdef module
>> globals are not initialized to 'None' in the module init function. And
>> this is a bit unsafe, Cython almost always assumes that PyObect*
>> pointers are not-NULL. I can imagine some code breaking with your
>> patch.
>>
> 
> Indeed, it is unsafe. Very much so, I just created a crashing testcase :-)

OK, my last attempt is up as a new patch on the ticket:

<http://trac.cython.org/cython_trac/ticket/90>

It works in this way: Assignment nodes which are created from cdef 
statements in the module scope doesn't get the "first" attribute set to 
True (as it is not guaranteed that the cdef statement is indeed the 
first assignment, indeed they are implicitly always set to None first).

I now realize what made my worry about your patch: It works ok, but it 
corrects something which was set wrongly in the first place. This should 
fix the root cause instead.

-- 
Dag Sverre

From dalcinl at gmail.com  Wed Oct 15 20:33:51 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 15:33:51 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F632C5.6080706@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<48F62285.80309@student.matnat.uio.no>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
Message-ID: <e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>

This look nicer. However, there is still something that does not work.
Try to compile the following:

cdef object foo1
cdef object foo2 = []

and then you will see that the declaration of  the (unused) 'foo1' is
not ever emited in the module declaration part. Some optimization
should be playing games. Or perhaps 'foo1' should be completely
ignored and never declared/initialized/cleaned-up?

On Wed, Oct 15, 2008 at 3:13 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Dag Sverre Seljebotn wrote:
>> Lisandro Dalcin wrote:
>>> Well, what I do not like about your patch is that the cdef module
>>> globals are not initialized to 'None' in the module init function. And
>>> this is a bit unsafe, Cython almost always assumes that PyObect*
>>> pointers are not-NULL. I can imagine some code breaking with your
>>> patch.
>>>
>>
>> Indeed, it is unsafe. Very much so, I just created a crashing testcase :-)
>
> OK, my last attempt is up as a new patch on the ticket:
>
> <http://trac.cython.org/cython_trac/ticket/90>
>
> It works in this way: Assignment nodes which are created from cdef
> statements in the module scope doesn't get the "first" attribute set to
> True (as it is not guaranteed that the cdef statement is indeed the
> first assignment, indeed they are implicitly always set to None first).
>
> I now realize what made my worry about your patch: It works ok, but it
> corrects something which was set wrongly in the first place. This should
> fix the root cause instead.
>
> --
> Dag Sverre
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Wed Oct 15 22:05:14 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Oct 2008 22:05:14 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>	<48F62EC2.6080907@student.matnat.uio.no>	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
Message-ID: <48F64CFA.8030100@student.matnat.uio.no>

Lisandro Dalcin wrote:
> This look nicer. However, there is still something that does not work.
> Try to compile the following:
> 
> cdef object foo1
> cdef object foo2 = []
> 
> and then you will see that the declaration of  the (unused) 'foo1' is
> not ever emited in the module declaration part. Some optimization
> should be playing games. Or perhaps 'foo1' should be completely
> ignored and never declared/initialized/cleaned-up?
> 

I believe this is an unrelated and long-standing bug?

Anyway, the following patch should fix it (unless we want the opposite 
behaviour).

Will you commit it together with your testcase? (I.e. try to print 
"foo1" from a doctest, which is only in a string and so will trigger 
this case, and it should be "None"...)

-- 
Dag Sverre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: always_declare_module_objects2.diff
Type: text/x-diff
Size: 621 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081015/d9ae32b0/attachment-0001.bin 

From dalcinl at gmail.com  Wed Oct 15 22:10:22 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 17:10:22 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F64CFA.8030100@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<48F62285.80309@student.matnat.uio.no>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
Message-ID: <e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>

On Wed, Oct 15, 2008 at 5:05 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Lisandro Dalcin wrote:
>>
>> This look nicer. However, there is still something that does not work.
>> Try to compile the following:
>>
>> cdef object foo1
>> cdef object foo2 = []
>>
>> and then you will see that the declaration of  the (unused) 'foo1' is
>> not ever emited in the module declaration part. Some optimization
>> should be playing games. Or perhaps 'foo1' should be completely
>> ignored and never declared/initialized/cleaned-up?
>>
>
> I believe this is an unrelated and long-standing bug?
> Anyway, the following patch should fix it (unless we want the opposite
> behaviour).

OK, I'll give a try, but perhaps tomorrow.

>
> Will you commit it together with your testcase? (I.e. try to print "foo1"
> from a doctest, which is only in a string and so will trigger this case, and
> it should be "None"...)

Well, I'll try to make a good test case for all this. But the 'print
foo1' will not work, as 'foo1' is not exported in the module dict, as
it is cdef stuff (print foo2 in a doctest docstring will not work
either).


> --
> Dag Sverre
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dagss at student.matnat.uio.no  Wed Oct 15 22:31:10 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Wed, 15 Oct 2008 22:31:10 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>	<48F62EC2.6080907@student.matnat.uio.no>	<48F632C5.6080706@student.matnat.uio.no>	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
Message-ID: <48F6530E.5080401@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On Wed, Oct 15, 2008 at 5:05 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Lisandro Dalcin wrote:
>>> This look nicer. However, there is still something that does not work.
>>> Try to compile the following:
>>>
>>> cdef object foo1
>>> cdef object foo2 = []
>>>
>>> and then you will see that the declaration of  the (unused) 'foo1' is
>>> not ever emited in the module declaration part. Some optimization
>>> should be playing games. Or perhaps 'foo1' should be completely
>>> ignored and never declared/initialized/cleaned-up?
>>>
>> I believe this is an unrelated and long-standing bug?
>> Anyway, the following patch should fix it (unless we want the opposite
>> behaviour).
> 
> OK, I'll give a try, but perhaps tomorrow.
> 
>> Will you commit it together with your testcase? (I.e. try to print "foo1"
>> from a doctest, which is only in a string and so will trigger this case, and
>> it should be "None"...)
> 
> Well, I'll try to make a good test case for all this. But the 'print
> foo1' will not work, as 'foo1' is not exported in the module dict, as
> it is cdef stuff (print foo2 in a doctest docstring will not work
> either).
> 

Ah right. Well my patch is a bad one then; it is the initialization to 
Py_None which should be removed instead.

Well, you seem to have a better grasp on this stuff than me so I'll 
leave it for you :-) You are right, it is optimization stuff going wrong 
-- "entry.used" isn't honoured by the code in ModuleNode.py initializing 
the entries to Py_None. Just search for the comments inserted in the 
.c-file at that point in ModuleNode.py and it will put you right on 
track for where to insert the if-test.

As for the tests, I suppose all of this is really difficult to test for 
(as possibly the doctests would interfer a lot with the refcount of 
Py_None and so on), so actually I'd agree with you if you didn't bother 
with them.

-- 
Dag Sverre

From dalcinl at gmail.com  Wed Oct 15 23:39:55 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 18:39:55 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F6530E.5080401@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<48F62285.80309@student.matnat.uio.no>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
Message-ID: <e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>

On Wed, Oct 15, 2008 at 5:31 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Well, you seem to have a better grasp on this stuff than me so I'll
> leave it for you :-) You are right, it is optimization stuff going wrong
> -- "entry.used" isn't honoured by the code in ModuleNode.py initializing
> the entries to Py_None. Just search for the comments inserted in the
> .c-file at that point in ModuleNode.py and it will put you right on
> track for where to insert the if-test.

I believe I will push my original patch with a BIG comment about what
is going one in Optimize.py

> As for the tests, I suppose all of this is really difficult to test for
> (as possibly the doctests would interfer a lot with the refcount of
> Py_None and so on), so actually I'd agree with you if you didn't bother
> with them.

BTW, perhaps if we could support the --clenaup option as a compiler
directive, then I would be able to write a rether good testcase.

Do this request make sense, I mean, adding a 'cleanup' directive
enabling the highest level of cleanup? I ALWAY build my code with
cleanup enabled, so I would love to be hable to use in my preambles::
#cython: cleanup=True


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Thu Oct 16 01:34:07 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 16 Oct 2008 12:34:07 +1300
Subject: [Cython] Literal struct syntax
In-Reply-To: <32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
Message-ID: <48F67DEF.9030800@canterbury.ac.nz>

Stefan Behnel wrote:

> Given this goal, I wonder why the literals are needed in the first place.
> Currently, you can just do this:
> 
>     p = Point(color=color, x=y)

I thought about that too, but something I had in mind
for Pyrex in the future was that

   p = Point()

where Point is a struct type would be shorthand for

   p = <Point *>malloc(sizeof(Point))

so I think I'd prefer to reserve the constructor
syntax for that.

-- 
Greg

From dalcinl at gmail.com  Thu Oct 16 03:31:11 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Wed, 15 Oct 2008 22:31:11 -0300
Subject: [Cython] Literal struct syntax
In-Reply-To: <48F67DEF.9030800@canterbury.ac.nz>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
Message-ID: <e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>

On Wed, Oct 15, 2008 at 8:34 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> I thought about that too, but something I had in mind
> for Pyrex in the future was that
>
>   p = Point()
>
> where Point is a struct type would be shorthand for
>
>   p = <Point *>malloc(sizeof(Point))
>
> so I think I'd prefer to reserve the constructor
> syntax for that.

And how are you going to free the memory? Are you thinking about C++
semantics for stack-allocated class instances with
user-defined/Cython-syntetized constructors/destructors?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Thu Oct 16 05:43:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 16 Oct 2008 16:43:54 +1300
Subject: [Cython] Literal struct syntax
In-Reply-To: <e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
	<e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>
Message-ID: <48F6B87A.7040903@canterbury.ac.nz>

Lisandro Dalcin wrote:

> And how are you going to free the memory?

I'm not, that's up to the programmer. It's just a
more convenient way of writing a malloc.

-- 
Greg

From dagss at student.matnat.uio.no  Thu Oct 16 10:00:52 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 10:00:52 +0200
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>	<48F62EC2.6080907@student.matnat.uio.no>	<48F632C5.6080706@student.matnat.uio.no>	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>	<48F64CFA.8030100@student.matnat.uio.no>	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
Message-ID: <48F6F4B4.1020602@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On Wed, Oct 15, 2008 at 5:31 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Well, you seem to have a better grasp on this stuff than me so I'll
>> leave it for you :-) You are right, it is optimization stuff going wrong
>> -- "entry.used" isn't honoured by the code in ModuleNode.py initializing
>> the entries to Py_None. Just search for the comments inserted in the
>> .c-file at that point in ModuleNode.py and it will put you right on
>> track for where to insert the if-test.
> 
> I believe I will push my original patch with a BIG comment about what
> is going one in Optimize.py

Huh? Your patch doesn't solve this problem either? (I just 
tried...fixnoneleaks.diff does nothing about the

cdef object foo1

not being declared issue).

The two should be orthogonal issues. I really believe that my patch is 
how the first problem should be solved. Attaching now how I believe the 
second problem should be solved, for which there seems to be no patch 
yet anyway.

If you agree I can simply make a push of my patches BTW, but there's 
been holes in my patches before so I'm waiting. If you find a way you 
think is better then feel free.

> BTW, perhaps if we could support the --clenaup option as a compiler
> directive, then I would be able to write a rether good testcase.
> 
> Do this request make sense, I mean, adding a 'cleanup' directive
> enabling the highest level of cleanup? I ALWAY build my code with
> cleanup enabled, so I would love to be hable to use in my preambles::
> #cython: cleanup=True

+1

But one should block its use as a decorator and with statement. Perhaps 
a more descriptive name -- module_cleanup or similar?

-- 
Dag Sverre
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix_issue2.diff
Type: text/x-diff
Size: 584 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081016/0392f6e9/attachment.bin 

From dalcinl at gmail.com  Thu Oct 16 16:02:46 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 11:02:46 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F6F4B4.1020602@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
Message-ID: <e7ba66e40810160702k60445d37n98a2eba3cc90c3a3@mail.gmail.com>

On Thu, Oct 16, 2008 at 5:00 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
>
> Huh? Your patch doesn't solve this problem either? (I just
> tried...fixnoneleaks.diff does nothing about the
>
> cdef object foo1
>
> not being declared issue).

Ups! I wrote that at home last night, I believe it worked. But perhaps
I got confused.

> The two should be orthogonal issues. I really believe that my patch is how
> the first problem should be solved. Attaching now how I believe the second
> problem should be solved, for which there seems to be no patch yet anyway.

OK, I'll give a look right now.

> If you agree I can simply make a push of my patches BTW, but there's been
> holes in my patches before so I'm waiting. If you find a way you think is
> better then feel free.

You are far more experienced with Cython internals than me. So if you
said you way is the way, then I trust you. Just let me play a bit with
the corner cases.

>> BTW, perhaps if we could support the --clenaup option as a compiler
>> directive, then I would be able to write a rether good testcase.
>>
>> Do this request make sense, I mean, adding a 'cleanup' directive
>> enabling the highest level of cleanup? I ALWAY build my code with
>> cleanup enabled, so I would love to be hable to use in my preambles::
>> #cython: cleanup=True
>
> +1
>
> But one should block its use as a decorator and with statement. Perhaps a
> more descriptive name -- module_cleanup or similar?

Well, I would ask for 'modulecleanup' for consistency with the naming
of the other directives.  But I do not know yet how to disable
decorator and with statement. BTW, should this directive be an
integer, supporting various levels of cleanups, like the former
--cleanup option? Or it should mean a full cleanup?



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From dalcinl at gmail.com  Thu Oct 16 16:20:02 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 11:20:02 -0300
Subject: [Cython] Literal struct syntax
In-Reply-To: <48F6B87A.7040903@canterbury.ac.nz>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
	<e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>
	<48F6B87A.7040903@canterbury.ac.nz>
Message-ID: <e7ba66e40810160720t1636c614m82c469ed7766be3b@mail.gmail.com>

On Thu, Oct 16, 2008 at 12:43 AM, Greg Ewing
<greg.ewing at canterbury.ac.nz> wrote:
> Lisandro Dalcin wrote:
>
>> And how are you going to free the memory?
>
> I'm not, that's up to the programmer. It's just a
> more convenient way of writing a malloc.

Then I have to say that I'm -1 in such magic. What about introducing
two new pseudo-keywords cnew/cdel, and use it like this (like in C++):

MyStruct *p = cnew MyStruct(a=1,b=2)
# use p
cdel p # also sets p to NULL
assert p==NULL
cdel p # success, but do nothing?

Such approach would make Cython/Pyrex closer to C++ behavior.
Moreover, this could be extended to support funny features, like
letting user choose the alloc/dealloc routines and perhaps
constructor/destructor routines.

What do you think?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From robertwb at math.washington.edu  Thu Oct 16 16:27:04 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 07:27:04 -0700
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F6F4B4.1020602@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>	<48F62285.80309@student.matnat.uio.no>	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>	<48F62EC2.6080907@student.matnat.uio.no>	<48F632C5.6080706@student.matnat.uio.no>	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>	<48F64CFA.8030100@student.matnat.uio.no>	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
Message-ID: <3FD8A375-1691-40D7-A375-2C62C18D53BE@math.washington.edu>


On Oct 16, 2008, at 1:00 AM, Dag Sverre Seljebotn wrote:

> Lisandro Dalcin wrote:
>> On Wed, Oct 15, 2008 at 5:31 PM, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>> Well, you seem to have a better grasp on this stuff than me so I'll
>>> leave it for you :-) You are right, it is optimization stuff  
>>> going wrong
>>> -- "entry.used" isn't honoured by the code in ModuleNode.py  
>>> initializing
>>> the entries to Py_None. Just search for the comments inserted in the
>>> .c-file at that point in ModuleNode.py and it will put you right on
>>> track for where to insert the if-test.
>> I believe I will push my original patch with a BIG comment about what
>> is going one in Optimize.py
>
> Huh? Your patch doesn't solve this problem either? (I just  
> tried...fixnoneleaks.diff does nothing about the
>
> cdef object foo1
>
> not being declared issue).
>
> The two should be orthogonal issues. I really believe that my patch  
> is how the first problem should be solved. Attaching now how I  
> believe the second problem should be solved, for which there seems  
> to be no patch yet anyway.

If foo1 is never used, it should never be delcared. This avoids a C  
warning in this case, and doesn't impact the operation of the module.

>
> If you agree I can simply make a push of my patches BTW, but  
> there's been holes in my patches before so I'm waiting. If you find  
> a way you think is better then feel free.
>
>> BTW, perhaps if we could support the --clenaup option as a compiler
>> directive, then I would be able to write a rether good testcase.
>> Do this request make sense, I mean, adding a 'cleanup' directive
>> enabling the highest level of cleanup? I ALWAY build my code with
>> cleanup enabled, so I would love to be hable to use in my preambles::
>> #cython: cleanup=True
>
> +1
>
> But one should block its use as a decorator and with statement.  
> Perhaps a more descriptive name -- module_cleanup or similar?
>
> -- 
> Dag Sverre
> diff -r 2dbb7b8520a5 Cython/Compiler/ModuleNode.py
> --- a/Cython/Compiler/ModuleNode.py	Wed Oct 15 21:48:31 2008 +0200
> +++ b/Cython/Compiler/ModuleNode.py	Thu Oct 16 09:56:51 2008 +0200
> @@ -1734,7 +1734,7 @@ class ModuleNode(Nodes.Node, Nodes.Block
>          # variables to None.
>          for entry in env.var_entries:
>              if entry.visibility != 'extern':
> -                if entry.type.is_pyobject:
> +                if entry.type.is_pyobject and entry.used:
>                      code.put_init_var_to_py_none(entry)
>
>      def generate_c_function_export_code(self, env, code):
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Thu Oct 16 16:27:51 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 07:27:51 -0700
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810160702k60445d37n98a2eba3cc90c3a3@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
	<e7ba66e40810160702k60445d37n98a2eba3cc90c3a3@mail.gmail.com>
Message-ID: <8942D37A-E521-4515-8F11-8F8BC2BDC02D@math.washington.edu>


On Oct 16, 2008, at 7:02 AM, Lisandro Dalcin wrote:

> On Thu, Oct 16, 2008 at 5:00 AM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>>
>> Huh? Your patch doesn't solve this problem either? (I just
>> tried...fixnoneleaks.diff does nothing about the
>>
>> cdef object foo1
>>
>> not being declared issue).
>
> Ups! I wrote that at home last night, I believe it worked. But perhaps
> I got confused.
>
>> The two should be orthogonal issues. I really believe that my  
>> patch is how
>> the first problem should be solved. Attaching now how I believe  
>> the second
>> problem should be solved, for which there seems to be no patch yet  
>> anyway.
>
> OK, I'll give a look right now.
>
>> If you agree I can simply make a push of my patches BTW, but  
>> there's been
>> holes in my patches before so I'm waiting. If you find a way you  
>> think is
>> better then feel free.
>
> You are far more experienced with Cython internals than me. So if you
> said you way is the way, then I trust you. Just let me play a bit with
> the corner cases.
>
>>> BTW, perhaps if we could support the --clenaup option as a compiler
>>> directive, then I would be able to write a rether good testcase.
>>>
>>> Do this request make sense, I mean, adding a 'cleanup' directive
>>> enabling the highest level of cleanup? I ALWAY build my code with
>>> cleanup enabled, so I would love to be hable to use in my  
>>> preambles::
>>> #cython: cleanup=True
>>
>> +1
>>
>> But one should block its use as a decorator and with statement.  
>> Perhaps a
>> more descriptive name -- module_cleanup or similar?
>
> Well, I would ask for 'modulecleanup' for consistency with the naming
> of the other directives.  But I do not know yet how to disable
> decorator and with statement. BTW, should this directive be an
> integer, supporting various levels of cleanups, like the former
> --cleanup option? Or it should mean a full cleanup?

More descriptive is better. I'd prefer cleanupmodule, and it should  
be an integer.

- Robert




From stefan_ml at behnel.de  Thu Oct 16 16:42:21 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 16 Oct 2008 16:42:21 +0200 (CEST)
Subject: [Cython] Literal struct syntax
In-Reply-To: <48F67DEF.9030800@canterbury.ac.nz>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
Message-ID: <55661.213.61.181.86.1224168141.squirrel@groupware.dvs.informatik.tu-darmstadt.de>

Greg Ewing wrote:
> Stefan Behnel wrote:
>
>> Given this goal, I wonder why the literals are needed in the first
>> place.
>> Currently, you can just do this:
>>
>>     p = Point(color=color, x=y)
>
> I thought about that too, but something I had in mind
> for Pyrex in the future was that
>
>    p = Point()
>
> where Point is a struct type would be shorthand for
>
>    p = <Point *>malloc(sizeof(Point))

IMHO, if we hide the malloc(), we should hide the free(), just as we do
(or Python does) everywhere else. If we can't, it's better to make the
malloc() explicit, too.

Stefan


From robertwb at math.washington.edu  Thu Oct 16 17:33:24 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 08:33:24 -0700
Subject: [Cython] Literal struct syntax
In-Reply-To: <e7ba66e40810160720t1636c614m82c469ed7766be3b@mail.gmail.com>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
	<e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>
	<48F6B87A.7040903@canterbury.ac.nz>
	<e7ba66e40810160720t1636c614m82c469ed7766be3b@mail.gmail.com>
Message-ID: <4F4749BD-D22F-4F6E-93DA-CF45557B15C2@math.washington.edu>

On Oct 16, 2008, at 7:20 AM, Lisandro Dalcin wrote:

> On Thu, Oct 16, 2008 at 12:43 AM, Greg Ewing
> <greg.ewing at canterbury.ac.nz> wrote:
>> Lisandro Dalcin wrote:
>>
>>> And how are you going to free the memory?
>>
>> I'm not, that's up to the programmer. It's just a
>> more convenient way of writing a malloc.
>
> Then I have to say that I'm -1 in such magic. What about introducing
> two new pseudo-keywords cnew/cdel, and use it like this (like in C++):
>
> MyStruct *p = cnew MyStruct(a=1,b=2)
> # use p
> cdel p # also sets p to NULL
> assert p==NULL
> cdel p # success, but do nothing?
>
> Such approach would make Cython/Pyrex closer to C++ behavior.
> Moreover, this could be extended to support funny features, like
> letting user choose the alloc/dealloc routines and perhaps
> constructor/destructor routines.
>
> What do you think?

First, thanks for all the feedback, and thanks Dag for bringing this up.

I'd rather focus on making Cython closer to the Python language than  
introducing new concepts. Also, I'm with Stefan that it is very  
dangerous to hide the malloc and expect the user to explicitly  
provide the free. If the user wants to manage their own memory, they  
can do so with malloc and friends (understanding the associated  
dangers), and any special memory-management stuff we add should be as  
implicit and easy to use as Python's garbage collection (and probably  
piggyback off of it).

I think the MyStruct(...) should result in an actual struct (not a  
pointer), but perhaps on conversion to an object it could be made  
into a memory managed version of the struct (e.g. using the same  
string allocation trick we talked about for arrays) giving C access,  
pass-by-reference, and memory management. This, however, is not going  
to happen in the next release of Cython and when it does it could be  
made backwards compatible with the current conversion to dict.  
(Syntax suggestions for how to declare such an object welcome,  
perhaps "cdef object MyStruct foo" and memory-managed arrays could be  
"cdef object int[] foo")

As for the current proposal, don't want to introduce new, special  
"struct instanciation dictionaries," so I'd opt for allowing the  
{"a": 1, "b": 2} literals with the plan of conversion from a runtime  
dict down the road (including runtime-created keys), but disabling  
the {a: 1, b: 2} as the MyStruct(a=1, b=2) is available (and, even  
better, more explicit).

- Robert


From dalcinl at gmail.com  Thu Oct 16 17:36:17 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 12:36:17 -0300
Subject: [Cython] removing GCC warnings for test suite, please review [part1]
Message-ID: <e7ba66e40810160836u3f3f7357ge317ae6deab83054@mail.gmail.com>

How often do you scan the full output of 'python runtests.py' in order
to look for suspicious C-code generation? I guess never ;-), as Cython
test suite generate many warning with GCC.

As this situation make me feel very uncomfortable, you have here for
review a patch touching many test cases for removing those nasty GCC
warnings. If there are no major objections, I would like to push this
ASAP.

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tests.diff
Type: text/x-patch
Size: 14832 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081016/24386980/attachment-0001.bin 

From dalcinl at gmail.com  Thu Oct 16 17:38:51 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 12:38:51 -0300
Subject: [Cython] removing GCC warnings for test suite, please review [part2]
Message-ID: <e7ba66e40810160838t548ab8e3p9559a34ea7cf2209@mail.gmail.com>

You can now see the gains. You have attached a 'runtests.log' file
with the full output of 'python runtest.py'. Please, look carefully at
the warnings. They are pointing real stuff: unused variables in buffer
syntax, docstrings that should never be emitted, and a 'statement with
no effect' in tests/run/pure.py (no time right now to dive the code in
order to see whats going on).

-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
-------------- next part --------------
A non-text attachment was scrubbed...
Name: runtests.log
Type: application/octet-stream
Size: 18136 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20081016/e48a0b11/attachment.obj 

From robertwb at math.washington.edu  Thu Oct 16 17:43:37 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 08:43:37 -0700
Subject: [Cython] removing GCC warnings for test suite,
	please review [part1]
In-Reply-To: <e7ba66e40810160836u3f3f7357ge317ae6deab83054@mail.gmail.com>
References: <e7ba66e40810160836u3f3f7357ge317ae6deab83054@mail.gmail.com>
Message-ID: <B21AE792-B42E-450B-BBDB-8E36CC5B5A23@math.washington.edu>

On Oct 16, 2008, at 8:36 AM, Lisandro Dalcin wrote:

> How often do you scan the full output of 'python runtests.py' in order
> to look for suspicious C-code generation? I guess never ;-), as Cython
> test suite generate many warning with GCC.

Sometimes I watch it as I go by, but I've gotten used to the unused  
variable errors by now...  (Maybe a better word would be  
desensitized. Not sure it's a good thing.)

> As this situation make me feel very uncomfortable, you have here for
> review a patch touching many test cases for removing those nasty GCC
> warnings. If there are no major objections, I would like to push this
> ASAP.

Yes, it looks good. Ideally valid Cython code would execute with no  
warnings at all (other than those in the users code, like bad  
declarations and using uninitalized data).

- Robert


From dalcinl at gmail.com  Thu Oct 16 18:37:45 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 13:37:45 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <48F6F4B4.1020602@student.matnat.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
Message-ID: <e7ba66e40810160937q2c008e4bn9cfa4103c466f425@mail.gmail.com>

On Thu, Oct 16, 2008 at 5:00 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> The two should be orthogonal issues. I really believe that my patch is how
> the first problem should be solved. Attaching now how I believe the second
> problem should be solved, for which there seems to be no patch yet anyway.

I've attached a new patch trying to fix all the issues following your
comments. And you are tight, this patch seems to be far better than my
previous ones.

http://trac.cython.org/cython_trac/attachment/ticket/90/GLOBALS.diff

I could add a testcase, but a really good one would need the
'modulecleanup' directive implemented, as I need to play with the
Py_AtExit() Python C-API call in order to detect that the generated
module cleanup is actually working.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From ondrej at certik.cz  Thu Oct 16 18:39:44 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 18:39:44 +0200
Subject: [Cython] docs: cython special methods
Message-ID: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>

Hi,

here is a stub:

http://docs.cython.org/docs/extension_types.html#special-methods

which says:

There is a separate page devoted to this subject, and you should read
it carefully before attempting to use any special methods in your
extension types.

Where is the page? I only found this:

http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Manual/special_methods.html

So maybe let's link to it?

Ondrej

From dagss at student.matnat.uio.no  Thu Oct 16 18:47:00 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: 16 Oct 2008 18:47:00 +0200
Subject: [Cython] removing GCC warnings for test suite,
	please review [part1]
Message-ID: <3307027641.1640445@smtp.netcom.no>

I am very against doing anything with the testcases - the fact that they trigger gcc warnings is a *good* thing.

The ultimate goal of any test suite  is full test coverage. If Cython is capable of producing gcc warnings, but the test suite does not, then that means that some code branches/some cases are never exercised, and hence a reduction in test coverage.

Of course, given resources, the warnings should be eliminated, and where that cannot be done, they should be verified as expected warnings and silenced. But that comes down to priorities...

At least my opinion is that ideally Cython should not produce C warnings if it can help it, instead any warnings should be emitted in Cython, for improved usability.

For instance with the buffer testcase, I definitely want to test *only* declaration and not using it, as a seperate testcase. Currently this leads to generating unused variables. I consider this a bug, but a bug that has been rather low on my list of priorities, especially since they would be automatically fixed by some stuff I'm thinking about down the line.

I refuse to call unused local variables a serious issue, compared to other stuff we could be improving. But, having the testcase there complaining  is much better than pretending the problem doesn't exist though.

(Disclaimer: I didn't actually read the patch as I am on my cell, answering to the contents of your post.)

Dag Sverre Seljebotn
-----Original Message-----
From: "Lisandro Dalcin" <dalcinl at gmail.com>
Date: Thursday, Oct 16, 2008 5:36 pm
Subject: [Cython] removing GCC warnings for test suite, please review [part1]
To: cython-dev <cython-dev at codespeak.net>Reply-To: cython-dev at codespeak.net

How often do you scan the full output of 'python runtests.py' in order
>to look for suspicious C-code generation? I guess never ;-), as Cython
>test suite generate many warning with GCC.
>
>As this situation make me feel very uncomfortable, you have here for
>review a patch touching many test cases for removing those nasty GCC
>warnings. If there are no major objections, I would like to push this
>ASAP.
>
>-- 
>Lisandro Dalc?n
>---------------
>Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>Tel/Fax: +54-(0)342-451.1594
>


From ondrej at certik.cz  Thu Oct 16 19:12:11 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 19:12:11 +0200
Subject: [Cython] wrapping C pointers
Message-ID: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>

Hi,

I want to use scipy's quadrature like this:

val, err = quadrature(func2, a, b, args=(f,))

where func2 is my Cython function and "f" is a C function pointer,
that will get executed from func2. The problem is, that "f" is passed
to the Python quadrature function, so it needs to be wrapped.
Currently what I do is:


cdef class MyFunc:
    cdef f2 thisptr

    cdef set_f(MyFunc self, f2 f):
        self.thisptr = f

    cdef f2 get_f(MyFunc self):
        return self.thisptr


cdef MyFunc mf = MyFunc()
mf.set_f(f)
val, err = quadrature(func2, a, b, args=(mf,))

Where func2 is:

def func2(a, MyFunc mf):
    cdef f2 f = mf.get_f()
    return array([f(x) for x in a])

This works nice. Is this the way to do it? Or is there some
better/simpler way. I don't know if it's a good idea to make Cython
clever enough to wrap things like this automatically?

Ondrej

From ondrej at certik.cz  Thu Oct 16 19:13:41 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 19:13:41 +0200
Subject: [Cython] wrapping C pointers
In-Reply-To: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
Message-ID: <85b5c3130810161013y1d3eab77q66b0a9926becd6c2@mail.gmail.com>

On Thu, Oct 16, 2008 at 7:12 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> Hi,
>
> I want to use scipy's quadrature like this:
>
> val, err = quadrature(func2, a, b, args=(f,))
>
> where func2 is my Cython function and "f" is a C function pointer,
> that will get executed from func2. The problem is, that "f" is passed
> to the Python quadrature function, so it needs to be wrapped.
> Currently what I do is:
>
>
> cdef class MyFunc:
>    cdef f2 thisptr

And here is a declaration of f2:

cdef extern from "cassembly.h":
    ctypedef double (*f2)(double x)

From robertwb at math.washington.edu  Thu Oct 16 19:37:53 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 10:37:53 -0700
Subject: [Cython] removing GCC warnings for test suite,
	please review [part1]
In-Reply-To: <3307027641.1640445@smtp.netcom.no>
References: <3307027641.1640445@smtp.netcom.no>
Message-ID: <8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>

On Oct 16, 2008, at 9:47 AM, Dag Sverre Seljebotn wrote:

> I am very against doing anything with the testcases - the fact that  
> they trigger gcc warnings is a *good* thing.
>
> The ultimate goal of any test suite  is full test coverage. If  
> Cython is capable of producing gcc warnings, but the test suite  
> does not, then that means that some code branches/some cases are  
> never exercised, and hence a reduction in test coverage.
>
> Of course, given resources, the warnings should be eliminated, and  
> where that cannot be done, they should be verified as expected  
> warnings and silenced. But that comes down to priorities...
>
> At least my opinion is that ideally Cython should not produce C  
> warnings if it can help it, instead any warnings should be emitted  
> in Cython, for improved usability.
>
> For instance with the buffer testcase, I definitely want to test  
> *only* declaration and not using it, as a seperate testcase.  
> Currently this leads to generating unused variables. I consider  
> this a bug, but a bug that has been rather low on my list of  
> priorities, especially since they would be automatically fixed by  
> some stuff I'm thinking about down the line.
>
> I refuse to call unused local variables a serious issue, compared  
> to other stuff we could be improving. But, having the testcase  
> there complaining  is much better than pretending the problem  
> doesn't exist though.

I would rather have a trac tickets about this, and have much better  
signal-to-noise ratio while running the doctest. If a change  
introduces warnings (especially real one) then I'd rather be able to  
see them loud and clear--right now I probably wouldn't notice.

> (Disclaimer: I didn't actually read the patch as I am on my cell,  
> answering to the contents of your post.)

The patch turns unused local vars into used local vars.

- Robert

>
> Dag Sverre Seljebotn
> -----Original Message-----
> From: "Lisandro Dalcin" <dalcinl at gmail.com>
> Date: Thursday, Oct 16, 2008 5:36 pm
> Subject: [Cython] removing GCC warnings for test suite, please  
> review [part1]
> To: cython-dev <cython-dev at codespeak.net>Reply-To: cython- 
> dev at codespeak.net
>
> How often do you scan the full output of 'python runtests.py' in order
>> to look for suspicious C-code generation? I guess never ;-), as  
>> Cython
>> test suite generate many warning with GCC.
>>
>> As this situation make me feel very uncomfortable, you have here for
>> review a patch touching many test cases for removing those nasty GCC
>> warnings. If there are no major objections, I would like to push this
>> ASAP.
>>
>> -- 
>> Lisandro Dalc?n
>> ---------------
>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev


From robertwb at math.washington.edu  Thu Oct 16 19:47:29 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 10:47:29 -0700
Subject: [Cython] wrapping C pointers
In-Reply-To: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
Message-ID: <78D2CD58-9759-435D-9FEC-3919865CD9D4@math.washington.edu>

On Oct 16, 2008, at 10:12 AM, Ondrej Certik wrote:

> Hi,
>
> I want to use scipy's quadrature like this:
>
> val, err = quadrature(func2, a, b, args=(f,))
>
> where func2 is my Cython function and "f" is a C function pointer,
> that will get executed from func2. The problem is, that "f" is passed
> to the Python quadrature function, so it needs to be wrapped.
> Currently what I do is:
>
>
> cdef class MyFunc:
>     cdef f2 thisptr
>
>     cdef set_f(MyFunc self, f2 f):
>         self.thisptr = f
>
>     cdef f2 get_f(MyFunc self):
>         return self.thisptr
>
>
> cdef MyFunc mf = MyFunc()
> mf.set_f(f)
> val, err = quadrature(func2, a, b, args=(mf,))
>
> Where func2 is:
>
> def func2(a, MyFunc mf):
>     cdef f2 f = mf.get_f()
>     return array([f(x) for x in a])
>
> This works nice. Is this the way to do it? Or is there some
> better/simpler way. I don't know if it's a good idea to make Cython
> clever enough to wrap things like this automatically?

If I understand right, you want an automatic python wrapper of a C  
function. Perhaps it could be done, as long as the arguments and  
return type could all be converted, but it's certainly not  
implemented currently and easily enough done by the code you have above.

- Robert


From ggellner at uoguelph.ca  Thu Oct 16 20:48:29 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Thu, 16 Oct 2008 14:48:29 -0400
Subject: [Cython] docs: cython special methods
In-Reply-To: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>
References: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>
Message-ID: <20081016184829.GA20573@encolpuis>

On Thu, Oct 16, 2008 at 06:39:44PM +0200, Ondrej Certik wrote:
> Hi,
> 
> here is a stub:
> 
> http://docs.cython.org/docs/extension_types.html#special-methods
> 
> which says:
> 
> There is a separate page devoted to this subject, and you should read
> it carefully before attempting to use any special methods in your
> extension types.
> 
> Where is the page? I only found this:
> 
> http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Manual/special_methods.html
> 
> So maybe let's link to it?
> 
I am very close to having it ported over, so we should have this at docs soon.

Gabriel

From dagss at student.matnat.uio.no  Thu Oct 16 21:07:26 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 21:07:26 +0200 (CEST)
Subject: [Cython] removing GCC warnings for test suite,
 please review [part1]
In-Reply-To: <8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>
References: <3307027641.1640445@smtp.netcom.no>
	<8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>
Message-ID: <32769.88.90.248.62.1224184046.squirrel@webmail.uio.no>

Robert wrote:
> On Oct 16, 2008, at 9:47 AM, Dag Sverre Seljebotn wrote:
>> I am very against doing anything with the testcases - the fact that
>> they trigger gcc warnings is a *good* thing.
>
> I would rather have a trac tickets about this, and have much better
> signal-to-noise ratio while running the doctest. If a change
> introduces warnings (especially real one) then I'd rather be able to
> see them loud and clear--right now I probably wouldn't notice.
>
>> (Disclaimer: I didn't actually read the patch as I am on my cell,
>> answering to the contents of your post.)
>
> The patch turns unused local vars into used local vars.

And Lisandro just reported a rather nasty bug where Cython would silently
produce uncompileable code (not warnings, plain errors) for unused
variables. (Global, but still.)

The "entry.used" stuff is non-trivial, and removing the testcases which
exercise that is not a good thing.

OK, time to find some common ground here:

- The patch is not bad in itself, but I strongly feel it should be
accompanied by a new testcase unused_symbols.pyx, where all necesarry,
non-duplicate tests are moved.

- Then this can be moved to tests/broken and a ticket opened for it,
referring to it.

BTW, Lisandro, thanks for doing this and your attention to detail! My
previous rant was a bit too harsh, and I'm slowly coming over to your side
now.

Dag Sverre


From dalcinl at gmail.com  Thu Oct 16 21:08:04 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 16:08:04 -0300
Subject: [Cython] removing GCC warnings for test suite,
	please review [part1]
In-Reply-To: <8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>
References: <3307027641.1640445@smtp.netcom.no>
	<8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>
Message-ID: <e7ba66e40810161208o2d8af230icf34b379edf2bfb5@mail.gmail.com>

Well, I'll carefully review my patch and post is to the tracker.

Dag, I'll try to make clear the spirit of this patch. Supose some
Cython test case like this:

cdef void foo():
   int var1, var2
   var1 = var2

Then GCC will emit TWO anoying warnings:
1) 'foo' is never used
2) 'var2' was used uninitialized.

Then my patch will fix test cases like the above as follow:

cdef void foo():
   int var1, var2=0
   var1 = var2

foo()

Note that I'm just initializing 'var2' and using 'foo'. Do you see any
drawback about that?


On Thu, Oct 16, 2008 at 2:37 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 16, 2008, at 9:47 AM, Dag Sverre Seljebotn wrote:
>
>> I am very against doing anything with the testcases - the fact that
>> they trigger gcc warnings is a *good* thing.
>>
>> The ultimate goal of any test suite  is full test coverage. If
>> Cython is capable of producing gcc warnings, but the test suite
>> does not, then that means that some code branches/some cases are
>> never exercised, and hence a reduction in test coverage.
>>
>> Of course, given resources, the warnings should be eliminated, and
>> where that cannot be done, they should be verified as expected
>> warnings and silenced. But that comes down to priorities...
>>
>> At least my opinion is that ideally Cython should not produce C
>> warnings if it can help it, instead any warnings should be emitted
>> in Cython, for improved usability.
>>
>> For instance with the buffer testcase, I definitely want to test
>> *only* declaration and not using it, as a seperate testcase.
>> Currently this leads to generating unused variables. I consider
>> this a bug, but a bug that has been rather low on my list of
>> priorities, especially since they would be automatically fixed by
>> some stuff I'm thinking about down the line.
>>
>> I refuse to call unused local variables a serious issue, compared
>> to other stuff we could be improving. But, having the testcase
>> there complaining  is much better than pretending the problem
>> doesn't exist though.
>
> I would rather have a trac tickets about this, and have much better
> signal-to-noise ratio while running the doctest. If a change
> introduces warnings (especially real one) then I'd rather be able to
> see them loud and clear--right now I probably wouldn't notice.
>
>> (Disclaimer: I didn't actually read the patch as I am on my cell,
>> answering to the contents of your post.)
>
> The patch turns unused local vars into used local vars.
>
> - Robert
>
>>
>> Dag Sverre Seljebotn
>> -----Original Message-----
>> From: "Lisandro Dalcin" <dalcinl at gmail.com>
>> Date: Thursday, Oct 16, 2008 5:36 pm
>> Subject: [Cython] removing GCC warnings for test suite, please
>> review [part1]
>> To: cython-dev <cython-dev at codespeak.net>Reply-To: cython-
>> dev at codespeak.net
>>
>> How often do you scan the full output of 'python runtests.py' in order
>>> to look for suspicious C-code generation? I guess never ;-), as
>>> Cython
>>> test suite generate many warning with GCC.
>>>
>>> As this situation make me feel very uncomfortable, you have here for
>>> review a patch touching many test cases for removing those nasty GCC
>>> warnings. If there are no major objections, I would like to push this
>>> ASAP.
>>>
>>> --
>>> Lisandro Dalc?n
>>> ---------------
>>> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
>>> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
>>> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
>>> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
>>> Tel/Fax: +54-(0)342-451.1594
>>>
>>
>> _______________________________________________
>> Cython-dev mailing list
>> Cython-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/cython-dev
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From ggellner at uoguelph.ca  Thu Oct 16 21:09:28 2008
From: ggellner at uoguelph.ca (Gabriel Gellner)
Date: Thu, 16 Oct 2008 15:09:28 -0400
Subject: [Cython] wrapping C pointers
In-Reply-To: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
Message-ID: <20081016190928.GB20573@encolpuis>

On Thu, Oct 16, 2008 at 07:12:11PM +0200, Ondrej Certik wrote:
> Hi,
> 
> I want to use scipy's quadrature like this:
> 
> val, err = quadrature(func2, a, b, args=(f,))
> 
> where func2 is my Cython function and "f" is a C function pointer,
> that will get executed from func2. The problem is, that "f" is passed
> to the Python quadrature function, so it needs to be wrapped.
> Currently what I do is:
> 
> 
> cdef class MyFunc:
>     cdef f2 thisptr
> 
>     cdef set_f(MyFunc self, f2 f):
>         self.thisptr = f
> 
>     cdef f2 get_f(MyFunc self):
>         return self.thisptr
> 
> 
> cdef MyFunc mf = MyFunc()
> mf.set_f(f)
> val, err = quadrature(func2, a, b, args=(mf,))
> 
> Where func2 is:
> 
> def func2(a, MyFunc mf):
>     cdef f2 f = mf.get_f()
>     return array([f(x) for x in a])
> 
> This works nice. Is this the way to do it? Or is there some
> better/simpler way. I don't know if it's a good idea to make Cython
> clever enough to wrap things like this automatically?
> 
I often just use a closure (I assume your are doing this all in cython), so
you can just have:

    cdef double f(double x):
        return x*x

    def func2(a):
        return array([f(x) for x in a])

    val, err = quadrature(func2, a, b) 

Though I am not sure if this works in your case. When we come to a consensus on
the idiomatic way of doing this we should write up some documentation!

Gabriel

From dagss at student.matnat.uio.no  Thu Oct 16 21:11:22 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 21:11:22 +0200 (CEST)
Subject: [Cython] Dag, please review this patch
In-Reply-To: <e7ba66e40810160937q2c008e4bn9cfa4103c466f425@mail.gmail.com>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<e7ba66e40810151014r2e529d31t41150de5878609e6@mail.gmail.com>
	<48F62EC2.6080907@student.matnat.uio.no>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
	<e7ba66e40810160937q2c008e4bn9cfa4103c466f425@mail.gmail.com>
Message-ID: <32879.88.90.248.62.1224184282.squirrel@webmail.uio.no>

Lisandro wrote:
> On Thu, Oct 16, 2008 at 5:00 AM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> The two should be orthogonal issues. I really believe that my patch is
>> how
>> the first problem should be solved. Attaching now how I believe the
>> second
>> problem should be solved, for which there seems to be no patch yet
>> anyway.
>
> I've attached a new patch trying to fix all the issues following your
> comments. And you are tight, this patch seems to be far better than my
> previous ones.
>
> http://trac.cython.org/cython_trac/attachment/ticket/90/GLOBALS.diff
>
> I could add a testcase, but a really good one would need the
> 'modulecleanup' directive implemented, as I need to play with the
> Py_AtExit() Python C-API call in order to detect that the generated
> module cleanup is actually working.

Positive review!

(A testcase that could be added is this one:

"""
cdef object unused
"""

(that's all!) as that one was pretty nasty, resulting in uncompileable
code, and we don't want regression on it if somebody changes what you did
in the future...)

Dag Sverre


From robertwb at math.washington.edu  Thu Oct 16 21:12:16 2008
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 16 Oct 2008 12:12:16 -0700
Subject: [Cython] wrapping C pointers
In-Reply-To: <20081016190928.GB20573@encolpuis>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
	<20081016190928.GB20573@encolpuis>
Message-ID: <0779941A-94FC-4EF4-ABEF-1A442F1EDFAF@math.washington.edu>

On Oct 16, 2008, at 12:09 PM, Gabriel Gellner wrote:

> On Thu, Oct 16, 2008 at 07:12:11PM +0200, Ondrej Certik wrote:
>> Hi,
>>
>> I want to use scipy's quadrature like this:
>>
>> val, err = quadrature(func2, a, b, args=(f,))
>>
>> where func2 is my Cython function and "f" is a C function pointer,
>> that will get executed from func2. The problem is, that "f" is passed
>> to the Python quadrature function, so it needs to be wrapped.
>> Currently what I do is:
>>
>>
>> cdef class MyFunc:
>>     cdef f2 thisptr
>>
>>     cdef set_f(MyFunc self, f2 f):
>>         self.thisptr = f
>>
>>     cdef f2 get_f(MyFunc self):
>>         return self.thisptr
>>
>>
>> cdef MyFunc mf = MyFunc()
>> mf.set_f(f)
>> val, err = quadrature(func2, a, b, args=(mf,))
>>
>> Where func2 is:
>>
>> def func2(a, MyFunc mf):
>>     cdef f2 f = mf.get_f()
>>     return array([f(x) for x in a])
>>
>> This works nice. Is this the way to do it? Or is there some
>> better/simpler way. I don't know if it's a good idea to make Cython
>> clever enough to wrap things like this automatically?
>>
> I often just use a closure (I assume your are doing this all in  
> cython), so
> you can just have:
>
>     cdef double f(double x):
>         return x*x
>
>     def func2(a):
>         return array([f(x) for x in a])
>
>     val, err = quadrature(func2, a, b)
>
> Though I am not sure if this works in your case. When we come to a  
> consensus on
> the idiomatic way of doing this we should write up some documentation!

I believe the issue is that f needs to be received as a function  
pointer at compile time, but if I'm mistaken then this is certainly  
cleaner.

- Robert



From stefan_ml at behnel.de  Thu Oct 16 21:17:19 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 16 Oct 2008 21:17:19 +0200
Subject: [Cython] removing GCC warnings for test suite,
 please review [part1]
In-Reply-To: <3307027641.1640445@smtp.netcom.no>
References: <3307027641.1640445@smtp.netcom.no>
Message-ID: <48F7933F.1070006@behnel.de>

Hi,

Dag Sverre Seljebotn wrote:
> I am very against doing anything with the testcases - the fact that they
> trigger gcc warnings is a *good* thing.

It's not intentional, though. The tests that currently trigger gcc compiler
warnings are left-overs from Pyrex's test suite, where tests aren't run and do
not even get compiled in gcc.

So, it's a good thing that Lisandro provides a patch to fix these to free the
view on the things the tests are designed to test. If we use tests to check
for broken code or compiler warnings, they should test for this explicitly,
not by accident.

Stefan

From ondrej at certik.cz  Thu Oct 16 21:18:27 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 21:18:27 +0200
Subject: [Cython] docs: cython special methods
In-Reply-To: <20081016184829.GA20573@encolpuis>
References: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>
	<20081016184829.GA20573@encolpuis>
Message-ID: <85b5c3130810161218r88ba27br8c85121ea8190496@mail.gmail.com>

On Thu, Oct 16, 2008 at 8:48 PM, Gabriel Gellner <ggellner at uoguelph.ca> wrote:
> On Thu, Oct 16, 2008 at 06:39:44PM +0200, Ondrej Certik wrote:
>> Hi,
>>
>> here is a stub:
>>
>> http://docs.cython.org/docs/extension_types.html#special-methods
>>
>> which says:
>>
>> There is a separate page devoted to this subject, and you should read
>> it carefully before attempting to use any special methods in your
>> extension types.
>>
>> Where is the page? I only found this:
>>
>> http://www.cosc.canterbury.ac.nz/greg.ewing/python/Pyrex/version/Doc/Manual/special_methods.html
>>
>> So maybe let's link to it?
>>
> I am very close to having it ported over, so we should have this at docs soon.

Excellent, thanks for all the work you do with docs.

Ondrej

From ondrej at certik.cz  Thu Oct 16 21:22:51 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 21:22:51 +0200
Subject: [Cython] wrapping C pointers
In-Reply-To: <0779941A-94FC-4EF4-ABEF-1A442F1EDFAF@math.washington.edu>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
	<20081016190928.GB20573@encolpuis>
	<0779941A-94FC-4EF4-ABEF-1A442F1EDFAF@math.washington.edu>
Message-ID: <85b5c3130810161222t56501297k9ab4e5144f70949c@mail.gmail.com>

On Thu, Oct 16, 2008 at 9:12 PM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 16, 2008, at 12:09 PM, Gabriel Gellner wrote:
>
>> On Thu, Oct 16, 2008 at 07:12:11PM +0200, Ondrej Certik wrote:
>>> Hi,
>>>
>>> I want to use scipy's quadrature like this:
>>>
>>> val, err = quadrature(func2, a, b, args=(f,))
>>>
>>> where func2 is my Cython function and "f" is a C function pointer,
>>> that will get executed from func2. The problem is, that "f" is passed
>>> to the Python quadrature function, so it needs to be wrapped.
>>> Currently what I do is:
>>>
>>>
>>> cdef class MyFunc:
>>>     cdef f2 thisptr
>>>
>>>     cdef set_f(MyFunc self, f2 f):
>>>         self.thisptr = f
>>>
>>>     cdef f2 get_f(MyFunc self):
>>>         return self.thisptr
>>>
>>>
>>> cdef MyFunc mf = MyFunc()
>>> mf.set_f(f)
>>> val, err = quadrature(func2, a, b, args=(mf,))
>>>
>>> Where func2 is:
>>>
>>> def func2(a, MyFunc mf):
>>>     cdef f2 f = mf.get_f()
>>>     return array([f(x) for x in a])
>>>
>>> This works nice. Is this the way to do it? Or is there some
>>> better/simpler way. I don't know if it's a good idea to make Cython
>>> clever enough to wrap things like this automatically?
>>>
>> I often just use a closure (I assume your are doing this all in
>> cython), so
>> you can just have:
>>
>>     cdef double f(double x):
>>         return x*x
>>
>>     def func2(a):
>>         return array([f(x) for x in a])
>>
>>     val, err = quadrature(func2, a, b)
>>
>> Though I am not sure if this works in your case. When we come to a
>> consensus on
>> the idiomatic way of doing this we should write up some documentation!
>
> I believe the issue is that f needs to be received as a function
> pointer at compile time, but if I'm mistaken then this is certainly
> cleaner.

Yes, that's the problem, that I get "f" at compile time as a C
function pointer. This arises when I have a C++ program that needs to
call some scipy functions for integration. So I just want to create my
C function and pass it to Cython->scipy->Cython->C.

> If I understand right, you want an automatic python wrapper of a C
> function. Perhaps it could be done, as long as the arguments and
> return type could all be converted, but it's certainly not
> implemented currently and easily enough done by the code you have above.

Yes, but I want it to be explicit, no magic. Maybe there is some nice
way to do that, if not, then we can at least add my way into the
documentation as Gabriel has suggested.

Ondrej

From dagss at student.matnat.uio.no  Thu Oct 16 21:29:05 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 21:29:05 +0200 (CEST)
Subject: [Cython] removing GCC warnings for test suite,
 please review [part1]
In-Reply-To: <e7ba66e40810161208o2d8af230icf34b379edf2bfb5@mail.gmail.com>
References: <3307027641.1640445@smtp.netcom.no>
	<8AFFD1C0-F2A4-4A94-AE15-1AA5FA92B9B0@math.washington.edu>
	<e7ba66e40810161208o2d8af230icf34b379edf2bfb5@mail.gmail.com>
Message-ID: <33347.88.90.248.62.1224185345.squirrel@webmail.uio.no>

Lisandro wrote:
> Then my patch will fix test cases like the above as follow:
>
> cdef void foo():
>    int var1, var2=0
>    var1 = var2
>
> foo()
>
> Note that I'm just initializing 'var2' and using 'foo'. Do you see any
> drawback about that?

Well, there are drawbacks, but I'm getting less worried about how serious
they are.

But I'll just explain the drawbacks for the sake of the explanation. We
just had a severe bug espace our attention for months (and which you
found) because

"""
cdef object unused
"""

was not a testcase. Now

"""
cdef object unusded = 3
"""

would not have helped -- in fact this is an entirely different testcase!
The latter one makes "entry.used" be set differently and would trigger
entirely different paths through the code in Cython.

Now, local variables are a different matter. In fact, I don't know how
they work myself. I just know that if they are initialized, the testcase
tests something else than what it originally tested -- some if-tests
within Cython (concerning entry.used and so on) may start taking another
route, and perhaps some code blocks are then left untested.

But this can be countered by creating a new testcase specifically targeted
for covering what you now remove, so I'm getting more friendly towards
your patch.

Dag Sverre


From dagss at student.matnat.uio.no  Thu Oct 16 21:37:00 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 21:37:00 +0200 (CEST)
Subject: [Cython] wrapping C pointers
In-Reply-To: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
Message-ID: <32830.88.90.248.62.1224185820.squirrel@webmail.uio.no>

Ondrej Certik wrote:
...
> Where func2 is:
>
> def func2(a, MyFunc mf):
>     cdef f2 f = mf.get_f()
>     return array([f(x) for x in a])
>
> This works nice. Is this the way to do it? Or is there some
> better/simpler way. I don't know if it's a good idea to make Cython
> clever enough to wrap things like this automatically?

I think the way you did it is more than elegant enough :-) As others have
commented, this is fine.

This is just a note on performance: That list comprehension is really
going to kill performance. If you do this instead:

def func2(np.ndarray[right_dtype_t, ndim=1] a, MyFunc mf):
    cdef f2 f = mf.get_f()
    cdef np.ndarray[right_dtype_t, ndim=1] result = np.empty(a.shape,
right_dtype)
    cdef unsigned int i
    for i in range(a.shape[0]):
        result[i] = f(a[i])
    return result

...then it should be much, much faster (avoiding conversion of every
single array element back and forth from/to Python objects).

Dag Sverre


From ondrej at certik.cz  Thu Oct 16 22:44:24 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 16 Oct 2008 22:44:24 +0200
Subject: [Cython] wrapping C pointers
In-Reply-To: <32830.88.90.248.62.1224185820.squirrel@webmail.uio.no>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
	<32830.88.90.248.62.1224185820.squirrel@webmail.uio.no>
Message-ID: <85b5c3130810161344nc448504g11673877788bf457@mail.gmail.com>

On Thu, Oct 16, 2008 at 9:37 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Ondrej Certik wrote:
> ...
>> Where func2 is:
>>
>> def func2(a, MyFunc mf):
>>     cdef f2 f = mf.get_f()
>>     return array([f(x) for x in a])
>>
>> This works nice. Is this the way to do it? Or is there some
>> better/simpler way. I don't know if it's a good idea to make Cython
>> clever enough to wrap things like this automatically?
>
> I think the way you did it is more than elegant enough :-) As others have
> commented, this is fine.
>
> This is just a note on performance: That list comprehension is really
> going to kill performance. If you do this instead:
>
> def func2(np.ndarray[right_dtype_t, ndim=1] a, MyFunc mf):
>    cdef f2 f = mf.get_f()
>    cdef np.ndarray[right_dtype_t, ndim=1] result = np.empty(a.shape,
> right_dtype)
>    cdef unsigned int i
>    for i in range(a.shape[0]):
>        result[i] = f(a[i])
>    return result
>
> ...then it should be much, much faster (avoiding conversion of every
> single array element back and forth from/to Python objects).

Thanks Dag! This is really useful, I was thinking how to do that
efficiently. This is basically vectorise optimized for C functions.

Maybe stuff like this could go to some numpy Cython file shipped with
numpy? Those are things that are needed over and over again.

Ondrej

From dagss at student.matnat.uio.no  Thu Oct 16 23:05:50 2008
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 16 Oct 2008 23:05:50 +0200 (CEST)
Subject: [Cython] wrapping C pointers
In-Reply-To: <85b5c3130810161344nc448504g11673877788bf457@mail.gmail.com>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
	<32830.88.90.248.62.1224185820.squirrel@webmail.uio.no>
	<85b5c3130810161344nc448504g11673877788bf457@mail.gmail.com>
Message-ID: <33553.88.90.248.62.1224191150.squirrel@webmail.uio.no>

Ondrej Certik wrote:
> On Thu, Oct 16, 2008 at 9:37 PM, Dag Sverre Seljebotn
> <dagss at student.matnat.uio.no> wrote:
>> Ondrej Certik wrote:
>> ...
>>> Where func2 is:
>>>
>>> def func2(a, MyFunc mf):
>>>     cdef f2 f = mf.get_f()
>>>     return array([f(x) for x in a])
>>>
>>> This works nice. Is this the way to do it? Or is there some
>>> better/simpler way. I don't know if it's a good idea to make Cython
>>> clever enough to wrap things like this automatically?
>>
>> I think the way you did it is more than elegant enough :-) As others
>> have
>> commented, this is fine.
>>
>> This is just a note on performance: That list comprehension is really
>> going to kill performance. If you do this instead:
>>
>> def func2(np.ndarray[right_dtype_t, ndim=1] a, MyFunc mf):
>>    cdef f2 f = mf.get_f()
>>    cdef np.ndarray[right_dtype_t, ndim=1] result = np.empty(a.shape,
>> right_dtype)
>>    cdef unsigned int i
>>    for i in range(a.shape[0]):
>>        result[i] = f(a[i])
>>    return result
>>
>> ...then it should be much, much faster (avoiding conversion of every
>> single array element back and forth from/to Python objects).
>
> Thanks Dag! This is really useful, I was thinking how to do that
> efficiently. This is basically vectorise optimized for C functions.
>
> Maybe stuff like this could go to some numpy Cython file shipped with
> numpy? Those are things that are needed over and over again.

Feel free to propose it for them :-) (I don't have anything to do with
NumPy development myself.) We could probably ship Cython utility libraries
available together with Cython as well. Someone needs to write it though,
and I don't have the time for it.

Note: It is only a 1D vectorise. To work with multi-dimensional arrays
more complicated code is needed (NumPy generic nd-iterators using the
NumPy C API). Probably the NumPy C API has some code one could utilize.

That's the thing with NumPy -- things always get very generic when one
wants to create something that's usable beyond your own specific usecase.

Dag Sverre


From ellisonbg.net at gmail.com  Fri Oct 17 00:16:51 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 16 Oct 2008 15:16:51 -0700
Subject: [Cython] Literal struct syntax
In-Reply-To: <4F4749BD-D22F-4F6E-93DA-CF45557B15C2@math.washington.edu>
References: <48F3B338.4060308@student.matnat.uio.no>
	<E72ABC2B-336C-4249-9D28-58CAE9D795DE@math.washington.edu>
	<32792.213.61.181.86.1224054204.squirrel@groupware.dvs.informatik.tu-darmstadt.de>
	<48F67DEF.9030800@canterbury.ac.nz>
	<e7ba66e40810151831s4ccd3a0ey1aa3e3be160bfc2@mail.gmail.com>
	<48F6B87A.7040903@canterbury.ac.nz>
	<e7ba66e40810160720t1636c614m82c469ed7766be3b@mail.gmail.com>
	<4F4749BD-D22F-4F6E-93DA-CF45557B15C2@math.washington.edu>
Message-ID: <6ce0ac130810161516y2c9f7472i98a4c36f658de624@mail.gmail.com>

Just watching from the sides....

But, I vote +1 on disallowing the "almost" dict notation {a:1, b:2}.

Brian

On Thu, Oct 16, 2008 at 8:33 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Oct 16, 2008, at 7:20 AM, Lisandro Dalcin wrote:
>
>> On Thu, Oct 16, 2008 at 12:43 AM, Greg Ewing
>> <greg.ewing at canterbury.ac.nz> wrote:
>>> Lisandro Dalcin wrote:
>>>
>>>> And how are you going to free the memory?
>>>
>>> I'm not, that's up to the programmer. It's just a
>>> more convenient way of writing a malloc.
>>
>> Then I have to say that I'm -1 in such magic. What about introducing
>> two new pseudo-keywords cnew/cdel, and use it like this (like in C++):
>>
>> MyStruct *p = cnew MyStruct(a=1,b=2)
>> # use p
>> cdel p # also sets p to NULL
>> assert p==NULL
>> cdel p # success, but do nothing?
>>
>> Such approach would make Cython/Pyrex closer to C++ behavior.
>> Moreover, this could be extended to support funny features, like
>> letting user choose the alloc/dealloc routines and perhaps
>> constructor/destructor routines.
>>
>> What do you think?
>
> First, thanks for all the feedback, and thanks Dag for bringing this up.
>
> I'd rather focus on making Cython closer to the Python language than
> introducing new concepts. Also, I'm with Stefan that it is very
> dangerous to hide the malloc and expect the user to explicitly
> provide the free. If the user wants to manage their own memory, they
> can do so with malloc and friends (understanding the associated
> dangers), and any special memory-management stuff we add should be as
> implicit and easy to use as Python's garbage collection (and probably
> piggyback off of it).
>
> I think the MyStruct(...) should result in an actual struct (not a
> pointer), but perhaps on conversion to an object it could be made
> into a memory managed version of the struct (e.g. using the same
> string allocation trick we talked about for arrays) giving C access,
> pass-by-reference, and memory management. This, however, is not going
> to happen in the next release of Cython and when it does it could be
> made backwards compatible with the current conversion to dict.
> (Syntax suggestions for how to declare such an object welcome,
> perhaps "cdef object MyStruct foo" and memory-managed arrays could be
> "cdef object int[] foo")
>
> As for the current proposal, don't want to introduce new, special
> "struct instanciation dictionaries," so I'd opt for allowing the
> {"a": 1, "b": 2} literals with the plan of conversion from a runtime
> dict down the road (including runtime-created keys), but disabling
> the {a: 1, b: 2} as the MyStruct(a=1, b=2) is available (and, even
> better, more explicit).
>
> - Robert
>
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From dalcinl at gmail.com  Fri Oct 17 01:20:07 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 16 Oct 2008 20:20:07 -0300
Subject: [Cython] Dag, please review this patch
In-Reply-To: <32879.88.90.248.62.1224184282.squirrel@webmail.uio.no>
References: <e7ba66e40810150924v5c9be975t163061611b09d2d@mail.gmail.com>
	<48F632C5.6080706@student.matnat.uio.no>
	<e7ba66e40810151133r4b2795f6x936b3e556acc11df@mail.gmail.com>
	<48F64CFA.8030100@student.matnat.uio.no>
	<e7ba66e40810151310m303c03dalfda8655b1a0784d5@mail.gmail.com>
	<48F6530E.5080401@student.matnat.uio.no>
	<e7ba66e40810151439k1f7959eam99ff51e67a8e3c0a@mail.gmail.com>
	<48F6F4B4.1020602@student.matnat.uio.no>
	<e7ba66e40810160937q2c008e4bn9cfa4103c466f425@mail.gmail.com>
	<32879.88.90.248.62.1224184282.squirrel@webmail.uio.no>
Message-ID: <e7ba66e40810161620w753b338cq3b28e72b59739e8d@mail.gmail.com>

On Thu, Oct 16, 2008 at 4:11 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
>
> Positive review!

OK, then I'll push it.

> (A testcase that could be added is this one:
>
> """
> cdef object unused
> """
>
> (that's all!) as that one was pretty nasty, resulting in uncompileable
> code, and we don't want regression on it if somebody changes what you did
> in the future...)

OK, Then I would add a unusedmoduleglobal.pyx inside tests/compile with

"""
cdef object unused
"""

and I'll try to implement some hackery (at the C level, using some
auxiliar include file) to make completelly sure that Cython actually
do not ever emitted the declaration.


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From greg.ewing at canterbury.ac.nz  Fri Oct 17 02:17:06 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 17 Oct 2008 13:17:06 +1300
Subject: [Cython] docs: cython special methods
In-Reply-To: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>
References: <85b5c3130810160939k364f0562h717e8fc7a0c96cfc@mail.gmail.com>
Message-ID: <48F7D982.1060204@canterbury.ac.nz>

Ondrej Certik wrote:

> There is a separate page devoted to this subject, and you should read
> it carefully before attempting to use any special methods in your
> extension types.
> 
> Where is the page?

In the original Pyrex docs, the words "separate page" in the above
paragraph are a link, which seems to have got lost.

Also, the "Special Methods" link in the table of contents of that
page in the Pyrex docs now point to the page instead of the stub,
and there is a link to it in the top-level contents as well. You
may want to update the Cython docs similarly.

-- 
Greg

From ondrej at certik.cz  Fri Oct 17 02:57:29 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Fri, 17 Oct 2008 02:57:29 +0200
Subject: [Cython] wrapping C pointers
In-Reply-To: <33553.88.90.248.62.1224191150.squirrel@webmail.uio.no>
References: <85b5c3130810161012r3dbe82aet855f3a2a66788318@mail.gmail.com>
	<32830.88.90.248.62.1224185820.squirrel@webmail.uio.no>
	<85b5c3130810161344nc448504g11673877788bf457@mail.gmail.com>
	<33553.88.90.248.62.1224191150.squirrel@webmail.uio.no>
Message-ID: <85b5c3130810161757j255ba010q89181ecc4c6e8916@mail.gmail.com>

On Thu, Oct 16, 2008 at 11:05 PM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:
> Ondrej Certik wrote:
>> On Thu, Oct 16, 2008 at 9:37 PM, Dag Sverre Seljebotn
>> <dagss at student.matnat.uio.no> wrote:
>>> Ondrej Certik wrote:
>>> ...
>>>> Where func2 is:
>>>>
>>>> def func2(a, MyFunc mf):
>>>>     cdef f2 f = mf.get_f()
>>>>     return array([f(x) for x in a])
>>>>
>>>> This works nice. Is this the way to do it? Or is there some
>>>> better/simpler way. I don't know if it's a good idea to make Cython
>>>> clever enough to wrap things like this automatically?
>>>
>>> I think the way you did it is more than elegant enough :-) As others
>>> have
>>> commented, this is fine.
>>>
>>> This is just a note on performance: That list comprehension is really
>>> going to kill performance. If you do this instead:
>>>
>>> def func2(np.ndarray[right_dtype_t, ndim=1] a, MyFunc mf):
>>>    cdef f2 f = mf.get_f()
>>>    cdef np.ndarray[right_dtype_t, ndim=1] result = np.empty(a.shape,
>>> right_dtype)
>>>    cdef unsigned int i
>>>    for i in range(a.shape[0]):
>>>        result[i] =