From robertwb at math.washington.edu  Wed Feb  2 09:08:27 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 2 Feb 2011 00:08:27 -0800
Subject: [Cython] Cython workshop
In-Reply-To: <4D3C07E9.3070505@behnel.de>
References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de>
	<AANLkTik3Yhd5N+WrYZAwn5QASKppu=a5LhZ_t9NrKO8G@mail.gmail.com>
	<4D3BF102.5040507@student.matnat.uio.no> <4D3C07E9.3070505@behnel.de>
Message-ID: <AANLkTikiqAUsam-=Yd-kheDvZET=qr-kPsFCaGN0gkfp@mail.gmail.com>

On Sun, Jan 23, 2011 at 2:50 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Dag Sverre Seljebotn, 23.01.2011 10:12:
>> On 01/23/2011 09:53 AM, Robert Bradshaw wrote:
>>> On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote:
>>>
>>>> Dag Sverre Seljebotn, 20.01.2011 12:26:
>>>>
>>>>> Starting new thread in case somebody did not see the hijacking.
>>>>>
>>>>> I've created this wiki page:
>>>>>
>>>>> http://wiki.cython.org/workshop1
>>>>>
>>>>> If you're interested in coming to a Cython workshop, please fill in your
>>>>> details in the table to help decide when and where a workshop should be
>>>>> held.
>>>>>
>>>> Current status so far: 4 core developers have signed in, 5 people in total,
>>>> all from Europe. Robert said he'd like to join in, too.
>>>>
>>>> Have there been any further off-list replies so far?
>>>>
>>> I've spoken to two people, both of whom won't be able to do much the
>>> first half of this year.
>
> Still, I think "close to now" is a very good time to do it, because 0.14.x
> is nearing stability and thus leaving the focus but needs a serious
> docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0
> is getting in sight and worth discussing. Certainly enough to fill four days.
>
>
>>>> In case we actually want to advertise and run this before the end of April
>>>> (or even March, and Vitja's visa will likely take a while to get), it would
>>>> be good if the remaining core developers and other interested parties could
>>>> speak up soon, say, within a week's time, so that we can decide on date and
>>>> location.
>>>>
>>> The first half of March doesn't work for me, and April would be
>>> tricky, but I might be able to do March 30-April 3. More ideal would
>>> be May-June-July, but that conflicts directly with Stefans
>>> availability--are there any weekends in there that would work or are
>>> those months right out?
>
> Tricky for me. The first weekend in June might work, but I'd have to check
> that with my employer. End of June may work better.
>
>
>>> I think 4 days is about the right amount of time.
>>
>> Within the months I listed there's good chances of things working for
>> me; although April-June is better than July. March 30 - April 3 should work.
>
> Assuming there's never a date that won't be tricky for any of us, it seems
> that the first weekend in April could make it.

Lets officially plan on that.

- Robert

From robertwb at math.washington.edu  Fri Feb  4 10:48:51 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 4 Feb 2011 01:48:51 -0800
Subject: [Cython] Sage problem after fixing #654
In-Reply-To: <AANLkTikUOEzMYjWMKapC1-oQQxUzM8DGPxEYG7WEkVP9@mail.gmail.com>
References: <4D4110CB.4050507@behnel.de>
	<AANLkTikU1DtHy8qn3Esk7e6YUeLq7MeOGc9CfR-vLyTZ@mail.gmail.com>
	<4D412F73.8080206@behnel.de> <4D432E8E.3010707@behnel.de>
	<AANLkTingYoCesAFmomQiUxazMNxNvKaD1hU3jArHnqmR@mail.gmail.com>
	<4D43A24B.1020901@behnel.de>
	<AANLkTikUOEzMYjWMKapC1-oQQxUzM8DGPxEYG7WEkVP9@mail.gmail.com>
Message-ID: <AANLkTik9DKMo0xkUo+yXt_MHX2jWA3Odxj0NJmLf64b0@mail.gmail.com>

On Sat, Jan 29, 2011 at 2:24 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> On Fri, Jan 28, 2011 at 9:14 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Robert Bradshaw, 28.01.2011 22:49:
>>> On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote:
>>>> And given that Cython cannot know that the pointer is actually not a
>>>> pointer, it generates the expected code here which only the C compiler can
>>>> detect as invalid.
>>>> This needs fixing in Sage in one way or another.
>>>
>>> Cython didn't used to support tyepdef'd arrays, so that's why it was
>>> declared as a pointer, but yes, that should be fixed. I'll do that in
>>> our copy, and it will go out with the 0.14.1 fixes (even though we
>>> don't need it yet). There's still the case of, say,
>>>
>>> ? ? ?mpfr_init2(self._args[f(i)], self.domain.prec())
>>>
>>> where f is a cdef int -> ?int function. Do we handle that?
>>
>> Sure. The problem isn't (any longer) that we *forget* to put things into a
>> temp, the problem is that we try to put things into a temp now that we
>> cannot store away.
>>
>> I've been dropping some cases in my latest commits where the temp copying
>> mechanism was triggered needlessly. It's basically a matter of properly
>> implementing is_simple() for an ExprNode in order to decide when a non-temp
>> result is side-effect free.
>>
>> I'm still not sure if a coerce_to_owned() method would be worthwhile. As
>> long as SimpleCallNode is the only place where this is actually needed,
>> it's fine to leave the code there. Can anyone think of other places where
>> the order of C value evaluation matters? Tuples and other container
>> literals may qualify, but given that they use only Python values, I don't
>> see it being a problem there.
>
> I'm not sure we handle the optimization of, e.g, "a in (x, y, z)"
> There might be others as well.
>
> Fixing Sage is turning out to be non-trivial...there are many places
> that assume mpz_t is (compatible with) a void*. We also have the
> problem that Cython doesn't let you take the address of an array, as
> it is a non-lvalue, but that's legal in C (and used in Sage).

There's another problem, assigning things to a temporary C++ class or
struct can have side effects (as well as be inefficient).

- Robert

From robertwb at gmail.com  Fri Feb  4 11:51:16 2011
From: robertwb at gmail.com (robertwb)
Date: Fri, 4 Feb 2011 02:51:16 -0800 (PST)
Subject: [Cython] Cython 0.14.1 has been released
Message-ID: <7eefe7dd-8c6f-4401-942a-7006f3475224@i39g2000prd.googlegroups.com>

Cython 0.14.1 has just been released. This release is primarily a bug
fix release building on top of 0.14.

Download: http://cython.org/release/Cython-0.14.1rc3.tar.gz

== New Features ==

- The gdb debugging support was extended to include all major Cython
features, including closures.
raise MemoryError() is now safe to use as Cython replaces it with the
correct C-API call.
- Better support for testing (including on Windows)
- General improvements and bug fixes

The bug tracker has a list of the major improvements and fixes:
http://trac.cython.org/cython_trac/query?status=closed&group=component&order=id&col=id&col=summary&col=milestone&col=status&col=type&col=priority&col=component&milestone=0.14.1&desc=1

== Incompatible changes ==

- Decorators on special methods of cdef classes now raise a compile
time error rather than being ignored.
- In Python 3 language level mode (-3 option), the 'str' type is now
mapped to 'unicode', so that cdef str s declares a Unicode string even
when running in Python 2.

== Contributors ==

Many people contributed to this release, including:

- Stefan Behnel
- Alexey Borzenkov
- Robert Bradshaw
- Lisandro Dalcin
- Mark Florisson
- W. Trevor King
- Vitja Makarov

A special thanks also to the many people who did testing and
contributed useful bug reports.

From stefan_ml at behnel.de  Fri Feb  4 16:00:57 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 04 Feb 2011 16:00:57 +0100
Subject: [Cython] Cython 0.14.1 has been released
In-Reply-To: <7eefe7dd-8c6f-4401-942a-7006f3475224@i39g2000prd.googlegroups.com>
References: <7eefe7dd-8c6f-4401-942a-7006f3475224@i39g2000prd.googlegroups.com>
Message-ID: <4D4C14A9.6030702@behnel.de>

robertwb, 04.02.2011 11:51:
> Cython 0.14.1 has just been released. This release is primarily a bug
> fix release building on top of 0.14.
>
> Download: http://cython.org/release/Cython-0.14.1rc3.tar.gz

You may prefer getting it from PyPI or from this URL:

http://cython.org/release/Cython-0.14.1.tar.gz

Stefan

From robertwb at math.washington.edu  Fri Feb  4 18:23:48 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 4 Feb 2011 09:23:48 -0800
Subject: [Cython] [cython-users] Re:  Cython 0.14.1 has been released
In-Reply-To: <4D4C14A9.6030702@behnel.de>
References: <7eefe7dd-8c6f-4401-942a-7006f3475224@i39g2000prd.googlegroups.com>
	<4D4C14A9.6030702@behnel.de>
Message-ID: <AANLkTi=X2S4a8EE3fg1ynkq075FLBMcP+Gztytw9wM1g@mail.gmail.com>

On Fri, Feb 4, 2011 at 7:00 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> robertwb, 04.02.2011 11:51:
>>
>> Cython 0.14.1 has just been released. This release is primarily a bug
>> fix release building on top of 0.14.
>>
>> Download: http://cython.org/release/Cython-0.14.1rc3.tar.gz
>
> You may prefer getting it from PyPI or from this URL:
>
> http://cython.org/release/Cython-0.14.1.tar.gz

Oops, I posted the rc3 link. I've done a 302 redirect to the correct
one for the time being.

- Robert

From robertwb at math.washington.edu  Fri Feb  4 19:50:59 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 4 Feb 2011 10:50:59 -0800
Subject: [Cython] Switching from Py_UNICODE to Py_UCS4
In-Reply-To: <4D43ED77.1070201@behnel.de>
References: <4D43C3D5.8070506@behnel.de>
	<AANLkTim=cmwjt5aqHPtsZ2bf3=ZPB_AURjwYQ_Q6Mcxs@mail.gmail.com>
	<4D43ED77.1070201@behnel.de>
Message-ID: <AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>

On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 29.01.2011 10:01:
>> On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel wrote:
>>> there is a recent discussion on python-dev about a new memory layout for
>>> the unicode type in CPython 3.3(?), proposed by Martin von L?wis (so it's
>>> serious ;)
>>>
>>> http://comments.gmane.org/gmane.comp.python.devel/120784
>>
>> That's an interesting PEP, I like it.
>
> Yep, after some discussion, I started liking it too. Even if it means I'll
> have to touch a lot of code in Cython again. ;)
>
>
>>> If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit
>>> unsigned int), which I had completely lost from sight. It's public and
>>> undocumented and has been there basically forever, but it's a much nicer
>>> type to support than Py_UNICODE, which changes size based on build time
>>> options. Py_UCS4 is capable of representing any Unicode code point on any
>>> platform.
>>>
>>> So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4
>>> internally (without breaking user code which can continue to use either of
>>> the two explicitly). This means that loops over unicode objects will infer
>>> Py_UCS4 as loop variable, as would indexing. It would basically become the
>>> native C type that 1 character unicode strings would coerce to and from.
>>> Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value
>>> is too large in the given CPython runtime, as would write access to unicode
>>> objects (in case anyone really does that) outside of the platform specific
>>> Py_UNICODE value range. Writing to unicode buffers will be dangerous and
>>> tricky anyway if the above PEP gets accepted.
>>
>> I am a bit concerned about the performance overhead of the Py_UCS4 to
>> Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
>> maybe that's both uncommon and negligible.
>
> I think so. If users deal with Py_UNICODE explicitly, they'll likely type
> their respective variables anyway, so that there won't be an intermediate
> step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all.
>
>
>>> One open question that I see is whether we should handle surrogate pairs
>>> automatically. They are basically a split of large Unicode code point
>>> values (>65535) into two code points in specific ranges that are safe to
>>> detect. So we could allow a 2 'character' surrogate pair in a unicode
>>> string to coerce to one Py_UCS4 character and coerce that back into a
>>> surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that
>>> this would only work for single characters, not for looping or indexing
>>> (without the PEP, that is). So it's somewhat inconsistent. It would work
>>> well for literals, though. Also, we'd have to support it for 'in' tests, as
>>> a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the
>>> character is in the string.
>>
>> No, I don't think we should handle surrogate pairs automatically, at
>> least without making it optional--this could be a significant
>> performance impact with little benefit for most users. Using these
>> higher characters is rare, but using them on a non USS4 build is
>> probably even rarer.
>
> Well, basically they are the only way to use 'wide' Unicode characters on
> 16bit Unicode builds.
>
> I think a unicode string of length 2 should be able to coerce into a
> Py_UCS4 value at runtime instead of raising the current exception because
> it's too long.

Sure, that's fine by me.

> For the opposite direction, integer to unicode string, you
> already get a string of length 2 on narrow builds, that's how
> unichr()/chr() work in Python 2/3. So, in a way, it's actually more
> consistent with how narrow builds work today.

OK.

> The only reason this isn't
> currently working in Cython is that Py_UNICODE is too small on narrow
> builds to represent the larger Unicode code points. If we switched to
> Py_UCS4, the problem would go away in narrow builds now and code could be
> written today that would easily continue to work efficiently in a post-PEP
> CPython as it wouldn't rely on the deprecated (and then inefficient)
> Py_UNICODE type anymore.
>
> What about supporting surrogate pairs in 'in' tests only on narrow
> platforms? I mean, we could simply duplicate the search code for that,
> depending on how large the code point value really is at runtime. That code
> will become a lot more involved anyway when the PEP gets implemented.

Sure. This shouldn't have non-negligible performance overhead for the
simple case, and would be consistent with coercing to a 2-character
Unicode as above then doing the Python in operator.

>> Also, this would be inconsistant with
>> python-level slicing, indexing, and range, right?
>
> Yes, it does not match well with slicing and indexing. That's the problem
> with narrow builds in both CPython and Cython. Only the PEP can fix that by
> basically dropping the restrictions of a narrow build.

Lets let indexing do what indexing does.

- Robert

From stefan_ml at behnel.de  Sat Feb  5 20:08:48 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 05 Feb 2011 20:08:48 +0100
Subject: [Cython] We missed a crash bug in 0.14.1
Message-ID: <4D4DA040.20609@behnel.de>

Hi,

sadly, we missed the opportunity to fix a crash bug in 0.14.1. It's been 
there for ages, so it's not a regression, but the recent runs of the Py2.6 
regression tests should have been red enough to tell us that it was there...

http://trac.cython.org/cython_trac/ticket/658

Basically, when you put "*args" or "**kwargs" into a closure, and anything 
bad happens during argument unpacking (such as an illegal argument type), 
the closure fields will get DECREF-ed but not set back to 0, so that the 
deallocation code crashes afterwards.

I don't think it's that uncommon to have args and kwargs in a closure, just 
think of a generic wrapper function. I opened a milestone 0.14.2 because I 
think this is worth fixing soon, maybe within a week or two, in case we 
want to wait for other problems to turn up.

Stefan

From stefan_ml at behnel.de  Sun Feb  6 09:45:31 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Feb 2011 09:45:31 +0100
Subject: [Cython] Switching from Py_UNICODE to Py_UCS4
In-Reply-To: <AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>
References: <4D43C3D5.8070506@behnel.de>	<AANLkTim=cmwjt5aqHPtsZ2bf3=ZPB_AURjwYQ_Q6Mcxs@mail.gmail.com>	<4D43ED77.1070201@behnel.de>
	<AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>
Message-ID: <4D4E5FAB.7020707@behnel.de>

Robert Bradshaw, 04.02.2011 19:50:
> On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel wrote:
>> Robert Bradshaw, 29.01.2011 10:01:
>>> On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel wrote:
>>>> there is a recent discussion on python-dev about a new memory layout for
>>>> the unicode type in CPython 3.3(?), proposed by Martin von L?wis (so it's
>>>> serious ;)
>>>>
>>>> http://comments.gmane.org/gmane.comp.python.devel/120784
>>>
>>> That's an interesting PEP, I like it.
>>
>> Yep, after some discussion, I started liking it too. Even if it means I'll
>> have to touch a lot of code in Cython again. ;)
>>
>>
>>>> If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit
>>>> unsigned int), which I had completely lost from sight. It's public and
>>>> undocumented and has been there basically forever, but it's a much nicer
>>>> type to support than Py_UNICODE, which changes size based on build time
>>>> options. Py_UCS4 is capable of representing any Unicode code point on any
>>>> platform.
>>>>
>>>> So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4
>>>> internally (without breaking user code which can continue to use either of
>>>> the two explicitly). This means that loops over unicode objects will infer
>>>> Py_UCS4 as loop variable, as would indexing. It would basically become the
>>>> native C type that 1 character unicode strings would coerce to and from.
>>>> Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value
>>>> is too large in the given CPython runtime, as would write access to unicode
>>>> objects (in case anyone really does that) outside of the platform specific
>>>> Py_UNICODE value range. Writing to unicode buffers will be dangerous and
>>>> tricky anyway if the above PEP gets accepted.
>>>
>>> I am a bit concerned about the performance overhead of the Py_UCS4 to
>>> Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
>>> maybe that's both uncommon and negligible.
>>
>> I think so. If users deal with Py_UNICODE explicitly, they'll likely type
>> their respective variables anyway, so that there won't be an intermediate
>> step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all.

Coming back to this once more: if the PEP gets implemented, we will only 
know at C compile time (Py>=3.3 or not) if the result of indexing 
(including for-loop iteration) is Py_UCS4 or Py_UNICODE. For Cython's type 
inference, Py_UCS4 is therefore the more correct guess. So my proposal 
stands to always infer Py_UCS4 instead of Py_UNICODE for indexing, even if 
we ignore surrogate pairs in narrow Python builds.

I will implement this for now, so that we can see what it gives.


>>>> One open question that I see is whether we should handle surrogate pairs
>>>> automatically. They are basically a split of large Unicode code point
>>>> values (>65535) into two code points in specific ranges that are safe to
>>>> detect. So we could allow a 2 'character' surrogate pair in a unicode
>>>> string to coerce to one Py_UCS4 character and coerce that back into a
>>>> surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that
>>>> this would only work for single characters, not for looping or indexing
>>>> (without the PEP, that is). So it's somewhat inconsistent. It would work
>>>> well for literals, though. Also, we'd have to support it for 'in' tests, as
>>>> a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the
>>>> character is in the string.
>>>
>>> No, I don't think we should handle surrogate pairs automatically, at
>>> least without making it optional--this could be a significant
>>> performance impact with little benefit for most users. Using these
>>> higher characters is rare, but using them on a non USS4 build is
>>> probably even rarer.
>>
>> Well, basically they are the only way to use 'wide' Unicode characters on
>> 16bit Unicode builds.
>>
>> I think a unicode string of length 2 should be able to coerce into a
>> Py_UCS4 value at runtime instead of raising the current exception because
>> it's too long.
>
> Sure, that's fine by me.

This is now implemented for narrow builds.


>> For the opposite direction, integer to unicode string, you
>> already get a string of length 2 on narrow builds, that's how
>> unichr()/chr() work in Python 2/3. So, in a way, it's actually more
>> consistent with how narrow builds work today.
>
> OK.
>
>> The only reason this isn't
>> currently working in Cython is that Py_UNICODE is too small on narrow
>> builds to represent the larger Unicode code points. If we switched to
>> Py_UCS4, the problem would go away in narrow builds now and code could be
>> written today that would easily continue to work efficiently in a post-PEP
>> CPython as it wouldn't rely on the deprecated (and then inefficient)
>> Py_UNICODE type anymore.
>>
>> What about supporting surrogate pairs in 'in' tests only on narrow
>> platforms? I mean, we could simply duplicate the search code for that,
>> depending on how large the code point value really is at runtime. That code
>> will become a lot more involved anyway when the PEP gets implemented.
>
> Sure. This shouldn't have non-negligible performance overhead for the
> simple case, and would be consistent with coercing to a 2-character
> Unicode as above then doing the Python in operator.

Also implemented for narrow builds now, if the character type is Py_UCS4 
and not Py_UNICODE.


>>> Also, this would be inconsistant with
>>> python-level slicing, indexing, and range, right?
>>
>> Yes, it does not match well with slicing and indexing. That's the problem
>> with narrow builds in both CPython and Cython. Only the PEP can fix that by
>> basically dropping the restrictions of a narrow build.
>
> Lets let indexing do what indexing does.

Ok. So you'd continue to get whatever CPython returns for indexing, i.e. 
Py_UNICODE in Py<=3.2 and Py_UCS4 in Python versions that implement the 
PEP. That includes separate code points for surrogate pairs on narrow builds.

Stefan

From stefan_ml at behnel.de  Sun Feb  6 10:08:31 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Feb 2011 10:08:31 +0100
Subject: [Cython] Switching from Py_UNICODE to Py_UCS4
In-Reply-To: <4D4E5FAB.7020707@behnel.de>
References: <4D43C3D5.8070506@behnel.de>	<AANLkTim=cmwjt5aqHPtsZ2bf3=ZPB_AURjwYQ_Q6Mcxs@mail.gmail.com>	<4D43ED77.1070201@behnel.de>	<AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>
	<4D4E5FAB.7020707@behnel.de>
Message-ID: <4D4E650F.7080909@behnel.de>

Stefan Behnel, 06.02.2011 09:45:
> Robert Bradshaw, 04.02.2011 19:50:
>> On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel wrote:
>>> Robert Bradshaw, 29.01.2011 10:01:
>>>> Also, this would be inconsistant with
>>>> python-level slicing, indexing, and range, right?
>>>
>>> Yes, it does not match well with slicing and indexing. That's the problem
>>> with narrow builds in both CPython and Cython. Only the PEP can fix that by
>>> basically dropping the restrictions of a narrow build.
>>
>> Lets let indexing do what indexing does.
>
> Ok. So you'd continue to get whatever CPython returns for indexing, i.e.
> Py_UNICODE in Py<=3.2 and Py_UCS4 in Python versions that implement the
> PEP. That includes separate code points for surrogate pairs on narrow builds.

... on narrow builds of pre-PEP Python versions, that is.

Stefan

From robertwb at math.washington.edu  Sun Feb  6 10:14:18 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Sun, 6 Feb 2011 01:14:18 -0800
Subject: [Cython] Switching from Py_UNICODE to Py_UCS4
In-Reply-To: <4D4E5FAB.7020707@behnel.de>
References: <4D43C3D5.8070506@behnel.de>
	<AANLkTim=cmwjt5aqHPtsZ2bf3=ZPB_AURjwYQ_Q6Mcxs@mail.gmail.com>
	<4D43ED77.1070201@behnel.de>
	<AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>
	<4D4E5FAB.7020707@behnel.de>
Message-ID: <AANLkTi=Hsxc6YFWGr=hS+64Qx7pNGTprWs_FMKLdYEbU@mail.gmail.com>

On Sun, Feb 6, 2011 at 12:45 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 04.02.2011 19:50:
>> On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel wrote:
>>>> I am a bit concerned about the performance overhead of the Py_UCS4 to
>>>> Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
>>>> maybe that's both uncommon and negligible.
>>>
>>> I think so. If users deal with Py_UNICODE explicitly, they'll likely type
>>> their respective variables anyway, so that there won't be an intermediate
>>> step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all.
>
> Coming back to this once more: if the PEP gets implemented, we will only
> know at C compile time (Py>=3.3 or not) if the result of indexing
> (including for-loop iteration) is Py_UCS4 or Py_UNICODE. For Cython's type
> inference, Py_UCS4 is therefore the more correct guess. So my proposal
> stands to always infer Py_UCS4 instead of Py_UNICODE for indexing, even if
> we ignore surrogate pairs in narrow Python builds.
>
> I will implement this for now, so that we can see what it gives.

Yes, that makes sense.

>>>> Also, this would be inconsistant with
>>>> python-level slicing, indexing, and range, right?
>>>
>>> Yes, it does not match well with slicing and indexing. That's the problem
>>> with narrow builds in both CPython and Cython. Only the PEP can fix that by
>>> basically dropping the restrictions of a narrow build.
>>
>> Lets let indexing do what indexing does.
>
> Ok. So you'd continue to get whatever CPython returns for indexing, i.e.
> Py_UNICODE in Py<=3.2 and Py_UCS4 in Python versions that implement the
> PEP. That includes separate code points for surrogate pairs on narrow builds.

Yep, exactly. Note that indexing taking into account surrogate pairs
can be O(n) rather than O(1) as well.

- Robert

From stefan_ml at behnel.de  Sun Feb  6 12:25:55 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 06 Feb 2011 12:25:55 +0100
Subject: [Cython] Switching from Py_UNICODE to Py_UCS4
In-Reply-To: <AANLkTi=Hsxc6YFWGr=hS+64Qx7pNGTprWs_FMKLdYEbU@mail.gmail.com>
References: <4D43C3D5.8070506@behnel.de>	<AANLkTim=cmwjt5aqHPtsZ2bf3=ZPB_AURjwYQ_Q6Mcxs@mail.gmail.com>	<4D43ED77.1070201@behnel.de>	<AANLkTikak5S+K9YMR1T5f09TLMDM40yYWpxbgUmJSnGS@mail.gmail.com>	<4D4E5FAB.7020707@behnel.de>
	<AANLkTi=Hsxc6YFWGr=hS+64Qx7pNGTprWs_FMKLdYEbU@mail.gmail.com>
Message-ID: <4D4E8543.5000900@behnel.de>

Robert Bradshaw, 06.02.2011 10:14:
> On Sun, Feb 6, 2011 at 12:45 AM, Stefan Behnel<stefan_ml at behnel.de>  wrote:
>> Robert Bradshaw, 04.02.2011 19:50:
>>> On Sat, Jan 29, 2011 at 2:35 AM, Stefan Behnel wrote:
>>>>> I am a bit concerned about the performance overhead of the Py_UCS4 to
>>>>> Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but
>>>>> maybe that's both uncommon and negligible.
>>>>
>>>> I think so. If users deal with Py_UNICODE explicitly, they'll likely type
>>>> their respective variables anyway, so that there won't be an intermediate
>>>> step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all.
>>
>> Coming back to this once more: if the PEP gets implemented, we will only
>> know at C compile time (Py>=3.3 or not) if the result of indexing
>> (including for-loop iteration) is Py_UCS4 or Py_UNICODE. For Cython's type
>> inference, Py_UCS4 is therefore the more correct guess. So my proposal
>> stands to always infer Py_UCS4 instead of Py_UNICODE for indexing, even if
>> we ignore surrogate pairs in narrow Python builds.
>>
>> I will implement this for now, so that we can see what it gives.
>
> Yes, that makes sense.

Done.


>>>>> Also, this would be inconsistant with
>>>>> python-level slicing, indexing, and range, right?
>>>>
>>>> Yes, it does not match well with slicing and indexing. That's the problem
>>>> with narrow builds in both CPython and Cython. Only the PEP can fix that by
>>>> basically dropping the restrictions of a narrow build.
>>>
>>> Lets let indexing do what indexing does.
>>
>> Ok. So you'd continue to get whatever CPython returns for indexing, i.e.
>> Py_UNICODE in Py<=3.2 and Py_UCS4 in Python versions that implement the
>> PEP. That includes separate code points for surrogate pairs on narrow builds.
>
> Yep, exactly. Note that indexing taking into account surrogate pairs
> can be O(n) rather than O(1) as well.

Sure, that was almost certainly the reason why the way indexing works 
wasn't changed when surrogate pair support was implemented in the codecs, 
in print etc.

Stefan

From vitja.makarov at gmail.com  Tue Feb  8 10:16:12 2011
From: vitja.makarov at gmail.com (Vitja Makarov)
Date: Tue, 8 Feb 2011 12:16:12 +0300
Subject: [Cython] Problem with generate_stararg_init_code()
Message-ID: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>

Trying to merge latest changes in argument parsing code I found that
it still uses direct returns

https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624

Like this:
        if self.starstar_arg:
            self.starstar_arg.entry.xdecref_cleanup = 0
            code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % (
                    self.starstar_arg.entry.cname,
                    self.starstar_arg.entry.cname,
                    self.error_value()))
            code.put_gotref(self.starstar_arg.entry.cname)
Or this:
            if self.starstar_arg:
                code.putln("")
                code.putln("if (unlikely(!%s)) {" % self.star_arg.entry.cname)
                code.put_decref_clear(self.starstar_arg.entry.cname,
py_object_type)
                code.putln('return %s;' % self.error_value())
                code.putln('}')
            else:
                code.putln("if (unlikely(!%s)) return %s;" % (
                        self.star_arg.entry.cname, self.error_value()))

That's not good because current scope and refnanny context is already
created and should be freed.

-- 
vitja.

From stefan_ml at behnel.de  Tue Feb  8 11:21:55 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 08 Feb 2011 11:21:55 +0100
Subject: [Cython] Problem with generate_stararg_init_code()
In-Reply-To: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>
References: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>
Message-ID: <4D511943.6070408@behnel.de>

Vitja Makarov, 08.02.2011 10:16:
> Trying to merge latest changes in argument parsing code I found that
> it still uses direct returns
>
> https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624
>
> Like this:
>          if self.starstar_arg:
>              self.starstar_arg.entry.xdecref_cleanup = 0
>              code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % (
>                      self.starstar_arg.entry.cname,
>                      self.starstar_arg.entry.cname,
>                      self.error_value()))
>              code.put_gotref(self.starstar_arg.entry.cname)
> Or this:
>              if self.starstar_arg:
>                  code.putln("")
>                  code.putln("if (unlikely(!%s)) {" % self.star_arg.entry.cname)
>                  code.put_decref_clear(self.starstar_arg.entry.cname,
> py_object_type)
>                  code.putln('return %s;' % self.error_value())
>                  code.putln('}')
>              else:
>                  code.putln("if (unlikely(!%s)) return %s;" % (
>                          self.star_arg.entry.cname, self.error_value()))
>
> That's not good because current scope and refnanny context is already
> created and should be freed.

These aren't really critical bugs as they only deal with memory problems. 
Unless you want to rework them now, I think this is something that we 
should clean up as part of the DefNode/CFuncDefNode refactoring during the 
workshop.

Stefan

From vitja.makarov at gmail.com  Tue Feb  8 11:28:13 2011
From: vitja.makarov at gmail.com (Vitja Makarov)
Date: Tue, 8 Feb 2011 13:28:13 +0300
Subject: [Cython] Problem with generate_stararg_init_code()
In-Reply-To: <4D511943.6070408@behnel.de>
References: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>
	<4D511943.6070408@behnel.de>
Message-ID: <AANLkTik2zyVTVsA-NGdRyDFPFeRQJ6D6SnogXM2wVQAS@mail.gmail.com>

2011/2/8 Stefan Behnel <stefan_ml at behnel.de>:
> Vitja Makarov, 08.02.2011 10:16:
>> Trying to merge latest changes in argument parsing code I found that
>> it still uses direct returns
>>
>> https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624
>>
>> Like this:
>> ? ? ? ? ?if self.starstar_arg:
>> ? ? ? ? ? ? ?self.starstar_arg.entry.xdecref_cleanup = 0
>> ? ? ? ? ? ? ?code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % (
>> ? ? ? ? ? ? ? ? ? ? ?self.starstar_arg.entry.cname,
>> ? ? ? ? ? ? ? ? ? ? ?self.starstar_arg.entry.cname,
>> ? ? ? ? ? ? ? ? ? ? ?self.error_value()))
>> ? ? ? ? ? ? ?code.put_gotref(self.starstar_arg.entry.cname)
>> Or this:
>> ? ? ? ? ? ? ?if self.starstar_arg:
>> ? ? ? ? ? ? ? ? ?code.putln("")
>> ? ? ? ? ? ? ? ? ?code.putln("if (unlikely(!%s)) {" % self.star_arg.entry.cname)
>> ? ? ? ? ? ? ? ? ?code.put_decref_clear(self.starstar_arg.entry.cname,
>> py_object_type)
>> ? ? ? ? ? ? ? ? ?code.putln('return %s;' % self.error_value())
>> ? ? ? ? ? ? ? ? ?code.putln('}')
>> ? ? ? ? ? ? ?else:
>> ? ? ? ? ? ? ? ? ?code.putln("if (unlikely(!%s)) return %s;" % (
>> ? ? ? ? ? ? ? ? ? ? ? ? ?self.star_arg.entry.cname, self.error_value()))
>>
>> That's not good because current scope and refnanny context is already
>> created and should be freed.
>
> These aren't really critical bugs as they only deal with memory problems.
> Unless you want to rework them now, I think this is something that we
> should clean up as part of the DefNode/CFuncDefNode refactoring during the
> workshop.
>

Ok.

-- 
vitja.

From robertwb at math.washington.edu  Tue Feb  8 22:01:45 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 8 Feb 2011 13:01:45 -0800
Subject: [Cython] Problem with generate_stararg_init_code()
In-Reply-To: <4D511943.6070408@behnel.de>
References: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>
	<4D511943.6070408@behnel.de>
Message-ID: <AANLkTikMXOopu-OR=5LsJTXrE5SAyDiAU7OnX+tv1We=@mail.gmail.com>

On Tue, Feb 8, 2011 at 2:21 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Vitja Makarov, 08.02.2011 10:16:
>> Trying to merge latest changes in argument parsing code I found that
>> it still uses direct returns
>>
>> https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624
>>
>> Like this:
>> ? ? ? ? ?if self.starstar_arg:
>> ? ? ? ? ? ? ?self.starstar_arg.entry.xdecref_cleanup = 0
>> ? ? ? ? ? ? ?code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % (
>> ? ? ? ? ? ? ? ? ? ? ?self.starstar_arg.entry.cname,
>> ? ? ? ? ? ? ? ? ? ? ?self.starstar_arg.entry.cname,
>> ? ? ? ? ? ? ? ? ? ? ?self.error_value()))
>> ? ? ? ? ? ? ?code.put_gotref(self.starstar_arg.entry.cname)
>> Or this:
>> ? ? ? ? ? ? ?if self.starstar_arg:
>> ? ? ? ? ? ? ? ? ?code.putln("")
>> ? ? ? ? ? ? ? ? ?code.putln("if (unlikely(!%s)) {" % self.star_arg.entry.cname)
>> ? ? ? ? ? ? ? ? ?code.put_decref_clear(self.starstar_arg.entry.cname,
>> py_object_type)
>> ? ? ? ? ? ? ? ? ?code.putln('return %s;' % self.error_value())
>> ? ? ? ? ? ? ? ? ?code.putln('}')
>> ? ? ? ? ? ? ?else:
>> ? ? ? ? ? ? ? ? ?code.putln("if (unlikely(!%s)) return %s;" % (
>> ? ? ? ? ? ? ? ? ? ? ? ? ?self.star_arg.entry.cname, self.error_value()))
>>
>> That's not good because current scope and refnanny context is already
>> created and should be freed.
>
> These aren't really critical bugs as they only deal with memory problems.
> Unless you want to rework them now, I think this is something that we
> should clean up as part of the DefNode/CFuncDefNode refactoring during the
> workshop.

They are certainly bugs, so please file a ticket or add a note in the
code at least.

- Robert

From stefan_ml at behnel.de  Wed Feb  9 17:04:32 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 09 Feb 2011 17:04:32 +0100
Subject: [Cython] Problem with generate_stararg_init_code()
In-Reply-To: <AANLkTikMXOopu-OR=5LsJTXrE5SAyDiAU7OnX+tv1We=@mail.gmail.com>
References: <AANLkTi=3KB3x8Rie=wjfP1YbH+2y8FGszuhj=4d9RAtu@mail.gmail.com>	<4D511943.6070408@behnel.de>
	<AANLkTikMXOopu-OR=5LsJTXrE5SAyDiAU7OnX+tv1We=@mail.gmail.com>
Message-ID: <4D52BB10.50903@behnel.de>

Robert Bradshaw, 08.02.2011 22:01:
> On Tue, Feb 8, 2011 at 2:21 AM, Stefan Behnel wrote:
>> Vitja Makarov, 08.02.2011 10:16:
>>> Trying to merge latest changes in argument parsing code I found that
>>> it still uses direct returns
>>>
>>> https://github.com/cython/cython/blob/master/Cython/Compiler/Nodes.py#L2624
>>>
>>> Like this:
>>>           if self.starstar_arg:
>>>               self.starstar_arg.entry.xdecref_cleanup = 0
>>>               code.putln('%s = PyDict_New(); if (unlikely(!%s)) return %s;' % (
>>>                       self.starstar_arg.entry.cname,
>>>                       self.starstar_arg.entry.cname,
>>>                       self.error_value()))
>>>               code.put_gotref(self.starstar_arg.entry.cname)
>>> Or this:
>>>               if self.starstar_arg:
>>>                   code.putln("")
>>>                   code.putln("if (unlikely(!%s)) {" % self.star_arg.entry.cname)
>>>                   code.put_decref_clear(self.starstar_arg.entry.cname,
>>> py_object_type)
>>>                   code.putln('return %s;' % self.error_value())
>>>                   code.putln('}')
>>>               else:
>>>                   code.putln("if (unlikely(!%s)) return %s;" % (
>>>                           self.star_arg.entry.cname, self.error_value()))
>>>
>>> That's not good because current scope and refnanny context is already
>>> created and should be freed.
>>
>> These aren't really critical bugs as they only deal with memory problems.
>> Unless you want to rework them now, I think this is something that we
>> should clean up as part of the DefNode/CFuncDefNode refactoring during the
>> workshop.
>
> They are certainly bugs, so please file a ticket or add a note in the
> code at least.

I agree that they are bugs. The (safe) quick fix (that could go into 
0.14.2) would be to insert code to clean up the refnanny and the closure 
context before the return statements. This can be cleaned up for 0.15.

Stefan

From animator333 at yahoo.com  Thu Feb 10 09:13:59 2011
From: animator333 at yahoo.com (Prashant Saxena)
Date: Thu, 10 Feb 2011 13:43:59 +0530 (IST)
Subject: [Cython] cython+pure python mode+mac+universal binary concept
Message-ID: <974437.75266.qm@web94906.mail.in2.yahoo.com>

Hi,

I have successfully built C-extensions using pure python mode on windows and 
linux platform. Now I have port same app
on Mac. (Leopard 10.5.5 Intel, Xcode 3.1.4). 

Q. When creating c-extensions using cython on mac (details above) are there any 
specific instructions to follow to make
sure that extensions work on both architecture (ppc and i686)? I would be using 
universal build of python as well as wxpython.

Cheers




From stefan_ml at behnel.de  Thu Feb 10 09:56:43 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 10 Feb 2011 09:56:43 +0100
Subject: [Cython] Fwd: [codespeak-ann] termination of codespeak.net services
 end FEB 2011
Message-ID: <4D53A84B.9020601@behnel.de>

Whoopsa ...

-------- Original-Nachricht --------
Betreff: [codespeak-ann] termination of codespeak.net services end FEB 2011
Datum: Thu, 10 Feb 2011 09:45:09 +0100
Von: holger krekel <holger(at)merlinux.eu>
An: codespeak-ann(at)codespeak.net

hi codespeak.net users, (sorry if you get mail twice, i wanted to make sure 
...)

after 8 years of operation codespeak.net services are bound to terminate, 
starting

     END OF FEBRUARY 2011

Background: one of the original codespeak purposes was to offer subversion 
(then
in version 0.17) for the PyPy and other projects but today this is not too
interesting given the pletora of VCS hosting solutions.  Also, there aren't too
many admins besides me, the hosting is costing money, PyPy's repository has
moved to Bitbucket and i am re-shuffling my priorities preparing for my soon to
emerge father-hood.  After February 2011 i probably won't be able to help
much with any transition issues or questions. The host will keep on running for
a while but i give no guaruantees.

Some remarks regarding termination wrt to the FEB 2011 deadline:

* the subversion repo will turn read-only (and will eventually be switched 
off).

* Shell accounts will be restricted to those people who need it *and* mail
   me about it.  Some time later they will be gone as well.

* Mailing lists will be terminated as well unless i get a mail asking
   me to postpone termination for a specific time. You can go to your 
respective
   mailman admin page and extract a list of members.  If you mail me i can 
also
   provide a list of members.

* Any remaining web docs/pages will probably continue to exist for a while
   but i also prefer them to be moved away by end Feb 2011.

Note that the codespeak svn repository contains a lot of projects.  For 
migration
you have two options: do a flat import just of your project checkout 
directory into
a new version system.  This is super-simple, obviously.  If you want to 
preserve
history for your project please mail me and i either provide you a full dump or
a filtered dump only containing your project.

So long and I hope you all had a good time and enjoyed the services and also
have a good transition now.

see you in other places,
holger krekel

From stefan_ml at behnel.de  Thu Feb 10 10:01:09 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 10 Feb 2011 10:01:09 +0100
Subject: [Cython] cython+pure python mode+mac+universal binary concept
In-Reply-To: <974437.75266.qm@web94906.mail.in2.yahoo.com>
References: <974437.75266.qm@web94906.mail.in2.yahoo.com>
Message-ID: <4D53A955.9060307@behnel.de>

Prashant Saxena, 10.02.2011 09:13:
> I have successfully built C-extensions using pure python mode on windows and
> linux platform. Now I have port same app
> on Mac. (Leopard 10.5.5 Intel, Xcode 3.1.4).
>
> Q. When creating c-extensions using cython on mac (details above) are there any
> specific instructions to follow to make
> sure that extensions work on both architecture (ppc and i686)? I would be using
> universal build of python as well as wxpython.

Hi,

note that this is a question about distutils, not so much about Cython. In 
any case, the Cython core developers list is the wrong place to discuss this.

I think you should ask your question on the general Python list (i.e. the 
comp.lang.python newsgroup).

Stefan

From dagss at student.matnat.uio.no  Thu Feb 10 10:08:13 2011
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Thu, 10 Feb 2011 10:08:13 +0100
Subject: [Cython] Fwd: [codespeak-ann] termination of codespeak.net
 services end FEB 2011
In-Reply-To: <4D53A84B.9020601@behnel.de>
References: <4D53A84B.9020601@behnel.de>
Message-ID: <4D53AAFD.4000108@student.matnat.uio.no>

(I think top-posting is OK in this one instance...)

How about librelist.com? Here's a blog post with some more background:

http://zedshaw.com/blog/2009-12-03.html

I don't think Google Groups is really up to scratch yet, or?

Dag Sverre


On 02/10/2011 09:56 AM, Stefan Behnel wrote:
> Whoopsa ...
>
> -------- Original-Nachricht --------
> Betreff: [codespeak-ann] termination of codespeak.net services end FEB 2011
> Datum: Thu, 10 Feb 2011 09:45:09 +0100
> Von: holger krekel<holger(at)merlinux.eu>
> An: codespeak-ann(at)codespeak.net
>
> hi codespeak.net users, (sorry if you get mail twice, i wanted to make sure
> ...)
>
> after 8 years of operation codespeak.net services are bound to terminate,
> starting
>
>       END OF FEBRUARY 2011
>
> Background: one of the original codespeak purposes was to offer subversion
> (then
> in version 0.17) for the PyPy and other projects but today this is not too
> interesting given the pletora of VCS hosting solutions.  Also, there aren't too
> many admins besides me, the hosting is costing money, PyPy's repository has
> moved to Bitbucket and i am re-shuffling my priorities preparing for my soon to
> emerge father-hood.  After February 2011 i probably won't be able to help
> much with any transition issues or questions. The host will keep on running for
> a while but i give no guaruantees.
>
> Some remarks regarding termination wrt to the FEB 2011 deadline:
>
> * the subversion repo will turn read-only (and will eventually be switched
> off).
>
> * Shell accounts will be restricted to those people who need it *and* mail
>     me about it.  Some time later they will be gone as well.
>
> * Mailing lists will be terminated as well unless i get a mail asking
>     me to postpone termination for a specific time. You can go to your
> respective
>     mailman admin page and extract a list of members.  If you mail me i can
> also
>     provide a list of members.
>
> * Any remaining web docs/pages will probably continue to exist for a while
>     but i also prefer them to be moved away by end Feb 2011.
>
> Note that the codespeak svn repository contains a lot of projects.  For
> migration
> you have two options: do a flat import just of your project checkout
> directory into
> a new version system.  This is super-simple, obviously.  If you want to
> preserve
> history for your project please mail me and i either provide you a full dump or
> a filtered dump only containing your project.
>
> So long and I hope you all had a good time and enjoyed the services and also
> have a good transition now.
>
> see you in other places,
> holger krekel
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>    


From stefan_ml at behnel.de  Thu Feb 10 10:39:38 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 10 Feb 2011 10:39:38 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <4D53AAFD.4000108@student.matnat.uio.no>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
Message-ID: <4D53B25A.2010101@behnel.de>

Dag Sverre Seljebotn, 10.02.2011 10:08:
> (I think top-posting is OK in this one instance...)

You're excused. ;)


> How about librelist.com? Here's a blog post with some more background:
>
> http://zedshaw.com/blog/2009-12-03.html

I like their philosophy and I like their archives. I also like that you can 
actually *get* their archives. Maybe we can even manage to drop our 
existing archive there, but it seems like we'd have to ask if/how that 
would work.

They do seem to have a couple of lists there already, including some Python 
or Ruby related lists. That doesn't guarantee librelist.com is there to 
stay forever, but I wouldn't mind giving them a try.

BTW, we should first set up the list, then re-subscribe gmane, then 
announce the new list to have people subscribe (or bulk subscribe them, in 
case that works).

Stefan

From dalcinl at gmail.com  Thu Feb 10 16:44:46 2011
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 10 Feb 2011 12:44:46 -0300
Subject: [Cython] cython+pure python mode+mac+universal binary concept
In-Reply-To: <974437.75266.qm@web94906.mail.in2.yahoo.com>
References: <974437.75266.qm@web94906.mail.in2.yahoo.com>
Message-ID: <AANLkTimPDno9z12JBB6j225MUkyxwuC=JGXxb0G7nsUH@mail.gmail.com>

On 10 February 2011 05:13, Prashant Saxena <animator333 at yahoo.com> wrote:
> Hi,
>
> I have successfully built C-extensions using pure python mode on windows and
> linux platform. Now I have port same app
> on Mac. (Leopard 10.5.5 Intel, Xcode 3.1.4).
>
> Q. When creating c-extensions using cython on mac (details above) are there any
> specific instructions to follow to make
> sure that extensions work on both architecture (ppc and i686)? I would be using
> universal build of python as well as wxpython.
>

If you use a universal build of Python, then distutils should pass
multiple -arch flags to the C compiler and linker, and you are done.
You should not need anything special, it should work out of the box
(as long as any C library your extensions depends on are also
universal binaries)

From stefan_ml at behnel.de  Thu Feb 10 18:59:40 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 10 Feb 2011 18:59:40 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <4D53B25A.2010101@behnel.de>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
Message-ID: <4D54278C.5010000@behnel.de>

Stefan Behnel, 10.02.2011 10:39:
> I also like that you can
> actually *get* their archives. Maybe we can even manage to drop our
> existing archive there, but it seems like we'd have to ask if/how that
> would work.

I asked: no import support.

Stefan

From robertwb at math.washington.edu  Thu Feb 10 20:30:57 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Thu, 10 Feb 2011 11:30:57 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <4D53B25A.2010101@behnel.de>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
Message-ID: <AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>

On Thu, Feb 10, 2011 at 1:39 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Dag Sverre Seljebotn, 10.02.2011 10:08:
>> (I think top-posting is OK in this one instance...)
>
> You're excused. ;)
>
>
>> How about librelist.com? Here's a blog post with some more background:
>>
>> http://zedshaw.com/blog/2009-12-03.html
>
> I like their philosophy and I like their archives. I also like that you can
> actually *get* their archives.

Yes, this is nice.

> Maybe we can even manage to drop our
> existing archive there, but it seems like we'd have to ask if/how that
> would work.
>
> They do seem to have a couple of lists there already, including some Python
> or Ruby related lists. That doesn't guarantee librelist.com is there to
> stay forever, but I wouldn't mind giving them a try.
>
> BTW, we should first set up the list, then re-subscribe gmane, then
> announce the new list to have people subscribe (or bulk subscribe them, in
> case that works).

Google groups works well for my workflow, but I know you don't like
it. I could go with librelist. The interface seems nice and clean and
it's fast enough. Most of the lists it hosts are empty or dead... Do
you know what the moderation options are like? Hopefully they'll be
around for a while, as this is the 2nd (3rd?) time we've had to move
lists...

- Robert

From stefan_ml at behnel.de  Fri Feb 11 13:57:56 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Feb 2011 13:57:56 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
Message-ID: <4D553254.4030909@behnel.de>

Robert Bradshaw, 10.02.2011 20:30:
> On Thu, Feb 10, 2011 at 1:39 AM, Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 10.02.2011 10:08:
>>> How about librelist.com? Here's a blog post with some more background:
>>>
>>> http://zedshaw.com/blog/2009-12-03.html
>>
>> I like their philosophy and I like their archives. I also like that you can
>> actually *get* their archives.
>
> Yes, this is nice.
>
>> They do seem to have a couple of lists there already, including some Python
>> or Ruby related lists. That doesn't guarantee librelist.com is there to
>> stay forever, but I wouldn't mind giving them a try.
>>
>> BTW, we should first set up the list, then re-subscribe gmane, then
>> announce the new list to have people subscribe (or bulk subscribe them, in
>> case that works).
>
> Google groups works well for my workflow, but I know you don't like
> it. I could go with librelist. The interface seems nice and clean and
> it's fast enough. Most of the lists it hosts are empty or dead... Do
> you know what the moderation options are like? Hopefully they'll be
> around for a while, as this is the 2nd (3rd?) time we've had to move
> lists...

One caveat I found: you can't name a list "cython-dev at librelist.com", 
because they don't allow hyphens in list names (reserved as command 
separators). They only allow dots as special characters, which is somewhat 
unusual IMHO. Then the list would become "cython.dev at librelist.com", or 
"cythondev at ..." or simply "cython at ...". The latter would be fine with me, 
given that we have a separate cython-users mailing list anyway.

Stefan

From robertwb at math.washington.edu  Fri Feb 11 19:00:41 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 11 Feb 2011 10:00:41 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <4D553254.4030909@behnel.de>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
	<4D553254.4030909@behnel.de>
Message-ID: <AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>

On Fri, Feb 11, 2011 at 4:57 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 10.02.2011 20:30:
>> On Thu, Feb 10, 2011 at 1:39 AM, Stefan Behnel wrote:
>>> Dag Sverre Seljebotn, 10.02.2011 10:08:
>>>> How about librelist.com? Here's a blog post with some more background:
>>>>
>>>> http://zedshaw.com/blog/2009-12-03.html
>>>
>>> I like their philosophy and I like their archives. I also like that you can
>>> actually *get* their archives.
>>
>> Yes, this is nice.
>>
>>> They do seem to have a couple of lists there already, including some Python
>>> or Ruby related lists. That doesn't guarantee librelist.com is there to
>>> stay forever, but I wouldn't mind giving them a try.
>>>
>>> BTW, we should first set up the list, then re-subscribe gmane, then
>>> announce the new list to have people subscribe (or bulk subscribe them, in
>>> case that works).
>>
>> Google groups works well for my workflow, but I know you don't like
>> it. I could go with librelist. The interface seems nice and clean and
>> it's fast enough. Most of the lists it hosts are empty or dead... Do
>> you know what the moderation options are like? Hopefully they'll be
>> around for a while, as this is the 2nd (3rd?) time we've had to move
>> lists...
>
> One caveat I found: you can't name a list "cython-dev at librelist.com",
> because they don't allow hyphens in list names (reserved as command
> separators). They only allow dots as special characters, which is somewhat
> unusual IMHO. Then the list would become "cython.dev at librelist.com", or
> "cythondev at ..." or simply "cython at ...". The latter would be fine with me,
> given that we have a separate cython-users mailing list anyway.

I think we should definitely have dev or devel in the name. Very
strange that they don't allow a hyphen: not a blocker, but as we have
not commitment yet it's worth keeping our eyes open for an equivalent
service that does.

- Robert

From njs at pobox.com  Fri Feb 11 19:25:08 2011
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 11 Feb 2011 10:25:08 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
	<4D553254.4030909@behnel.de>
	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
Message-ID: <AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>

On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
<robertwb at math.washington.edu> wrote:
> I think we should definitely have dev or devel in the name. Very
> strange that they don't allow a hyphen: not a blocker, but as we have
> not commitment yet it's worth keeping our eyes open for an equivalent
> service that does.

Of course you could also just find mailman hosting somewhere -- I host
some lists myself, and it works well and people are familiar with it.
It's also easy to import archives[1] and the default archive pages
have links to download the raw archive. Perhaps the scipy.org folks
would be willing to help? They already host a number of lists, not
just for scipy[2].

[1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
[2] http://mail.scipy.org/mailman/listinfo

-- Nathaniel

From stefan_ml at behnel.de  Fri Feb 11 19:33:42 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Feb 2011 19:33:42 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>	<4D553254.4030909@behnel.de>	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
Message-ID: <4D558106.5010902@behnel.de>

Nathaniel Smith, 11.02.2011 19:25:
> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw wrote:
>> I think we should definitely have dev or devel in the name. Very
>> strange that they don't allow a hyphen: not a blocker, but as we have
>> not commitment yet it's worth keeping our eyes open for an equivalent
>> service that does.
>
> Of course you could also just find mailman hosting somewhere -- I host
> some lists myself, and it works well and people are familiar with it.
> It's also easy to import archives[1] and the default archive pages
> have links to download the raw archive. Perhaps the scipy.org folks
> would be willing to help? They already host a number of lists, not
> just for scipy[2].
>
> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
> [2] http://mail.scipy.org/mailman/listinfo

That's a good idea. SciPy is related enough to ask them, and I doubt that 
scipy.org is going away any time soon.

Stefan

From jpe at wingware.com  Fri Feb 11 19:23:37 2011
From: jpe at wingware.com (John Ehresman)
Date: Fri, 11 Feb 2011 13:23:37 -0500
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>	<4D553254.4030909@behnel.de>
	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
Message-ID: <4D557EA9.3030700@wingware.com>

On 2/11/11 1:00 PM, Robert Bradshaw wrote:
> I think we should definitely have dev or devel in the name. Very
> strange that they don't allow a hyphen: not a blocker, but as we have
> not commitment yet it's worth keeping our eyes open for an equivalent
> service that does.

This may have been discussed, but is python.org a possibility?

John

From robertwb at math.washington.edu  Fri Feb 11 19:50:31 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 11 Feb 2011 10:50:31 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
	<4D553254.4030909@behnel.de>
	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
Message-ID: <AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>

On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith <njs at pobox.com> wrote:
> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
> <robertwb at math.washington.edu> wrote:
>> I think we should definitely have dev or devel in the name. Very
>> strange that they don't allow a hyphen: not a blocker, but as we have
>> not commitment yet it's worth keeping our eyes open for an equivalent
>> service that does.
>
> Of course you could also just find mailman hosting somewhere -- I host
> some lists myself, and it works well and people are familiar with it.
> It's also easy to import archives[1] and the default archive pages
> have links to download the raw archive. Perhaps the scipy.org folks
> would be willing to help? They already host a number of lists, not
> just for scipy[2].
>
> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
> [2] http://mail.scipy.org/mailman/listinfo

I like that idea a lot better than going with a random, un-related
list hosting service. (I had thought about python.org too, would that
be an appropriate fit?) SciPy is certainly very related, but I don't
want to pigeon-hole ourselves as being only for scientific computation
(even if this is our biggest userbase).

- Robert

From stefan_ml at behnel.de  Fri Feb 11 19:58:27 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Feb 2011 19:58:27 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>	<4D553254.4030909@behnel.de>	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
	<AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>
Message-ID: <4D5586D3.3020605@behnel.de>

Robert Bradshaw, 11.02.2011 19:50:
> On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
>> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
>> <robertwb at math.washington.edu>  wrote:
>>> I think we should definitely have dev or devel in the name. Very
>>> strange that they don't allow a hyphen: not a blocker, but as we have
>>> not commitment yet it's worth keeping our eyes open for an equivalent
>>> service that does.
>>
>> Of course you could also just find mailman hosting somewhere -- I host
>> some lists myself, and it works well and people are familiar with it.
>> It's also easy to import archives[1] and the default archive pages
>> have links to download the raw archive. Perhaps the scipy.org folks
>> would be willing to help? They already host a number of lists, not
>> just for scipy[2].
>>
>> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
>> [2] http://mail.scipy.org/mailman/listinfo
>
> I like that idea a lot better than going with a random, un-related
> list hosting service. (I had thought about python.org too, would that
> be an appropriate fit?) SciPy is certainly very related, but I don't
> want to pigeon-hole ourselves as being only for scientific computation
> (even if this is our biggest userbase).

:) You read my mind here.

However, my first thought about "python.org" was that "python-dev" and 
"cython-dev" are pretty close, especially given that "python" is often 
referred to as "cpython". This may trigger some confusion.

Stefan

From robertwb at math.washington.edu  Fri Feb 11 20:02:13 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 11 Feb 2011 11:02:13 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <4D5586D3.3020605@behnel.de>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
	<4D553254.4030909@behnel.de>
	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
	<AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>
	<4D5586D3.3020605@behnel.de>
Message-ID: <AANLkTimKjOQLWPzKoxB+G4n3RKCM6Kwcr7aJ3spYzfmv@mail.gmail.com>

On Fri, Feb 11, 2011 at 10:58 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 11.02.2011 19:50:
>> On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
>>> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
>>> <robertwb at math.washington.edu> ?wrote:
>>>> I think we should definitely have dev or devel in the name. Very
>>>> strange that they don't allow a hyphen: not a blocker, but as we have
>>>> not commitment yet it's worth keeping our eyes open for an equivalent
>>>> service that does.
>>>
>>> Of course you could also just find mailman hosting somewhere -- I host
>>> some lists myself, and it works well and people are familiar with it.
>>> It's also easy to import archives[1] and the default archive pages
>>> have links to download the raw archive. Perhaps the scipy.org folks
>>> would be willing to help? They already host a number of lists, not
>>> just for scipy[2].
>>>
>>> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
>>> [2] http://mail.scipy.org/mailman/listinfo
>>
>> I like that idea a lot better than going with a random, un-related
>> list hosting service. (I had thought about python.org too, would that
>> be an appropriate fit?) SciPy is certainly very related, but I don't
>> want to pigeon-hole ourselves as being only for scientific computation
>> (even if this is our biggest userbase).
>
> :) You read my mind here.
>
> However, my first thought about "python.org" was that "python-dev" and
> "cython-dev" are pretty close, especially given that "python" is often
> referred to as "cpython". This may trigger some confusion.

We could go with cython-devel. The fact that the first letter is
different is more comforting to me than a different suffix in these
days of address auto-completion. Should I go ahead and ask?

- Robert

From stefan_ml at behnel.de  Fri Feb 11 20:06:15 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Feb 2011 20:06:15 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTimKjOQLWPzKoxB+G4n3RKCM6Kwcr7aJ3spYzfmv@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>	<4D553254.4030909@behnel.de>	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>	<AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>	<4D5586D3.3020605@behnel.de>
	<AANLkTimKjOQLWPzKoxB+G4n3RKCM6Kwcr7aJ3spYzfmv@mail.gmail.com>
Message-ID: <4D5588A7.8090504@behnel.de>

Robert Bradshaw, 11.02.2011 20:02:
> On Fri, Feb 11, 2011 at 10:58 AM, Stefan Behnel wrote:
>> Robert Bradshaw, 11.02.2011 19:50:
>>> On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
>>>> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
>>>> <robertwb at math.washington.edu>    wrote:
>>>>> I think we should definitely have dev or devel in the name. Very
>>>>> strange that they don't allow a hyphen: not a blocker, but as we have
>>>>> not commitment yet it's worth keeping our eyes open for an equivalent
>>>>> service that does.
>>>>
>>>> Of course you could also just find mailman hosting somewhere -- I host
>>>> some lists myself, and it works well and people are familiar with it.
>>>> It's also easy to import archives[1] and the default archive pages
>>>> have links to download the raw archive. Perhaps the scipy.org folks
>>>> would be willing to help? They already host a number of lists, not
>>>> just for scipy[2].
>>>>
>>>> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
>>>> [2] http://mail.scipy.org/mailman/listinfo
>>>
>>> I like that idea a lot better than going with a random, un-related
>>> list hosting service. (I had thought about python.org too, would that
>>> be an appropriate fit?) SciPy is certainly very related, but I don't
>>> want to pigeon-hole ourselves as being only for scientific computation
>>> (even if this is our biggest userbase).
>>
>> :) You read my mind here.
>>
>> However, my first thought about "python.org" was that "python-dev" and
>> "cython-dev" are pretty close, especially given that "python" is often
>> referred to as "cpython". This may trigger some confusion.
>
> We could go with cython-devel. The fact that the first letter is
> different is more comforting to me than a different suffix in these
> days of address auto-completion. Should I go ahead and ask?

+1

Stefan

From robertwb at math.washington.edu  Fri Feb 11 20:22:00 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 11 Feb 2011 11:22:00 -0800
Subject: [Cython] Cython mailing list
Message-ID: <AANLkTinv4AUem2+zWMActdcSw2joOj0S6LdVOtCAwrMT@mail.gmail.com>

Python mailman admins,

The Cython project (http://cython.org) is in need of a new mailing
list, as our current hosting provider is discontinuing services. We
were thinking it would make sense to host it at mail.python.org. Would
you be willing to provide us with cython-devel at python.org?

Thanks,
Robert

From stefan_ml at behnel.de  Fri Feb 11 20:44:49 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 11 Feb 2011 20:44:49 +0100
Subject: [Cython] Fwd: [codespeak-ann] termination of codespeak.net
 services end FEB 2011
In-Reply-To: <4D53AAFD.4000108@student.matnat.uio.no>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
Message-ID: <4D5591B1.5040603@behnel.de>

Dag Sverre Seljebotn, 10.02.2011 10:08:
> How about librelist.com? Here's a blog post with some more background:
>
> http://zedshaw.com/blog/2009-12-03.html

Ok, here's a good reason *not* to use librelist.com:

http://librelist.com/browser/meta/2011/2/11/hyphens-in-list-names/

(note that the archives are still catching up, I left the last words to Zed 
Shaw here)

Stefan

From robertwb at math.washington.edu  Fri Feb 11 22:42:31 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Fri, 11 Feb 2011 13:42:31 -0800
Subject: [Cython] switching mailing lists
In-Reply-To: <4D5588A7.8090504@behnel.de>
References: <4D53A84B.9020601@behnel.de>
	<4D53AAFD.4000108@student.matnat.uio.no>
	<4D53B25A.2010101@behnel.de>
	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>
	<4D553254.4030909@behnel.de>
	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>
	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>
	<AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>
	<4D5586D3.3020605@behnel.de>
	<AANLkTimKjOQLWPzKoxB+G4n3RKCM6Kwcr7aJ3spYzfmv@mail.gmail.com>
	<4D5588A7.8090504@behnel.de>
Message-ID: <AANLkTi=33mr+O+DuUWw5wKHpCsnT72in6kFuhwF=CWjm@mail.gmail.com>

On Fri, Feb 11, 2011 at 11:06 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Robert Bradshaw, 11.02.2011 20:02:
>> On Fri, Feb 11, 2011 at 10:58 AM, Stefan Behnel wrote:
>>> Robert Bradshaw, 11.02.2011 19:50:
>>>> On Fri, Feb 11, 2011 at 10:25 AM, Nathaniel Smith wrote:
>>>>> On Fri, Feb 11, 2011 at 10:00 AM, Robert Bradshaw
>>>>> <robertwb at math.washington.edu> ? ?wrote:
>>>>>> I think we should definitely have dev or devel in the name. Very
>>>>>> strange that they don't allow a hyphen: not a blocker, but as we have
>>>>>> not commitment yet it's worth keeping our eyes open for an equivalent
>>>>>> service that does.
>>>>>
>>>>> Of course you could also just find mailman hosting somewhere -- I host
>>>>> some lists myself, and it works well and people are familiar with it.
>>>>> It's also easy to import archives[1] and the default archive pages
>>>>> have links to download the raw archive. Perhaps the scipy.org folks
>>>>> would be willing to help? They already host a number of lists, not
>>>>> just for scipy[2].
>>>>>
>>>>> [1] http://wiki.list.org/pages/viewpage.action?pageId=4030624
>>>>> [2] http://mail.scipy.org/mailman/listinfo
>>>>
>>>> I like that idea a lot better than going with a random, un-related
>>>> list hosting service. (I had thought about python.org too, would that
>>>> be an appropriate fit?) SciPy is certainly very related, but I don't
>>>> want to pigeon-hole ourselves as being only for scientific computation
>>>> (even if this is our biggest userbase).
>>>
>>> :) You read my mind here.
>>>
>>> However, my first thought about "python.org" was that "python-dev" and
>>> "cython-dev" are pretty close, especially given that "python" is often
>>> referred to as "cpython". This may trigger some confusion.
>>
>> We could go with cython-devel. The fact that the first letter is
>> different is more comforting to me than a different suffix in these
>> days of address auto-completion. Should I go ahead and ask?
>
> +1

We now have cython-devel at python.org

- Robert

From stefan_ml at behnel.de  Sat Feb 12 06:33:52 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 12 Feb 2011 06:33:52 +0100
Subject: [Cython] switching mailing lists
In-Reply-To: <AANLkTi=33mr+O+DuUWw5wKHpCsnT72in6kFuhwF=CWjm@mail.gmail.com>
References: <4D53A84B.9020601@behnel.de>	<4D53AAFD.4000108@student.matnat.uio.no>	<4D53B25A.2010101@behnel.de>	<AANLkTinAxv3Xc5Nw5riKH6c-8pgVx=t_mj-5OcqpMyD2@mail.gmail.com>	<4D553254.4030909@behnel.de>	<AANLkTi=qx5hywEPOSrJn6QSkr8yvUsq-8440n5YjcK0G@mail.gmail.com>	<AANLkTi=aPH4ZiWXqaozcr52DYO9har7z5WG-An283jPK@mail.gmail.com>	<AANLkTikXzhNtGbfz5rXk15u7f+0MNacVVkx3kaxRvOH1@mail.gmail.com>	<4D5586D3.3020605@behnel.de>	<AANLkTimKjOQLWPzKoxB+G4n3RKCM6Kwcr7aJ3spYzfmv@mail.gmail.com>	<4D5588A7.8090504@behnel.de>
	<AANLkTi=33mr+O+DuUWw5wKHpCsnT72in6kFuhwF=CWjm@mail.gmail.com>
Message-ID: <4D561BC0.1020904@behnel.de>

Robert Bradshaw, 11.02.2011 22:42:
> We now have cython-devel at python.org

I mass-subscribed the current regular subscribers to the new list, and 
invited the digest subscribers.

Sorry for any inconvenience due to the mailing list switch.

The migrated subscriptions also include the archives, however, I guess that 
at least mail-archive.com will consider it a new list as it receives all 
mail under a single address (and then likely splits it up by list header 
entries). Let's see what Gmane does.

Stefan

From stefan_ml at behnel.de  Tue Feb 15 11:24:40 2011
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 15 Feb 2011 11:24:40 +0100
Subject: [Cython] Control flow graph
In-Reply-To: <AANLkTinQqc37P_VwiwpFN6+5RAT1m-3tBLh0STNCNGNN@mail.gmail.com>
References: <AANLkTin+1cDFwSpvCyROGyByL9vcaALeRfJLYzHqoK5U@mail.gmail.com>	<AANLkTikw4H6PnucBWRV3=_mQD81+dCqxH+aM+NwBK_Ci@mail.gmail.com>	<AANLkTimX+_dBNh7q0U1ym7nzohfkn2+rBbpm+gpnNp7h@mail.gmail.com>	<AANLkTimX_hnOEAspvTghCRRGLyvEOYstRWFodxpyYKwa@mail.gmail.com>	<4D5A2D6E.9060406@behnel.de>
	<AANLkTinQqc37P_VwiwpFN6+5RAT1m-3tBLh0STNCNGNN@mail.gmail.com>
Message-ID: <4D5A5468.9040502@behnel.de>

Vitja Makarov, 15.02.2011 10:59:
> 2011/2/15 Stefan Behnel<stefan_ml at behnel.de>:
>> Robert Bradshaw, 15.02.2011 08:21:
>>>
>>> On Mon, Feb 14, 2011 at 9:49 PM, Vitja Makarov wrote:
>>>>
>>>> 2011/2/15 Robert Bradshaw:
>>>>>
>>>>> On Sun, Feb 13, 2011 at 11:40 PM, Vitja Makarov wrote:
>>>>>>
>>>>>> Hi!
>>>>>>
>>>>>> In order to implement "reaching definitions" algorithm.
>>>>>> I'm now working on control-flow (or data-flow) graph.
>>>>>>
>>>>>> Here is funny picture made with graphviz ;)
>>>>>>
>>>>>> http://piccy.info/view3/1099337/ca29d7054d09bd0503cefa25f5f49420/1200/
>>>>>
>>>>> Cool. Any plans on handling exceptions?
>>>>
>>>> Sure, but I don't have much time for this :(
>>>>
>>>> Linear block inside try...except body should be split by assignments
>>>> and each subblock should point to exception handling entry point.
>>>
>>> Would every possible failing sub-expression have to point to the
>>> exception handling point(s)?
>>
>
> Not sure here as now graph node is list of NameNode ref and AssignmentNode.
> I'm not sure but it seems that every sub-expression could raise
> exception now, is there any flag?
>
>> Well, in most cases (especially the interesting ones), this will be the
>> function exit point, so it'll be easy. And in some cases, we may be able to
>> infer that a specific exception that an expression (e.g. arithmetics or a
>> 'raise' statement) can raise will not get caught by a given except clause
>> (although that's certainly a tricky optimisation).
>>
>> But in general, I think any subexpression that potentially raises an
>> exception must point to the next exception handling point.
>>
>
> Right, and each exception handling point should have reference to the
> next one or function exit point.
>
>>
>>> I suppose it depends on whether you'll be handling more than assignment
>>> tracking.
>>
>> We *may* get away with a statement-level graph in that case, but I somehow
>> doubt it already. For example, list comprehensions leak their variable in
>> Py2 code, so it's important to know if they are executed or not, and they
>> may appear in any kind of expression.
>>
>
> This should be special case for scoped expression node.
>
> Now I build graph "from scratch" it include only name node references
> and assignments, and positions marks to draw gv.
>
> May be node tree should be translated into graph and then local
> variables graph could be created from it.
>
> On the other hand CreateAbstractGraph transformation could be used to
> create specialized graph.

Note that building that graph is only a smaller part of the work. It needs 
to be queriable efficiently. These graphs can easily get huge, so if the 
graph needs to get traversed for each little piece of information, that'll 
seriously slow things down.

The current (limited) support for control flow analysis initially crashed 
for a beautiful code example that had lots of if-blocks in a row, because 
it was using recursive traversal. I refactored it back then, but he have to 
make sure the new implementation stays scalable.

It's worth reading a bit of literature here. AFAIR, I posted a couple of 
sources to the list a while back.

Stefan

From robertwb at math.washington.edu  Tue Feb 15 11:29:14 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Tue, 15 Feb 2011 02:29:14 -0800
Subject: [Cython] Control flow graph
In-Reply-To: <4D5A5468.9040502@behnel.de>
References: <AANLkTin+1cDFwSpvCyROGyByL9vcaALeRfJLYzHqoK5U@mail.gmail.com>
	<AANLkTikw4H6PnucBWRV3=_mQD81+dCqxH+aM+NwBK_Ci@mail.gmail.com>
	<AANLkTimX+_dBNh7q0U1ym7nzohfkn2+rBbpm+gpnNp7h@mail.gmail.com>
	<AANLkTimX_hnOEAspvTghCRRGLyvEOYstRWFodxpyYKwa@mail.gmail.com>
	<4D5A2D6E.9060406@behnel.de>
	<AANLkTinQqc37P_VwiwpFN6+5RAT1m-3tBLh0STNCNGNN@mail.gmail.com>
	<4D5A5468.9040502@behnel.de>
Message-ID: <AANLkTimwv7n2+B2UGag1cTU4Bhs=-2xyCyc5my2OeTHm@mail.gmail.com>

On Tue, Feb 15, 2011 at 2:24 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Vitja Makarov, 15.02.2011 10:59:
>> 2011/2/15 Stefan Behnel<stefan_ml at behnel.de>:
>>> Robert Bradshaw, 15.02.2011 08:21:
>>>>
>>>> On Mon, Feb 14, 2011 at 9:49 PM, Vitja Makarov wrote:
>>>>>
>>>>> 2011/2/15 Robert Bradshaw:
>>>>>>
>>>>>> On Sun, Feb 13, 2011 at 11:40 PM, Vitja Makarov wrote:
>>>>>>>
>>>>>>> Hi!
>>>>>>>
>>>>>>> In order to implement "reaching definitions" algorithm.
>>>>>>> I'm now working on control-flow (or data-flow) graph.
>>>>>>>
>>>>>>> Here is funny picture made with graphviz ;)
>>>>>>>
>>>>>>> http://piccy.info/view3/1099337/ca29d7054d09bd0503cefa25f5f49420/1200/
>>>>>>
>>>>>> Cool. Any plans on handling exceptions?
>>>>>
>>>>> Sure, but I don't have much time for this :(
>>>>>
>>>>> Linear block inside try...except body should be split by assignments
>>>>> and each subblock should point to exception handling entry point.
>>>>
>>>> Would every possible failing sub-expression have to point to the
>>>> exception handling point(s)?
>>>
>>
>> Not sure here as now graph node is list of NameNode ref and AssignmentNode.
>> I'm not sure but it seems that every sub-expression could raise
>> exception now, is there any flag?
>>
>>> Well, in most cases (especially the interesting ones), this will be the
>>> function exit point, so it'll be easy. And in some cases, we may be able to
>>> infer that a specific exception that an expression (e.g. arithmetics or a
>>> 'raise' statement) can raise will not get caught by a given except clause
>>> (although that's certainly a tricky optimisation).
>>>
>>> But in general, I think any subexpression that potentially raises an
>>> exception must point to the next exception handling point.
>>>
>>
>> Right, and each exception handling point should have reference to the
>> next one or function exit point.
>>
>>>
>>>> I suppose it depends on whether you'll be handling more than assignment
>>>> tracking.
>>>
>>> We *may* get away with a statement-level graph in that case, but I somehow
>>> doubt it already. For example, list comprehensions leak their variable in
>>> Py2 code, so it's important to know if they are executed or not, and they
>>> may appear in any kind of expression.
>>>
>>
>> This should be special case for scoped expression node.
>>
>> Now I build graph "from scratch" it include only name node references
>> and assignments, and positions marks to draw gv.
>>
>> May be node tree should be translated into graph and then local
>> variables graph could be created from it.
>>
>> On the other hand CreateAbstractGraph transformation could be used to
>> create specialized graph.
>
> Note that building that graph is only a smaller part of the work. It needs
> to be queriable efficiently. These graphs can easily get huge, so if the
> graph needs to get traversed for each little piece of information, that'll
> seriously slow things down.

On this note, one thought that I had is that one could traverse the
graph while holding a single state object which nodes could read from
and write to. This would alleviate the need to need to query the graph
and simultaneously store the (large) number of potential states at any
point in the code path.

> The current (limited) support for control flow analysis initially crashed
> for a beautiful code example that had lots of if-blocks in a row, because
> it was using recursive traversal. I refactored it back then, but he have to
> make sure the new implementation stays scalable.
>
> It's worth reading a bit of literature here. AFAIR, I posted a couple of
> sources to the list a while back.
>
> Stefan
> _______________________________________________
> Cython-dev mailing list
> Cython-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/cython-dev
>

From vitja.makarov at gmail.com  Tue Feb 15 11:52:07 2011
From: vitja.makarov at gmail.com (Vitja Makarov)
Date: Tue, 15 Feb 2011 13:52:07 +0300
Subject: [Cython] Control flow graph
In-Reply-To: <AANLkTimwv7n2+B2UGag1cTU4Bhs=-2xyCyc5my2OeTHm@mail.gmail.com>
References: <AANLkTin+1cDFwSpvCyROGyByL9vcaALeRfJLYzHqoK5U@mail.gmail.com>
	<AANLkTikw4H6PnucBWRV3=_mQD81+dCqxH+aM+NwBK_Ci@mail.gmail.com>
	<AANLkTimX+_dBNh7q0U1ym7nzohfkn2+rBbpm+gpnNp7h@mail.gmail.com>
	<AANLkTimX_hnOEAspvTghCRRGLyvEOYstRWFodxpyYKwa@mail.gmail.com>
	<4D5A2D6E.9060406@behnel.de>
	<AANLkTinQqc37P_VwiwpFN6+5RAT1m-3tBLh0STNCNGNN@mail.gmail.com>
	<4D5A5468.9040502@behnel.de>
	<AANLkTimwv7n2+B2UGag1cTU4Bhs=-2xyCyc5my2OeTHm@mail.gmail.com>
Message-ID: <AANLkTimScLhz7cY-k60py8jxfEJ-XQGifMoNUKnufCPN@mail.gmail.com>

2011/2/15 Robert Bradshaw <robertwb at math.washington.edu>:
> On Tue, Feb 15, 2011 at 2:24 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Vitja Makarov, 15.02.2011 10:59:
>>> 2011/2/15 Stefan Behnel<stefan_ml at behnel.de>:
>>>> Robert Bradshaw, 15.02.2011 08:21:
>>>>>
>>>>> On Mon, Feb 14, 2011 at 9:49 PM, Vitja Makarov wrote:
>>>>>>
>>>>>> 2011/2/15 Robert Bradshaw:
>>>>>>>
>>>>>>> On Sun, Feb 13, 2011 at 11:40 PM, Vitja Makarov wrote:
>>>>>>>>
>>>>>>>> Hi!
>>>>>>>>
>>>>>>>> In order to implement "reaching definitions" algorithm.
>>>>>>>> I'm now working on control-flow (or data-flow) graph.
>>>>>>>>
>>>>>>>> Here is funny picture made with graphviz ;)
>>>>>>>>
>>>>>>>> http://piccy.info/view3/1099337/ca29d7054d09bd0503cefa25f5f49420/1200/
>>>>>>>
>>>>>>> Cool. Any plans on handling exceptions?
>>>>>>
>>>>>> Sure, but I don't have much time for this :(
>>>>>>
>>>>>> Linear block inside try...except body should be split by assignments
>>>>>> and each subblock should point to exception handling entry point.
>>>>>
>>>>> Would every possible failing sub-expression have to point to the
>>>>> exception handling point(s)?
>>>>
>>>
>>> Not sure here as now graph node is list of NameNode ref and AssignmentNode.
>>> I'm not sure but it seems that every sub-expression could raise
>>> exception now, is there any flag?
>>>
>>>> Well, in most cases (especially the interesting ones), this will be the
>>>> function exit point, so it'll be easy. And in some cases, we may be able to
>>>> infer that a specific exception that an expression (e.g. arithmetics or a
>>>> 'raise' statement) can raise will not get caught by a given except clause
>>>> (although that's certainly a tricky optimisation).
>>>>
>>>> But in general, I think any subexpression that potentially raises an
>>>> exception must point to the next exception handling point.
>>>>
>>>
>>> Right, and each exception handling point should have reference to the
>>> next one or function exit point.
>>>
>>>>
>>>>> I suppose it depends on whether you'll be handling more than assignment
>>>>> tracking.
>>>>
>>>> We *may* get away with a statement-level graph in that case, but I somehow
>>>> doubt it already. For example, list comprehensions leak their variable in
>>>> Py2 code, so it's important to know if they are executed or not, and they
>>>> may appear in any kind of expression.
>>>>
>>>
>>> This should be special case for scoped expression node.
>>>
>>> Now I build graph "from scratch" it include only name node references
>>> and assignments, and positions marks to draw gv.
>>>
>>> May be node tree should be translated into graph and then local
>>> variables graph could be created from it.
>>>
>>> On the other hand CreateAbstractGraph transformation could be used to
>>> create specialized graph.
>>
>> Note that building that graph is only a smaller part of the work. It needs
>> to be queriable efficiently. These graphs can easily get huge, so if the
>> graph needs to get traversed for each little piece of information, that'll
>> seriously slow things down.
>
> On this note, one thought that I had is that one could traverse the
> graph while holding a single state object which nodes could read from
> and write to. This would alleviate the need to need to query the graph
> and simultaneously store the (large) number of potential states at any
> point in the code path.
>
>> The current (limited) support for control flow analysis initially crashed
>> for a beautiful code example that had lots of if-blocks in a row, because
>> it was using recursive traversal. I refactored it back then, but he have to
>> make sure the new implementation stays scalable.
>>
>> It's worth reading a bit of literature here. AFAIR, I posted a couple of
>> sources to the list a while back.
>>

I can push my code into unitialized branch for initial review and criticism.

It's rather dirty now but it should be good not to make already known mistakes.

-- 
vitja.

From robertwb at math.washington.edu  Thu Feb 17 00:55:19 2011
From: robertwb at math.washington.edu (Robert Bradshaw)
Date: Wed, 16 Feb 2011 15:55:19 -0800
Subject: [Cython] [cython-users] Cython .pxd introspection: listing
	defined constants
In-Reply-To: <20110216161721.GA16736@tyr.home.net>
References: <AANLkTikv_Gs2M7o1QEXQ6hfiFL=3DkaZLUQXkHr+j8vb@mail.gmail.com>
	<20110209172325.GA4661@tyr.home.net>
	<20110216161721.GA16736@tyr.home.net>
Message-ID: <AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>

On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King <wking at drexel.edu> wrote:
> First, here's a reminder of my end goal:
>
> On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote:
>> Now I want to expose those constants in Python, so I have `expose.pyx`:
>>
>> ? cimport mylib
>>
>> ? CONST_A = mylib.CONST_A
>> ? ? ? CONST_B = mylib.CONST_B
>> ? ...
>>
>> But the last part seems pretty silly. ?I'd like to do something like
>>
>> ? ? ? cimport mylib
>> ? import sys
>>
>> ? for name in dir(mylib):
>> ? ? ? setattr(sys.modules[__name__], name, getattr(mylib, name))
>>
>> which compiles fine, but fails to import with...
>
> The discussion eventually led to:
>
> On Wed, Feb 09, 2011 at 12:22:41PM -0800, Robert Bradshaw wrote:
>> cimport * will pollute expose.pyx's C-level namespace. You can use
>> them just fine there. If you need to access these constants from
>> Python, you do have to explicitly expose them.
>
> In an attempt to make it easier to explicitly expose them to other
> Python modules at compile time, I've been working through the Cython
> source, and found that when cythoning expose.pyx, all the items
> declared in mylib.pxd are stored in `Cython.Compiler.Symtab.Entry`s in
> mylib.pxd's `Cython.Compiler.Symtab.ModuleScope` instance.
>
> By tweaking
> ?Cython.Compiler.Node.CImportStatNode.generate_function_definitions
> to be
> ?def generate_function_definitions(self, env, code):
> ? ? ?modules = [m for m in env.cimported_modules if m.module_name == self.module_name]
> ? ? ?if len(modules) > 0:
> ? ? ? ? ?module = modules[0]
> ? ? ? ? ?for name,entry in module.entries.iteritems():
> ? ? ? ? ? ? ?code.putln('/* %s.%s */' % (self.module_name, name))
> I can add a comment to extern.c listing all the stuff I'd declared in
> `cdef extern` blocks in mylib.pxd. ?That's nice, but not very useful
> (it does show, however, that Cython is not "forgetting" about the
> definitions as I previously thought).
>
> What I'm missing is a way to bind the ModuleScope namespace to a name
> in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib,
> name)` will work in expose.pyx. ?If it would be easier to pre-compute
> the result of these commands and hard-code them in `extern.c` at
> compile time, that's fine.
>
> For example, my comment-generating version of
> `generate_function_definitions` is not far from being able to generate
> the results of `dir(mylib)`, it would just need to wrap the results in
> a list and place references to that list in place of the stock `dir()`
> execution code when the dir argument resolved to the cimported module.
> Yuck ;).

Cython modules have two namespaces, the Python level one and the C
level one. This separation is necessary, as the C level one may
contain objects that are unrepresentable in the Python-level layer,
and get statically bound (e.g. one would want to mark them as
read-only at the very least). Many users also like the fact that the
C-level implementation details of their Cython modules do not get
leaked out to their Python namespaces (though use of __all__ could
provide, but not enforce, similar behavior).

You have also hit into the thorny issue that .pxd files are used for
many things. They may be pure C library declarations with no Python
module backing, they may be declarations of (externally implemented)
Python modules (such as numpy.pxd), or they may be declarations for
Cython-implemented modules.

In terms of your specific question, I don't think hijacking the
builtin dir and getattr to blur this line.

> It seems like it would be easier to generate some kind of wrapper
> class (PxdModule?) for mylib when it is cimported (at compile time),
> and then further interactions would take care of themselves (at run
> time).
>
> Does anyone know how I would go about doing this?

Would such an object be created anew for every module that cimports
the declaration file?

I have toyed with the idea of subclassing the module object itself for
better support of C-level attributes from the Python (and Cython)
namespaces.

Here's another idea, what if extern blocks could contain cpdef
declarations, which would automatically generate a Python-level
wrappers for the declared members (if possible, otherwise an error)?

> On an administrative level, since this would seem to require altering
> the Cython source code, should I move this discussion to cython-devel,
> or does splitting the thread across two lists make it too hard to
> follow?

Yes, it would probably be best to move this thread over there. A
succinct summary of what issue you are trying to solve would probably
be helpful now anyways.

- Robert

From wking at drexel.edu  Thu Feb 17 14:44:24 2011
From: wking at drexel.edu (W. Trevor King)
Date: Thu, 17 Feb 2011 08:44:24 -0500
Subject: [Cython] [cython-users] Cython .pxd introspection: listing
 defined constants
In-Reply-To: <20110217132940.GA28686@tyr.home.net>
References: <AANLkTikv_Gs2M7o1QEXQ6hfiFL=3DkaZLUQXkHr+j8vb@mail.gmail.com>
	<20110209172325.GA4661@tyr.home.net>
	<20110216161721.GA16736@tyr.home.net>
	<AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>
	<20110217132940.GA28686@tyr.home.net>
Message-ID: <20110217134424.GA9353@tyr.home.net>

On Thu, Feb 17, 2011 at 08:29:41AM -0500, W. Trevor King wrote:
>     cpdef struct Foo:
>         cpdef public int intA
>         cpdef readonly int intB
>         cdef void *ptr

Oops, for consistency with classes, the variables declarations should
read `cdef public` and and `cdef readonly`.  Perhaps `cdef struct`
too, to match `cdef class`?

I get a bit confused, because for some things (functions, methods)
`cpdef` adds a Python interface.  For others (attributes) it's `cdef
public/readonly`.  There are even some things (classes), where a plain
`cdef` is enough to provide a Python interface.  Perhaps I am just
missing some subtle distinction between the effects of the various
incantations?

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20110217/34ec21e3/attachment.pgp 

From dalcinl at gmail.com  Thu Feb 17 14:53:15 2011
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Thu, 17 Feb 2011 10:53:15 -0300
Subject: [Cython] [cython-users] Cython .pxd introspection: listing
 defined constants
In-Reply-To: <20110217134424.GA9353@tyr.home.net>
References: <AANLkTikv_Gs2M7o1QEXQ6hfiFL=3DkaZLUQXkHr+j8vb@mail.gmail.com>
	<20110209172325.GA4661@tyr.home.net>
	<20110216161721.GA16736@tyr.home.net>
	<AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>
	<20110217132940.GA28686@tyr.home.net>
	<20110217134424.GA9353@tyr.home.net>
Message-ID: <AANLkTi=06=t-Q84MC_2ZfThAN+vZqF_YppWyVYUBHKLC@mail.gmail.com>

On 17 February 2011 10:44, W. Trevor King <wking at drexel.edu> wrote:
> On Thu, Feb 17, 2011 at 08:29:41AM -0500, W. Trevor King wrote:
>> ? ? cpdef struct Foo:
>> ? ? ? ? cpdef public int intA
>> ? ? ? ? cpdef readonly int intB
>> ? ? ? ? cdef void *ptr
>
> Oops, for consistency with classes, the variables declarations should
> read `cdef public` and and `cdef readonly`. ?Perhaps `cdef struct`
> too, to match `cdef class`?
>
> I get a bit confused, because for some things (functions, methods)
> `cpdef` adds a Python interface. ?For others (attributes) it's `cdef
> public/readonly`. ?There are even some things (classes), where a plain
> `cdef` is enough to provide a Python interface. ?Perhaps I am just
> missing some subtle distinction between the effects of the various
> incantations?
>

Cython could certainly support "cpdef struct", it is just a matter to
define a proposal and find a contributor to implement it :-)




-- 
Lisandro Dalcin
---------------
CIMEC (INTEC/CONICET-UNL)
Predio CONICET-Santa Fe
Colectora RN 168 Km 472, Paraje El Pozo
3000 Santa Fe, Argentina
Tel: +54-342-4511594 (ext 1011)
Tel/Fax: +54-342-4511169

From wking at drexel.edu  Thu Feb 17 14:29:41 2011
From: wking at drexel.edu (W. Trevor King)
Date: Thu, 17 Feb 2011 08:29:41 -0500
Subject: [Cython] [cython-users] Cython .pxd introspection: listing
 defined constants
In-Reply-To: <AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>
References: <AANLkTikv_Gs2M7o1QEXQ6hfiFL=3DkaZLUQXkHr+j8vb@mail.gmail.com>
	<20110209172325.GA4661@tyr.home.net>
	<20110216161721.GA16736@tyr.home.net>
	<AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>
Message-ID: <20110217132940.GA28686@tyr.home.net>

This thread is coming over to cython-dev (and the new cython-devel)
from cython-users because it turns out it will probably require
chaning the Cython code.  To get everyone who hasn't been following on
cython-users up to speed, here's a summary of what I'm trying to do:

That's what I was trying to give with this:

On Wed, Feb 09, 2011 at 12:23:25PM -0500, W. Trevor King wrote:
> I'm wrapping an external C library with Cython, so I have `mylib.pxd`:
>
>   cdef extern from 'mylib.h'
>       enum: CONST_A
>       enum: CONST_B
>       ...
>
> where I declare each constant macro from the library's header `mylib.h`:
>
>   #define CONST_A 1
>   #define CONST_B 2
>   ...
>
> Now I want to expose those constants in Python, so I have `expose.pyx`:
>
>   cimport mylib
>
>   CONST_A = mylib.CONST_A
>   CONST_B = mylib.CONST_B
>   ...
>
> But the last part seems pretty silly.  I'd like to do something like
>
>   cimport mylib
>   import sys
>
>   for name in dir(mylib):
>       setattr(sys.modules[__name__], name, getattr(mylib, name))
>
> which compiles fine, but fails to import with...

Looking into the Cython internals, everything defined in mylib.pxd is
stored as `Entry`s in a `ModuleScope`, and...

On Wed, Feb 16, 2011 at 03:55:19PM -0800, Robert Bradshaw wrote:
> On Wed, Feb 16, 2011 at 8:17 AM, W. Trevor King <wking at drexel.edu> wrote:
> > What I'm missing is a way to bind the ModuleScope namespace to a name
> > in expose.pyx so that commands like `dir(mylib)` and `getattr(mylib,
> > name)` will work in expose.pyx.
>
> You have also hit into the thorny issue that .pxd files are used for
> many things. They may be pure C library declarations with no Python
> module backing, they may be declarations of (externally implemented)
> Python modules (such as numpy.pxd), or they may be declarations for
> Cython-implemented modules.

> > It seems like it would be easier to generate some kind of wrapper
> > class (PxdModule?) for mylib when it is cimported (at compile time),
> > and then further interactions would take care of themselves (at run
> > time).
> 
> Would such an object be created anew for every module that cimports
> the declaration file?

Hmm, That doesn't sound very nice, does it.  However, .pxd files
declaring C libraries have no Python-space presence, so that was my
initial idea.

> I have toyed with the idea of subclassing the module object itself for
> better support of C-level attributes from the Python (and Cython)
> namespaces.

Sorry, I don't understand "better support of C-level attributes".  Can
you give an example?

> Here's another idea, what if extern blocks could contain cpdef
> declarations, which would automatically generate a Python-level
> wrappers for the declared members (if possible, otherwise an error)?

Ah, this sounds good!  Of the three .pxd roles you list above,
external Python modules (e.g. numpy) and Cython-implemented modules
(e.g. matched .pxd/.pyx) both already have a presence in Python-space.
What's missing is a way to give (where possible) declarations of
external C libraries a Python presence.  cpdef fills this hole nicely,
since its whole purpose is to expose Python interfaces to
C-based elements.

A side effect of this cpdef change would be that now even bare .pxd
files (no matching .pyx) would have a Python presence, so You could do
something like

   cimport mylib as mylib_c
   import mylib as mylib_py
   import sys

   # Access through Python
   for name in dir(mylib_py):
       setattr(sys.modules[__name__], name, getattr(mylib_py, name))

   # Direct C access
   cdef get_a():
       return mylib_c.CONST_A

Where the Python access would be the new feature, and list all
cpdef-ed stuff.  However, from Parsing.py:2369:

    error(pos, "C struct/union/enum cannot be declared cpdef")

From pyrex_differences.rst:

    If a function is declared :keyword:`cpdef` it can be called from
    and overridden by both extension and normal python subclasses.

I believe the reason that cpdef-ed enums and similar are currently
illegal is confusion between "can be called from Python" and "can be
overridden from Python".  I think these should be just like methods
already are, in that you can "override" a method by subclassing it,
but not by rebinding the name in the base class:

    >>> import pyximport; pyximport.install()
    >>> import rectangle as R
    >>> r = R.Rectangle(1, 2, 3, 4)
    >>> r.area = lambda(self): r.x1'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    AttributeError: 'rectangle.Rectangle' object attribute 'area' is read-only

Where rectangle.pyx is a minorly patched version of the last example
from early_binding_for_speed.rst [1] and `area` is a cpdef-ed method.

Why can't enums share this handling, with the enum taking the place of
the method and the enum's module taking the place of the class?  After
all, enums have a Python-side tyoe (int or long).  Unions don't really
have a Python parallel, but structs do, so long as you can select
which attributes should have (writable) Python interfaces.  If we
change the struct declaration syntax to be closer to the `cdef class`
declaration syntax:

    cpdef struct Foo:
        cpdef public int intA
        cpdef readonly int intB
        cdef void *ptr

We would both declare the important members of the C struct (as we can
already do in Cython) and also have Cython automatically generate a
Python class wrapping the struct (because of `cpdef struct`).  The
Python class would have:

* Cython-generated getter/setter for intA (because of `cpdef public`)
  using the standard Python<->int coercion.
* Similar Cython-generated getter for int B (because of `cpdef
  readonly`).
* No Python access to ptr (standard C-access still possible through
  Cython).

Doing something crazy like `cdef public void *ptr` would raise a
compile-time error.

I'm definately willing to help out with this (if someone will point me
in the right direction), as the enum stuff would fix my original
problem, and the struct stuff would allow me to rip out a bunch of
boilerplate like

    cdef class Foo (object):
        cdef mylib.Foo _Foo
    
        def _intA_get(self):
            return self._Foo.intA
        def _intA_set(self, value):
            self._Foo.intA = value
        intA = property(fget=_intA_get, fset=_intA_set)
    
        def _intB_get(self):
            return self._Foo.intB
        intB = property(fget=_intB_get)

from my wrapper code.

Thanks,
Trevor

[1] While testing my overriding method example, I found a small typo
in cython-docs' early_binding_for_speed.rst.  Patch attached.

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
From 80a52b3c0224e73a969b88fb414b6d026029a85e Mon Sep 17 00:00:00 2001
From: W. Trevor King <wking at drexel.edu>
Date: Thu, 17 Feb 2011 07:57:11 -0500
Subject: [PATCH] `int` -> `cdef int` when declaring local variables in early binding example.

---
 src/userguide/early_binding_for_speed.rst |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/userguide/early_binding_for_speed.rst b/src/userguide/early_binding_for_speed.rst
index 07e0047..d44498d 100644
--- a/src/userguide/early_binding_for_speed.rst
+++ b/src/userguide/early_binding_for_speed.rst
@@ -53,7 +53,7 @@ where calls occur within Cython code. For example:
         def __init__(self, int x0, int y0, int x1, int y1):
             self.x0 = x0; self.y0 = y0; self.x1 = x1; self.y1 = y1
         cdef int _area(self):
-            int area
+            cdef int area
             area = (self.x1 - self.x0) * (self.y1 - self.y0)
             if area < 0:
                 area = -area
@@ -88,7 +88,7 @@ overheads. Consider this code:
         def __init__(self, int x0, int y0, int x1, int y1):
             self.x0 = x0; self.y0 = y0; self.x1 = x1; self.y1 = y1
         cpdef int area(self):
-            int area
+            cdef int area
             area = (self.x1 - self.x0) * (self.y1 - self.y0)
             if area < 0:
                 area = -area
-- 
1.7.3.4

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20110217/83dfed52/attachment-0001.pgp 

From wking at drexel.edu  Thu Feb 17 15:35:22 2011
From: wking at drexel.edu (W. Trevor King)
Date: Thu, 17 Feb 2011 09:35:22 -0500
Subject: [Cython] [cython-users] Cython .pxd introspection: listing
 defined constants
In-Reply-To: <AANLkTi=06=t-Q84MC_2ZfThAN+vZqF_YppWyVYUBHKLC@mail.gmail.com>
References: <AANLkTikv_Gs2M7o1QEXQ6hfiFL=3DkaZLUQXkHr+j8vb@mail.gmail.com>
	<20110209172325.GA4661@tyr.home.net>
	<20110216161721.GA16736@tyr.home.net>
	<AANLkTi=__WiequnRKK_DCX1toZDgvr662qUWvUh7mOQ4@mail.gmail.com>
	<20110217132940.GA28686@tyr.home.net>
	<20110217134424.GA9353@tyr.home.net>
	<AANLkTi=06=t-Q84MC_2ZfThAN+vZqF_YppWyVYUBHKLC@mail.gmail.com>
Message-ID: <20110217143521.GB14083@tyr.home.net>

On Thu, Feb 17, 2011 at 10:53:15AM -0300, Lisandro Dalcin wrote:
> Cython could certainly support "cpdef struct", it is just a matter to
> define a proposal and find a contributor to implement it :-)

Is there a CEP template (a la PEPs 9 and 12) that should be discussed
on the mailing list, or do I develop it free-form on the wiki [2]?

p.s. should I be picking one of cython-dev at codespeak or
cython-devel at python?  Is the shift not yet official?

[2]: http://wiki.cython.org/enhancements/

-- 
This email may be signed or encrypted with GPG (http://www.gnupg.org).
The GPG signature (if present) will be attached as 'signature.asc'.
For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy

My public key is at http://www.physics.drexel.edu/~wking/pubkey.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: not available
Url : http://codespeak.net/pipermail/cython-dev/attachments/20110217/b4ac169a/attachment.pgp