From stefan_ml at behnel.de Mon Jan 3 08:41:39 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 03 Jan 2011 08:41:39 +0100 Subject: [Cython] Fixing #602 - type inference for byte string literals Message-ID: <4D217DB3.7010209@behnel.de> Hi, I've been working on a fix for ticket #602, negative indexing for inferred char*. http://trac.cython.org/cython_trac/ticket/602 Currently, when you write this: s = b'abc' s is inferred as char*. This has several drawbacks. For one, we loose the length information, so "len(s)" becomes O(n) instead of O(1). Negative indexing fails completely because it will use pointer arithmetic, thus leaving the allocated memory area of the string. Also, code like the following is extremely inefficient because it requires multiple conversions from a char* of unknown length to a Python bytes object: s = b'abc' a = s1 + s b = s2 + s I came to the conclusion that the right fix is to stop letting byte string literals start off as char*. This immediately fixes these issues and improves Python compatibility while still allowing automatic coercion, but it also comes with its own drawbacks. In nogil blocks, you will have to explicitly declare a variable as char* when assigning a byte string literal to it, otherwise you'd get a compile time error for a Python object assignment. I think this is a minor issue as most users would declare their variables anyway when using nogil blocks. Given that there isn't much you can do with a Python string inside of a nogil block, we could also honour nogil blocks during type inference and automatically infer char* for literals here. I don't think it would hurt anyone to do that. The second drawback is that it impacts type inference for char loops. Previously, you could write s = b'abc' for c in s: print c and Cython would infer 'char' for c and print integer byte values. When s is inferred as 'bytes', c will be inferred as 'Python object' because Python 2 returns 1-byte strings and Python 3 returns integers on iteration. Thus the loop will run entirely in Python code and return different things in Py2 and Py3. I do not expect that this is a major issue either. Iteration over literals should be rare, after all, and if the byte string is constructed in any way, the type either becomes a bytes object through Python operations (like concatenation) or is explicitly provided, e.g. as a return type of a function call. But it is a clear behavioural change for the type inference in an area where Cython's (and Python's) semantics are tricky anyway. Personally, I think that the advantages outweigh the disadvantages here. Most common use cases won't notice the change because coercion will not be impacted, and most existing code (IMHO) either uses explicit typing or expects a Python bytes object anyway. So my preferred change would be to make byte string literals 'bytes' by default, except in nogil blocks. Opinions? Stefan From dalcinl at gmail.com Mon Jan 3 13:01:57 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 3 Jan 2011 09:01:57 -0300 Subject: [Cython] Fixing #602 - type inference for byte string literals In-Reply-To: <4D217DB3.7010209@behnel.de> References: <4D217DB3.7010209@behnel.de> Message-ID: On 3 January 2011 04:41, Stefan Behnel wrote: > Hi, > > I've been working on a fix for ticket #602, negative indexing for inferred > char*. > > http://trac.cython.org/cython_trac/ticket/602 > > Currently, when you write this: > > ? ? s = b'abc' > > s is inferred as char*. This has several drawbacks. For one, we loose the > length information, so "len(s)" becomes O(n) instead of O(1). Negative > indexing fails completely because it will use pointer arithmetic, thus > leaving the allocated memory area of the string. Also, code like the > following is extremely inefficient because it requires multiple conversions > from a char* of unknown length to a Python bytes object: > > ? ? s = b'abc' > ? ? a = s1 + s > ? ? b = s2 + s > > I came to the conclusion that the right fix is to stop letting byte string > literals start off as char*. This immediately fixes these issues and > improves Python compatibility while still allowing automatic coercion, but > it also comes with its own drawbacks. > > In nogil blocks, you will have to explicitly declare a variable as char* > when assigning a byte string literal to it, otherwise you'd get a compile > time error for a Python object assignment. I think this is a minor issue as > most users would declare their variables anyway when using nogil blocks. > Given that there isn't much you can do with a Python string inside of a > nogil block, we could also honour nogil blocks during type inference and > automatically infer char* for literals here. I don't think it would hurt > anyone to do that. > > The second drawback is that it impacts type inference for char loops. > Previously, you could write > > ? ? s = b'abc' > ? ? for c in s: > ? ? ? ? print c > > and Cython would infer 'char' for c and print integer byte values. When s > is inferred as 'bytes', c will be inferred as 'Python object' because > Python 2 returns 1-byte strings and Python 3 returns integers on iteration. > Thus the loop will run entirely in Python code and return different things > in Py2 and Py3. > > I do not expect that this is a major issue either. Iteration over literals > should be rare, after all, and if the byte string is constructed in any > way, the type either becomes a bytes object through Python operations (like > concatenation) or is explicitly provided, e.g. as a return type of a > function call. But it is a clear behavioural change for the type inference > in an area where Cython's (and Python's) semantics are tricky anyway. > > Personally, I think that the advantages outweigh the disadvantages here. > Most common use cases won't notice the change because coercion will not be > impacted, and most existing code (IMHO) either uses explicit typing or > expects a Python bytes object anyway. So my preferred change would be to > make byte string literals 'bytes' by default, except in nogil blocks. > > Opinions? > +1 -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From robertwb at math.washington.edu Mon Jan 3 18:35:49 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 3 Jan 2011 09:35:49 -0800 Subject: [Cython] Fixing #602 - type inference for byte string literals In-Reply-To: References: <4D217DB3.7010209@behnel.de> Message-ID: On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin wrote: > On 3 January 2011 04:41, Stefan Behnel wrote: >> Hi, >> >> I've been working on a fix for ticket #602, negative indexing for inferred >> char*. >> >> http://trac.cython.org/cython_trac/ticket/602 >> >> Currently, when you write this: >> >> ? ? s = b'abc' >> >> s is inferred as char*. This has several drawbacks. For one, we loose the >> length information, so "len(s)" becomes O(n) instead of O(1). Negative >> indexing fails completely because it will use pointer arithmetic, thus >> leaving the allocated memory area of the string. Also, code like the >> following is extremely inefficient because it requires multiple conversions >> from a char* of unknown length to a Python bytes object: >> >> ? ? s = b'abc' >> ? ? a = s1 + s >> ? ? b = s2 + s >> >> I came to the conclusion that the right fix is to stop letting byte string >> literals start off as char*. This immediately fixes these issues and >> improves Python compatibility while still allowing automatic coercion, but >> it also comes with its own drawbacks. >> >> In nogil blocks, you will have to explicitly declare a variable as char* >> when assigning a byte string literal to it, otherwise you'd get a compile >> time error for a Python object assignment. I think this is a minor issue as >> most users would declare their variables anyway when using nogil blocks. >> Given that there isn't much you can do with a Python string inside of a >> nogil block, we could also honour nogil blocks during type inference and >> automatically infer char* for literals here. I don't think it would hurt >> anyone to do that. >> >> The second drawback is that it impacts type inference for char loops. >> Previously, you could write >> >> ? ? s = b'abc' >> ? ? for c in s: >> ? ? ? ? print c >> >> and Cython would infer 'char' for c and print integer byte values. When s >> is inferred as 'bytes', c will be inferred as 'Python object' because >> Python 2 returns 1-byte strings and Python 3 returns integers on iteration. >> Thus the loop will run entirely in Python code and return different things >> in Py2 and Py3. >> >> I do not expect that this is a major issue either. Iteration over literals >> should be rare, after all, and if the byte string is constructed in any >> way, the type either becomes a bytes object through Python operations (like >> concatenation) or is explicitly provided, e.g. as a return type of a >> function call. But it is a clear behavioural change for the type inference >> in an area where Cython's (and Python's) semantics are tricky anyway. >> >> Personally, I think that the advantages outweigh the disadvantages here. >> Most common use cases won't notice the change because coercion will not be >> impacted, and most existing code (IMHO) either uses explicit typing or >> expects a Python bytes object anyway. So my preferred change would be to >> make byte string literals 'bytes' by default, except in nogil blocks. >> >> Opinions? >> > > +1 > +1 I might say it should even be required in nogil blocks for consistency. - Robert From dagss at student.matnat.uio.no Mon Jan 3 19:10:13 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Mon, 03 Jan 2011 19:10:13 +0100 Subject: [Cython] Fixing #602 - type inference for byte string literals In-Reply-To: References: <4D217DB3.7010209@behnel.de> Message-ID: <4D221105.5000703@student.matnat.uio.no> On 01/03/2011 06:35 PM, Robert Bradshaw wrote: > On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin wrote: > >> On 3 January 2011 04:41, Stefan Behnel wrote: >> >>> Hi, >>> >>> I've been working on a fix for ticket #602, negative indexing for inferred >>> char*. >>> >>> http://trac.cython.org/cython_trac/ticket/602 >>> >>> Currently, when you write this: >>> >>> s = b'abc' >>> >>> s is inferred as char*. This has several drawbacks. For one, we loose the >>> length information, so "len(s)" becomes O(n) instead of O(1). Negative >>> indexing fails completely because it will use pointer arithmetic, thus >>> leaving the allocated memory area of the string. Also, code like the >>> following is extremely inefficient because it requires multiple conversions >>> from a char* of unknown length to a Python bytes object: >>> >>> s = b'abc' >>> a = s1 + s >>> b = s2 + s >>> >>> I came to the conclusion that the right fix is to stop letting byte string >>> literals start off as char*. This immediately fixes these issues and >>> improves Python compatibility while still allowing automatic coercion, but >>> it also comes with its own drawbacks. >>> >>> In nogil blocks, you will have to explicitly declare a variable as char* >>> when assigning a byte string literal to it, otherwise you'd get a compile >>> time error for a Python object assignment. I think this is a minor issue as >>> most users would declare their variables anyway when using nogil blocks. >>> Given that there isn't much you can do with a Python string inside of a >>> nogil block, we could also honour nogil blocks during type inference and >>> automatically infer char* for literals here. I don't think it would hurt >>> anyone to do that. >>> >>> The second drawback is that it impacts type inference for char loops. >>> Previously, you could write >>> >>> s = b'abc' >>> for c in s: >>> print c >>> >>> and Cython would infer 'char' for c and print integer byte values. When s >>> is inferred as 'bytes', c will be inferred as 'Python object' because >>> Python 2 returns 1-byte strings and Python 3 returns integers on iteration. >>> Thus the loop will run entirely in Python code and return different things >>> in Py2 and Py3. >>> >>> I do not expect that this is a major issue either. Iteration over literals >>> should be rare, after all, and if the byte string is constructed in any >>> way, the type either becomes a bytes object through Python operations (like >>> concatenation) or is explicitly provided, e.g. as a return type of a >>> function call. But it is a clear behavioural change for the type inference >>> in an area where Cython's (and Python's) semantics are tricky anyway. >>> >>> Personally, I think that the advantages outweigh the disadvantages here. >>> Most common use cases won't notice the change because coercion will not be >>> impacted, and most existing code (IMHO) either uses explicit typing or >>> expects a Python bytes object anyway. So my preferred change would be to >>> make byte string literals 'bytes' by default, except in nogil blocks. >>> >>> Opinions? >>> >>> >> +1 >> >> > +1 I might say it should even be required in nogil blocks for consistency. > +1 to not making nogil blocks a special case, the disadvantage of another special case to remember outweighs the advantage of syntactic brevity IMO. Dag Sverre From no-reply at dropboxmail.com Mon Jan 3 23:05:09 2011 From: no-reply at dropboxmail.com (Dropbox) Date: Mon, 03 Jan 2011 22:05:09 +0000 Subject: [Cython] Wing Sit invited you to Dropbox Message-ID: <20110103220509.06FE93F529D@mailman-2.dropboxmail.com> Wing Sit wants you to use Dropbox to sync and share files online and across computers. Get started here: http://www.dropbox.com/link/20.1N9bhcFieX/NjU2NTA1ODc2Nw?src=referrals_bulk2 - The Dropbox Team ____________________________________________________ To stop receiving invites from Dropbox, please go to http://www.dropbox.com/bl/c18ae65ddb5e/cython-dev%40codespeak.net -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110103/b62ce160/attachment.htm From stefan_ml at behnel.de Tue Jan 4 08:45:44 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 04 Jan 2011 08:45:44 +0100 Subject: [Cython] Fixing #602 - type inference for byte string literals In-Reply-To: <4D221105.5000703@student.matnat.uio.no> References: <4D217DB3.7010209@behnel.de> <4D221105.5000703@student.matnat.uio.no> Message-ID: <4D22D028.2000809@behnel.de> Dag Sverre Seljebotn, 03.01.2011 19:10: > On 01/03/2011 06:35 PM, Robert Bradshaw wrote: >> On Mon, Jan 3, 2011 at 4:01 AM, Lisandro Dalcin wrote: >> >>> On 3 January 2011 04:41, Stefan Behnel wrote: >>> >>>> Hi, >>>> >>>> I've been working on a fix for ticket #602, negative indexing for inferred >>>> char*. >>>> >>>> http://trac.cython.org/cython_trac/ticket/602 >>>> >>>> Currently, when you write this: >>>> >>>> s = b'abc' >>>> >>>> s is inferred as char*. This has several drawbacks. For one, we loose the >>>> length information, so "len(s)" becomes O(n) instead of O(1). Negative >>>> indexing fails completely because it will use pointer arithmetic, thus >>>> leaving the allocated memory area of the string. Also, code like the >>>> following is extremely inefficient because it requires multiple conversions >>>> from a char* of unknown length to a Python bytes object: >>>> >>>> s = b'abc' >>>> a = s1 + s >>>> b = s2 + s >>>> >>>> I came to the conclusion that the right fix is to stop letting byte string >>>> literals start off as char*. This immediately fixes these issues and >>>> improves Python compatibility while still allowing automatic coercion, but >>>> it also comes with its own drawbacks. >>>> >>>> In nogil blocks, you will have to explicitly declare a variable as char* >>>> when assigning a byte string literal to it, otherwise you'd get a compile >>>> time error for a Python object assignment. I think this is a minor issue as >>>> most users would declare their variables anyway when using nogil blocks. >>>> Given that there isn't much you can do with a Python string inside of a >>>> nogil block, we could also honour nogil blocks during type inference and >>>> automatically infer char* for literals here. I don't think it would hurt >>>> anyone to do that. >>>> >>>> The second drawback is that it impacts type inference for char loops. >>>> Previously, you could write >>>> >>>> s = b'abc' >>>> for c in s: >>>> print c >>>> >>>> and Cython would infer 'char' for c and print integer byte values. When s >>>> is inferred as 'bytes', c will be inferred as 'Python object' because >>>> Python 2 returns 1-byte strings and Python 3 returns integers on iteration. >>>> Thus the loop will run entirely in Python code and return different things >>>> in Py2 and Py3. >>>> >>>> I do not expect that this is a major issue either. Iteration over literals >>>> should be rare, after all, and if the byte string is constructed in any >>>> way, the type either becomes a bytes object through Python operations (like >>>> concatenation) or is explicitly provided, e.g. as a return type of a >>>> function call. But it is a clear behavioural change for the type inference >>>> in an area where Cython's (and Python's) semantics are tricky anyway. >>>> >>>> Personally, I think that the advantages outweigh the disadvantages here. >>>> Most common use cases won't notice the change because coercion will not be >>>> impacted, and most existing code (IMHO) either uses explicit typing or >>>> expects a Python bytes object anyway. So my preferred change would be to >>>> make byte string literals 'bytes' by default, except in nogil blocks. >>>> >>> +1 >>> >> +1 I might say it should even be required in nogil blocks for consistency. > > +1 to not making nogil blocks a special case, the disadvantage of > another special case to remember outweighs the advantage of syntactic > brevity IMO. Ok, then it's the proposed change without a special case for nogil. That's better anyway because I just noticed that type inference doesn't know about nogil environments. They are not determined before the subsequent type analysis step. https://github.com/cython/cython/commit/342eb45a2fd19869273ec038144c71ac6e49db0e Stefan From daniel.norberg at gmail.com Thu Jan 6 12:20:52 2011 From: daniel.norberg at gmail.com (Daniel Norberg) Date: Thu, 6 Jan 2011 12:20:52 +0100 Subject: [Cython] Cython 0.14 Bugreport Message-ID: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> Hi folks, I'm getting a curious error in Cython 0.14 when trying to compile this: def bar(foo): qux = foo quux = foo[qux.baz] The error message: $ cython bar.py Error compiling Cython file: ------------------------------------------------------------ ... def bar(foo): qux = foo quux = foo[qux.baz] ^ ------------------------------------------------------------ /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. Regards, Daniel From stefan_ml at behnel.de Thu Jan 6 19:30:47 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 06 Jan 2011 19:30:47 +0100 Subject: [Cython] Cython 0.14 Bugreport In-Reply-To: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> References: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> Message-ID: <4D260A57.8010600@behnel.de> Daniel Norberg, 06.01.2011 12:20: > I'm getting a curious error in Cython 0.14 when trying to compile this: > > def bar(foo): > qux = foo > quux = foo[qux.baz] > > The error message: > > $ cython bar.py > > Error compiling Cython file: > ------------------------------------------------------------ > ... > def bar(foo): > qux = foo > quux = foo[qux.baz] > ^ > ------------------------------------------------------------ > > /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' > > Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. I can reproduce this. From a quick test, it seems like the type inference machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before 'qux'. Anyone interested in taking a closer look? Stefan From robertwb at math.washington.edu Thu Jan 6 19:45:22 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 6 Jan 2011 10:45:22 -0800 Subject: [Cython] Cython 0.14 Bugreport In-Reply-To: <4D260A57.8010600@behnel.de> References: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> <4D260A57.8010600@behnel.de> Message-ID: On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel wrote: > Daniel Norberg, 06.01.2011 12:20: >> I'm getting a curious error in Cython 0.14 when trying to compile this: >> >> ? ?def bar(foo): >> ? ? ? ?qux = foo >> ? ? ? ?quux = foo[qux.baz] >> >> The error message: >> >> ? ? ? $ cython bar.py >> >> ? ? ? Error compiling Cython file: >> ? ? ? ------------------------------------------------------------ >> ? ? ? ... >> ? ? ? def bar(foo): >> ? ? ? ? ? ? ? qux = foo >> ? ? ? ? ? ? ? quux = foo[qux.baz] >> ? ? ? ? ? ? ? ? ? ? ^ >> ? ? ? ------------------------------------------------------------ >> >> ? ? ? /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' >> >> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. > > I can reproduce this. From a quick test, it seems like the type inference > machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before > 'qux'. Anyone interested in taking a closer look? That shouldn't be happening, as it should know the inferred type of quux depends on the inferred type of qux. I'll take a look. - Robert From vitja.makarov at gmail.com Thu Jan 6 19:48:50 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 6 Jan 2011 21:48:50 +0300 Subject: [Cython] Cython 0.14 Bugreport In-Reply-To: References: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> <4D260A57.8010600@behnel.de> Message-ID: 2011/1/6 Robert Bradshaw : > On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel wrote: >> Daniel Norberg, 06.01.2011 12:20: >>> I'm getting a curious error in Cython 0.14 when trying to compile this: >>> >>> ? ?def bar(foo): >>> ? ? ? ?qux = foo >>> ? ? ? ?quux = foo[qux.baz] >>> >>> The error message: >>> >>> ? ? ? $ cython bar.py >>> >>> ? ? ? Error compiling Cython file: >>> ? ? ? ------------------------------------------------------------ >>> ? ? ? ... >>> ? ? ? def bar(foo): >>> ? ? ? ? ? ? ? qux = foo >>> ? ? ? ? ? ? ? quux = foo[qux.baz] >>> ? ? ? ? ? ? ? ? ? ? ^ >>> ? ? ? ------------------------------------------------------------ >>> >>> ? ? ? /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' >>> >>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. >> >> I can reproduce this. From a quick test, it seems like the type inference >> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before >> 'qux'. Anyone interested in taking a closer look? > > That shouldn't be happening, as it should know the inferred type of > quux depends on the inferred type of qux. I'll take a look. > > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > Strange thing this code compiles: def bar(foo): qux = foo xquux = foo[qux.baz] but this doesn't: def bar(foo): qux = foo aquux = foo[qux.baz] -- vitja. From robertwb at math.washington.edu Thu Jan 6 19:54:04 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 6 Jan 2011 10:54:04 -0800 Subject: [Cython] Cython 0.14 Bugreport In-Reply-To: References: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> <4D260A57.8010600@behnel.de> Message-ID: On Thu, Jan 6, 2011 at 10:48 AM, Vitja Makarov wrote: > 2011/1/6 Robert Bradshaw : >> On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel wrote: >>> Daniel Norberg, 06.01.2011 12:20: >>>> I'm getting a curious error in Cython 0.14 when trying to compile this: >>>> >>>> ? ?def bar(foo): >>>> ? ? ? ?qux = foo >>>> ? ? ? ?quux = foo[qux.baz] >>>> >>>> The error message: >>>> >>>> ? ? ? $ cython bar.py >>>> >>>> ? ? ? Error compiling Cython file: >>>> ? ? ? ------------------------------------------------------------ >>>> ? ? ? ... >>>> ? ? ? def bar(foo): >>>> ? ? ? ? ? ? ? qux = foo >>>> ? ? ? ? ? ? ? quux = foo[qux.baz] >>>> ? ? ? ? ? ? ? ? ? ? ^ >>>> ? ? ? ------------------------------------------------------------ >>>> >>>> ? ? ? /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' >>>> >>>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. >>> >>> I can reproduce this. From a quick test, it seems like the type inference >>> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before >>> 'qux'. Anyone interested in taking a closer look? >> >> That shouldn't be happening, as it should know the inferred type of >> quux depends on the inferred type of qux. I'll take a look. >> >> - Robert >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > Strange thing this code compiles: > > def bar(foo): > ? ?qux = foo > ? ?xquux = foo[qux.baz] > > but this doesn't: > > def bar(foo): > ? ?qux = foo > ? ?aquux = foo[qux.baz] Since it doesn't detect the dependency, half the time it'll get lucky (probably as a function of the variable names). - Robert From vitja.makarov at gmail.com Fri Jan 7 13:27:46 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 7 Jan 2011 15:27:46 +0300 Subject: [Cython] General generator expressions In-Reply-To: References: <4CFBD711.5060206@behnel.de> <4D00A586.8060801@behnel.de> <4D01D7EF.6060906@behnel.de> <4D02BB12.7060306@behnel.de> <4D05B116.2020900@behnel.de> <4D06930B.2090205@behnel.de> <4D077CA5.4010103@behnel.de> <4D078717.4050105@behnel.de> <4D1CC9F4.60408@behnel.de> Message-ID: 2010/12/31 Robert Bradshaw : > On Thu, Dec 30, 2010 at 10:05 AM, Stefan Behnel wrote: >> Vitja Makarov, 30.12.2010 18:53: >>> Why did you reverted decorators stuff? >>> >>> https://github.com/vitek/cython/commit/e0d366d9409680849e6f429992ac9724e2ad1016 >> >> Because I didn't get it finished, it broke the Sage build and cython-devel >> should stay cleanly working before the release. > > +1 > >> I added a link to the build >> failure log to the trac ticket but didn't even manage to come up with a >> suitable test case before I had to stop working on it. >> >> What I implemented so far works in most cases but there are issues with >> functions at the module scope. I didn't check the history but I think I >> recall from the code comments that Robert worked in this area (method >> binding) a while ago, and the comments hint at an unfinished refactoring >> that would help with this change. > > I'm not working on anything in the area at the moment though (or, > contrary to my expectations, much of anything Cython-wise... having > too much fun hanging out with the family :). > > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > I've added GenerartorDefNode and GeneratorBodyDefNode now all the generators stuff is handled there. Please review. -- vitja. From robertwb at math.washington.edu Fri Jan 7 19:17:34 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 7 Jan 2011 10:17:34 -0800 Subject: [Cython] General generator expressions In-Reply-To: References: <4CFBD711.5060206@behnel.de> <4D00A586.8060801@behnel.de> <4D01D7EF.6060906@behnel.de> <4D02BB12.7060306@behnel.de> <4D05B116.2020900@behnel.de> <4D06930B.2090205@behnel.de> <4D077CA5.4010103@behnel.de> <4D078717.4050105@behnel.de> <4D1CC9F4.60408@behnel.de> Message-ID: On Fri, Jan 7, 2011 at 4:27 AM, Vitja Makarov wrote: > 2010/12/31 Robert Bradshaw : >> On Thu, Dec 30, 2010 at 10:05 AM, Stefan Behnel wrote: >>> Vitja Makarov, 30.12.2010 18:53: >>>> Why did you reverted decorators stuff? >>>> >>>> https://github.com/vitek/cython/commit/e0d366d9409680849e6f429992ac9724e2ad1016 >>> >>> Because I didn't get it finished, it broke the Sage build and cython-devel >>> should stay cleanly working before the release. >> >> +1 >> >>> I added a link to the build >>> failure log to the trac ticket but didn't even manage to come up with a >>> suitable test case before I had to stop working on it. >>> >>> What I implemented so far works in most cases but there are issues with >>> functions at the module scope. I didn't check the history but I think I >>> recall from the code comments that Robert worked in this area (method >>> binding) a while ago, and the comments hint at an unfinished refactoring >>> that would help with this change. >> >> I'm not working on anything in the area at the moment though (or, >> contrary to my expectations, much of anything Cython-wise... having >> too much fun hanging out with the family :). >> >> - Robert >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > I've added GenerartorDefNode and GeneratorBodyDefNode now all the > generators stuff is handled there. Yay! I'm pretty busy this weekend, but have been following your branch from a distance and so reviewing this shouldn't take too long. I'm thinking we should get a 0.14.1 bug fix out ASAP (e.g. once those Windows and type inference bugs are in) and then 0.15 will follow with generators. - Robert From stefan_ml at behnel.de Fri Jan 7 19:40:40 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 07 Jan 2011 19:40:40 +0100 Subject: [Cython] General generator expressions In-Reply-To: References: <4CFBD711.5060206@behnel.de> <4D00A586.8060801@behnel.de> <4D01D7EF.6060906@behnel.de> <4D02BB12.7060306@behnel.de> <4D05B116.2020900@behnel.de> <4D06930B.2090205@behnel.de> <4D077CA5.4010103@behnel.de> <4D078717.4050105@behnel.de> <4D1CC9F4.60408@behnel.de> Message-ID: <4D275E28.9030106@behnel.de> Robert Bradshaw, 07.01.2011 19:17: > On Fri, Jan 7, 2011 at 4:27 AM, Vitja Makarov wrote: >> I've added GenerartorDefNode and GeneratorBodyDefNode now all the >> generators stuff is handled there. > > Yay! I'm pretty busy this weekend, but have been following your branch > from a distance and so reviewing this shouldn't take too long. I'm > thinking we should get a 0.14.1 bug fix out ASAP (e.g. once those > Windows and type inference bugs are in) and then 0.15 will follow with > generators. Absolutely. I should find some time for a review early next week, but more eyes are always better. Stefan From robertwb at math.washington.edu Sat Jan 8 08:56:49 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 7 Jan 2011 23:56:49 -0800 Subject: [Cython] Cython 0.14 Bugreport In-Reply-To: References: <068A69FD-5D11-44D6-9306-3CFB6325CD45@gmail.com> <4D260A57.8010600@behnel.de> Message-ID: On Thu, Jan 6, 2011 at 10:54 AM, Robert Bradshaw wrote: > On Thu, Jan 6, 2011 at 10:48 AM, Vitja Makarov wrote: >> 2011/1/6 Robert Bradshaw : >>> On Thu, Jan 6, 2011 at 10:30 AM, Stefan Behnel wrote: >>>> Daniel Norberg, 06.01.2011 12:20: >>>>> I'm getting a curious error in Cython 0.14 when trying to compile this: >>>>> >>>>> ? ?def bar(foo): >>>>> ? ? ? ?qux = foo >>>>> ? ? ? ?quux = foo[qux.baz] >>>>> >>>>> The error message: >>>>> >>>>> ? ? ? $ cython bar.py >>>>> >>>>> ? ? ? Error compiling Cython file: >>>>> ? ? ? ------------------------------------------------------------ >>>>> ? ? ? ... >>>>> ? ? ? def bar(foo): >>>>> ? ? ? ? ? ? ? qux = foo >>>>> ? ? ? ? ? ? ? quux = foo[qux.baz] >>>>> ? ? ? ? ? ? ? ? ? ? ^ >>>>> ? ? ? ------------------------------------------------------------ >>>>> >>>>> ? ? ? /Users/daniel/Desktop/cython-test/bar.py:3:15: Object of type '' has no attribute 'baz' >>>>> >>>>> Cython 0.13 compiles this just fine. I also tried the latest revision of cython-devel (b816b03ff502) and it fails. >>>> >>>> I can reproduce this. From a quick test, it seems like the type inference >>>> machinery processes 'quux' and 'qux' in the wrong order, i.e. 'quux' before >>>> 'qux'. Anyone interested in taking a closer look? >>> >>> That shouldn't be happening, as it should know the inferred type of >>> quux depends on the inferred type of qux. I'll take a look. >>> >>> - Robert >>> _______________________________________________ >>> Cython-dev mailing list >>> Cython-dev at codespeak.net >>> http://codespeak.net/mailman/listinfo/cython-dev >>> >> >> Strange thing this code compiles: >> >> def bar(foo): >> ? ?qux = foo >> ? ?xquux = foo[qux.baz] >> >> but this doesn't: >> >> def bar(foo): >> ? ?qux = foo >> ? ?aquux = foo[qux.baz] > > Since it doesn't detect the dependency, half the time it'll get lucky > (probably as a function of the variable names). The problem was that the indexing operator inference was changed to depend on the index type as well as the base type, but its type_dependencies method wasn't updated to reflect this. Thanks for catching that, fix at https://github.com/cython/cython/commit/43a3dbea06e00bf6eb6592017964b79e35bae2d6 - Robert From vitja.makarov at gmail.com Mon Jan 10 09:05:59 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Mon, 10 Jan 2011 11:05:59 +0300 Subject: [Cython] General generator expressions In-Reply-To: <4D275E28.9030106@behnel.de> References: <4CFBD711.5060206@behnel.de> <4D00A586.8060801@behnel.de> <4D01D7EF.6060906@behnel.de> <4D02BB12.7060306@behnel.de> <4D05B116.2020900@behnel.de> <4D06930B.2090205@behnel.de> <4D077CA5.4010103@behnel.de> <4D078717.4050105@behnel.de> <4D1CC9F4.60408@behnel.de> <4D275E28.9030106@behnel.de> Message-ID: 2011/1/7 Stefan Behnel : > Robert Bradshaw, 07.01.2011 19:17: >> On Fri, Jan 7, 2011 at 4:27 AM, Vitja Makarov wrote: >>> I've added GenerartorDefNode and GeneratorBodyDefNode now all the >>> generators stuff is handled there. >> >> Yay! I'm pretty busy this weekend, but have been following your branch >> from a distance and so reviewing this shouldn't take too long. I'm >> thinking we should get a 0.14.1 bug fix out ASAP (e.g. once those >> Windows and type inference bugs are in) and then 0.15 will follow with >> generators. > > Absolutely. I should find some time for a review early next week, but more > eyes are always better. > I've added some new things: - return with no value inside generator - nested yields (yield (yield (yield))) - yield inside lambda (strange thing btw), was used in test_collections - genexpr support, inlining is now disabled Now I'm thinking how to enable genexpr inlining. -- vitja. From markflorisson88 at gmail.com Mon Jan 10 12:28:29 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Mon, 10 Jan 2011 12:28:29 +0100 Subject: [Cython] Closure scope object initialization Message-ID: Hello, There is a slight problem with the scope object in closures because it is not initialized to NULL in closure frames, it is instead initialized later by assigning a function argument to it. For the debugger this is a problem when execution of the debuggee is halted before this assignment happens, because there is no way to tell whether closed-over variables are valid or not. For e.g. the listing of local variables this is not a big issue, as this is entirely safe. But for code execution this means the debugger will have the debuggee try to put an invalid pointer in a dict in which the code will be executed, with a near 100% chance of segfaulting the debuggee. So the question is, could we simply initialize the scope objects to NULL? Cheers, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110110/564e8512/attachment.htm From robertwb at math.washington.edu Mon Jan 10 19:31:13 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 10 Jan 2011 10:31:13 -0800 Subject: [Cython] Closure scope object initialization In-Reply-To: References: Message-ID: On Mon, Jan 10, 2011 at 3:28 AM, mark florisson wrote: > Hello, > There is a slight problem with the scope object in closures because it is > not initialized to NULL in closure frames, it is instead initialized later > by assigning a function argument to it. For the debugger this is a problem > when execution of the debuggee is halted before this assignment happens, > because there is no way to tell whether closed-over variables are valid or > not. For e.g. the listing of local variables this is not a big issue, as > this is entirely safe. But for code execution this means the debugger will > have the debuggee try to put an invalid pointer in a dict in which the code > will be executed, with a near 100% chance of segfaulting the debuggee. > So the question is, could we simply initialize the scope objects to NULL? I think that'd be a fine thing to do. Are there any code paths that don't initialize this before its actually used (in which case the initial assignment may get optimized away)? - Robert From markflorisson88 at gmail.com Mon Jan 10 20:05:27 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Mon, 10 Jan 2011 20:05:27 +0100 Subject: [Cython] Closure scope object initialization In-Reply-To: References: Message-ID: On 10 January 2011 19:31, Robert Bradshaw wrote: > On Mon, Jan 10, 2011 at 3:28 AM, mark florisson > wrote: > > Hello, > > There is a slight problem with the scope object in closures because it is > > not initialized to NULL in closure frames, it is instead initialized > later > > by assigning a function argument to it. For the debugger this is a > problem > > when execution of the debuggee is halted before this assignment happens, > > because there is no way to tell whether closed-over variables are valid > or > > not. For e.g. the listing of local variables this is not a big issue, as > > this is entirely safe. But for code execution this means the debugger > will > > have the debuggee try to put an invalid pointer in a dict in which the > code > > will be executed, with a near 100% chance of segfaulting the debuggee. > > So the question is, could we simply initialize the scope objects to NULL? > > I think that'd be a fine thing to do. Are there any code paths that > don't initialize this before its actually used (in which case the > initial assignment may get optimized away)? > No the scope object is always initialized (it has to be as it's only present when inner scopes access free variables from outer scopes, or when the outer scope needs to store user-variables in scope objects), but the problem is that execution may be halted by the debugger before it is initialized. When automatic variables are set to NULL and the debugger breaks on the function, accesses to such variables will return NULL even before the actual assignment and declaration happen, instead of "something random". In any case, I think automatic variables that are initialized to NULL but not used may get optimized anyway. So then we come to the next point, who wants to implement this? :) I had a brief initial attempt by assigning 'NULL' to the 'init' attribute of the scope AST entry, which was unfruitful. I could take a deeper look into this issue, if I should, please tell me (and please don't hesitate to give some pointers :). - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110110/cd14d928/attachment.htm From robertwb at math.washington.edu Mon Jan 10 20:40:30 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 10 Jan 2011 11:40:30 -0800 Subject: [Cython] Closure scope object initialization In-Reply-To: References: Message-ID: On Mon, Jan 10, 2011 at 11:05 AM, mark florisson wrote: > > > On 10 January 2011 19:31, Robert Bradshaw > wrote: >> >> On Mon, Jan 10, 2011 at 3:28 AM, mark florisson >> wrote: >> > Hello, >> > There is a slight problem with the scope object in closures because it >> > is >> > not initialized to NULL in closure frames, it is instead initialized >> > later >> > by assigning a function argument to it. For the debugger this is a >> > problem >> > when execution of the debuggee is halted before this assignment happens, >> > because there is no way to tell whether closed-over variables are valid >> > or >> > not. For e.g. the listing of local variables this is not a big issue, as >> > this is entirely safe. But for code execution this means the debugger >> > will >> > have the debuggee try to put an invalid pointer in a dict in which the >> > code >> > will be executed, with a near 100% chance of segfaulting the debuggee. >> > So the question is, could we simply initialize the scope objects to >> > NULL? >> >> I think that'd be a fine thing to do. Are there any code paths that >> don't initialize this before its actually used (in which case the >> initial assignment may get optimized away)? > > > No the scope object is always initialized (it has to be as it's only present > when inner scopes access free variables from outer scopes, or when the outer > scope needs to store user-variables in scope objects), but the problem is > that execution may be halted by the debugger before it is initialized. When > automatic variables are set to NULL and the debugger breaks on the function, > accesses to such variables will return NULL even before the actual > assignment and declaration happen, instead of "something random". In any > case, I think automatic variables that are initialized to NULL but not used > may get optimized anyway. > So then we come to the next point, who wants to implement this? :) I had a > brief initial attempt by assigning 'NULL' to the 'init' attribute of the > scope AST entry, which was unfruitful. I could take a deeper look into this > issue, if I should, please tell me (and please don't hesitate to give some > pointers :). Since you're dealing with the state of objects before argument parsing, it may be more fruitful to initialize it directly as part of the header generation code than via the AST (whose first statement assumes the argument parsing has been atomically done previously). - Robert From markflorisson88 at gmail.com Mon Jan 10 23:24:37 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Mon, 10 Jan 2011 23:24:37 +0100 Subject: [Cython] Closure scope object initialization In-Reply-To: References: Message-ID: On 10 January 2011 20:40, Robert Bradshaw wrote: > On Mon, Jan 10, 2011 at 11:05 AM, mark florisson > wrote: > > > > > > On 10 January 2011 19:31, Robert Bradshaw > > wrote: > >> > >> On Mon, Jan 10, 2011 at 3:28 AM, mark florisson > >> wrote: > >> > Hello, > >> > There is a slight problem with the scope object in closures because it > >> > is > >> > not initialized to NULL in closure frames, it is instead initialized > >> > later > >> > by assigning a function argument to it. For the debugger this is a > >> > problem > >> > when execution of the debuggee is halted before this assignment > happens, > >> > because there is no way to tell whether closed-over variables are > valid > >> > or > >> > not. For e.g. the listing of local variables this is not a big issue, > as > >> > this is entirely safe. But for code execution this means the debugger > >> > will > >> > have the debuggee try to put an invalid pointer in a dict in which the > >> > code > >> > will be executed, with a near 100% chance of segfaulting the debuggee. > >> > So the question is, could we simply initialize the scope objects to > >> > NULL? > >> > >> I think that'd be a fine thing to do. Are there any code paths that > >> don't initialize this before its actually used (in which case the > >> initial assignment may get optimized away)? > > > > > > No the scope object is always initialized (it has to be as it's only > present > > when inner scopes access free variables from outer scopes, or when the > outer > > scope needs to store user-variables in scope objects), but the problem is > > that execution may be halted by the debugger before it is initialized. > When > > automatic variables are set to NULL and the debugger breaks on the > function, > > accesses to such variables will return NULL even before the actual > > assignment and declaration happen, instead of "something random". In any > > case, I think automatic variables that are initialized to NULL but not > used > > may get optimized anyway. > > So then we come to the next point, who wants to implement this? :) I had > a > > brief initial attempt by assigning 'NULL' to the 'init' attribute of the > > scope AST entry, which was unfruitful. I could take a deeper look into > this > > issue, if I should, please tell me (and please don't hesitate to give > some > > pointers :). > > Since you're dealing with the state of objects before argument > parsing, it may be more fruitful to initialize it directly as part of > the header generation code than via the AST (whose first statement > assumes the argument parsing has been atomically done previously). Thanks that did it, but of course I was mistaken as the automatic variables still point to uninitialized stack memory until they are initialized. I will have to find another way to detect whether the scope object was initialized, I think I will take the same approach as with the other variables, in this case comparing the current line number of C code with the C line number associated with the first Cython statement in the function. Unfortunately this requires some changes in the XML as this information is currently not available. > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110110/7033571b/attachment.htm From vitja.makarov at gmail.com Wed Jan 12 19:29:07 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Wed, 12 Jan 2011 21:29:07 +0300 Subject: [Cython] raw string problem Message-ID: Hi! It seems that cython parses raw strings as usual strings: Try this: print r'\"' -- vitja. From robertwb at math.washington.edu Wed Jan 12 19:50:35 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 12 Jan 2011 10:50:35 -0800 Subject: [Cython] raw string problem In-Reply-To: References: Message-ID: On Wed, Jan 12, 2011 at 10:29 AM, Vitja Makarov wrote: > Hi! > > It seems that cython parses raw strings as usual strings: > > Try this: > print r'\"' This is certainly a bug. http://trac.cython.org/cython_trac/ticket/641 - Robert From vitja.makarov at gmail.com Wed Jan 12 20:03:16 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Wed, 12 Jan 2011 22:03:16 +0300 Subject: [Cython] raw string problem In-Reply-To: References: Message-ID: 2011/1/12 Robert Bradshaw : > On Wed, Jan 12, 2011 at 10:29 AM, Vitja Makarov wrote: >> Hi! >> >> It seems that cython parses raw strings as usual strings: >> >> Try this: >> print r'\"' > > This is certainly a bug. http://trac.cython.org/cython_trac/ticket/641 > This should be easy to fix... Why do you think that this could be a major change? Does some code use this bug? -- vitja. From robertwb at math.washington.edu Wed Jan 12 20:10:09 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 12 Jan 2011 11:10:09 -0800 Subject: [Cython] raw string problem In-Reply-To: References: Message-ID: On Wed, Jan 12, 2011 at 11:03 AM, Vitja Makarov wrote: > 2011/1/12 Robert Bradshaw : >> On Wed, Jan 12, 2011 at 10:29 AM, Vitja Makarov wrote: >>> Hi! >>> >>> It seems that cython parses raw strings as usual strings: >>> >>> Try this: >>> print r'\"' >> >> This is certainly a bug. http://trac.cython.org/cython_trac/ticket/641 >> > > This should be easy to fix... > Why do you think that this could be a major change? > Does some code use this bug? It is very possible that there is code out there using this bug, and if so it would be a silent, backwards incompatible change. - Robert From stefan_ml at behnel.de Wed Jan 12 21:51:11 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 12 Jan 2011 21:51:11 +0100 Subject: [Cython] raw string problem In-Reply-To: References: Message-ID: <4D2E143F.60007@behnel.de> Vitja Makarov, 12.01.2011 19:29: > It seems that cython parses raw strings as usual strings: > > Try this: > print r'\"' That's "slightly" exaggerated. The only broken cases are '\"' and '\'', everything else works AFAICT. Stefan From vitja.makarov at gmail.com Wed Jan 12 22:08:13 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 13 Jan 2011 00:08:13 +0300 Subject: [Cython] raw string problem In-Reply-To: <4D2E143F.60007@behnel.de> References: <4D2E143F.60007@behnel.de> Message-ID: 2011/1/12 Stefan Behnel : > Vitja Makarov, 12.01.2011 19:29: >> It seems that cython parses raw strings as usual strings: >> >> Try this: >> print r'\"' > > That's "slightly" exaggerated. The only broken cases are '\"' and '\'', > everything else works AFAICT. > > Stefan That breaks StringEncoding._to_escape_sequence btw... -- vitja. From stefan_ml at behnel.de Wed Jan 12 22:13:10 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 12 Jan 2011 22:13:10 +0100 Subject: [Cython] raw string problem In-Reply-To: References: <4D2E143F.60007@behnel.de> Message-ID: <4D2E1966.1020607@behnel.de> Vitja Makarov, 12.01.2011 22:08: > 2011/1/12 Stefan Behnel: >> Vitja Makarov, 12.01.2011 19:29: >>> It seems that cython parses raw strings as usual strings: >>> >>> Try this: >>> print r'\"' >> >> That's "slightly" exaggerated. The only broken cases are '\"' and '\'', >> everything else works AFAICT. > > That breaks StringEncoding._to_escape_sequence btw... What do you mean? When I fix the parser, it works for me. Stefan From robertwb at math.washington.edu Wed Jan 12 22:16:23 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 12 Jan 2011 13:16:23 -0800 Subject: [Cython] raw string problem In-Reply-To: <4D2E143F.60007@behnel.de> References: <4D2E143F.60007@behnel.de> Message-ID: On Wed, Jan 12, 2011 at 12:51 PM, Stefan Behnel wrote: > Vitja Makarov, 12.01.2011 19:29: >> It seems that cython parses raw strings as usual strings: >> >> Try this: >> print r'\"' > > That's "slightly" exaggerated. The only broken cases are '\"' and '\'', > everything else works AFAICT. That is comforting, and given that it's a tiny corner case I think it's worth and safe fixing in 0.14.1. - Robert From stefan_ml at behnel.de Wed Jan 12 22:24:37 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 12 Jan 2011 22:24:37 +0100 Subject: [Cython] raw string problem In-Reply-To: References: <4D2E143F.60007@behnel.de> Message-ID: <4D2E1C15.8000801@behnel.de> Robert Bradshaw, 12.01.2011 22:16: > On Wed, Jan 12, 2011 at 12:51 PM, Stefan Behnel wrote: >> Vitja Makarov, 12.01.2011 19:29: >>> It seems that cython parses raw strings as usual strings: >>> >>> Try this: >>> print r'\"' >> >> That's "slightly" exaggerated. The only broken cases are '\"' and '\'', >> everything else works AFAICT. > > That is comforting, and given that it's a tiny corner case I think > it's worth and safe fixing in 0.14.1. I pushed a fix. Stefan From vitja.makarov at gmail.com Wed Jan 12 22:25:19 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 13 Jan 2011 00:25:19 +0300 Subject: [Cython] raw string problem In-Reply-To: <4D2E1C15.8000801@behnel.de> References: <4D2E143F.60007@behnel.de> <4D2E1C15.8000801@behnel.de> Message-ID: 2011/1/13 Stefan Behnel : > Robert Bradshaw, 12.01.2011 22:16: >> On Wed, Jan 12, 2011 at 12:51 PM, Stefan Behnel wrote: >>> Vitja Makarov, 12.01.2011 19:29: >>>> It seems that cython parses raw strings as usual strings: >>>> >>>> Try this: >>>> print r'\"' >>> >>> That's "slightly" exaggerated. The only broken cases are '\"' and '\'', >>> everything else works AFAICT. >> >> That is comforting, and given that it's a tiny corner case I think >> it's worth and safe fixing in 0.14.1. > > I pushed a fix. > Thanks! -- vitja. From stefan_ml at behnel.de Thu Jan 13 11:39:55 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 11:39:55 +0100 Subject: [Cython] Cython 0.14.1? Message-ID: <4D2ED67B.2060101@behnel.de> Hi, I looked through the tickets that are scheduled for 0.14.1 but couldn't see any that are truly critical, especially no regressions. http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 I think we fixed all known bugs that appeared with 0.14 by now, so I'd say it's time for at least a release candidate. Stefan From markflorisson88 at gmail.com Thu Jan 13 11:42:15 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Thu, 13 Jan 2011 11:42:15 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D2ED67B.2060101@behnel.de> References: <4D2ED67B.2060101@behnel.de> Message-ID: On 13 January 2011 11:39, Stefan Behnel wrote: > Hi, > > I looked through the tickets that are scheduled for 0.14.1 but couldn't see > any that are truly critical, especially no regressions. > > > http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 > > I think we fixed all known bugs that appeared with 0.14 by now, so I'd say > it's time for at least a release candidate. > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > Great, I presume the pending pull requests will be merged for this release? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110113/f2234865/attachment.htm From stefan_ml at behnel.de Thu Jan 13 11:50:52 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 11:50:52 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> Message-ID: <4D2ED90C.5040103@behnel.de> mark florisson, 13.01.2011 11:42: > On 13 January 2011 11:39, Stefan Behnel wrote: >> I looked through the tickets that are scheduled for 0.14.1 but couldn't see >> any that are truly critical, especially no regressions. >> >> http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 >> >> I think we fixed all known bugs that appeared with 0.14 by now, so I'd say >> it's time for at least a release candidate. > > Great, I presume the pending pull requests will be merged for this release? The test runner change by Vitja should be ok. Can't comment on the debugging changes. Stefan From dagss at student.matnat.uio.no Thu Jan 13 11:56:01 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 13 Jan 2011 11:56:01 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> Message-ID: <4D2EDA41.4000109@student.matnat.uio.no> On 01/13/2011 11:42 AM, mark florisson wrote: > > > On 13 January 2011 11:39, Stefan Behnel > wrote: > > Hi, > > I looked through the tickets that are scheduled for 0.14.1 but > couldn't see > any that are truly critical, especially no regressions. > > http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 > > > I think we fixed all known bugs that appeared with 0.14 by now, so > I'd say > it's time for at least a release candidate. > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > > > Great, I presume the pending pull requests will be merged for this > release? As we use Trac to coordinate fixes with releases I think it'd be good if we went into the habit of always creating a small Trac ticket for each pull request. (If we move to the github tracker, I suppose this will happen automatically, but for now we should do it manually.) Your pull request is now http://trac.cython.org/cython_trac/ticket/642 Dag Sverre -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110113/ee3b1346/attachment.htm From stefan_ml at behnel.de Thu Jan 13 12:02:58 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 12:02:58 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D2EDA41.4000109@student.matnat.uio.no> References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> Message-ID: <4D2EDBE2.8030500@behnel.de> Dag Sverre Seljebotn, 13.01.2011 11:56: > As we use Trac to coordinate fixes with releases I think it'd be good if we > went into the habit of always creating a small Trac ticket for each pull > request. +1. My immediate thoughts were "oh great, one place more to look before a release...". Stefan From markflorisson88 at gmail.com Thu Jan 13 12:05:48 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Thu, 13 Jan 2011 12:05:48 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D2EDA41.4000109@student.matnat.uio.no> References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> Message-ID: On 13 January 2011 11:56, Dag Sverre Seljebotn wrote: > On 01/13/2011 11:42 AM, mark florisson wrote: > > > > On 13 January 2011 11:39, Stefan Behnel wrote: > >> Hi, >> >> I looked through the tickets that are scheduled for 0.14.1 but couldn't >> see >> any that are truly critical, especially no regressions. >> >> >> http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 >> >> I think we fixed all known bugs that appeared with 0.14 by now, so I'd say >> it's time for at least a release candidate. >> >> Stefan >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > Great, I presume the pending pull requests will be merged for this release? > > > As we use Trac to coordinate fixes with releases I think it'd be good if we > went into the habit of always creating a small Trac ticket for each pull > request. (If we move to the github tracker, I suppose this will happen > automatically, but for now we should do it manually.) > > Your pull request is now http://trac.cython.org/cython_trac/ticket/642 > > Dag Sverre > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > > Ok, sounds fine. In any case, there are virtually no changes when the --gdb flag is inactive. These last additions would be very welcome however, as it supports fast Python stepping and stepping over (by using a watchpoint on the bytecode instruction pointer instead of a "step-over loop"), support for closures, the debugger now also detects when there is a pending exception in Python code, there are more tests, and additionally I fixed the debug command line flags such as --debug-trace-code-generation, as the module which set the code tracer functionality globally through a metaclass was imported before the command line option was processed. So basically I think the code could be reviewed by running the testsuite and by looking at which files outside the Cython.Debugger package were changed in what way. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110113/191667bf/attachment.htm From sable at users.sourceforge.net Thu Jan 13 13:45:16 2011 From: sable at users.sourceforge.net (=?ISO-8859-1?Q?S=E9bastien_Sabl=E9?=) Date: Thu, 13 Jan 2011 13:45:16 +0100 Subject: [Cython] Regression in cython trunk Message-ID: <4D2EF3DC.4050005@users.sourceforge.net> Hi, Thanks for Cython it is a great tool that I use everyday in my work. I wanted to test cython 0.14.1 before its release and I found a regression related to type inference compared to 0.13 that impacts my project. The following code will not correctly compile: #cython: infer_types=True cdef get_field_flags(): flags = [] append = flags.append append('fetched') return flags cythoning magnum/magpy.pyx to magnum/magpy.c Error compiling Cython file: ------------------------------------------------------------ ... #cython: infer_types=True cdef get_field_flags(): flags = [] append = flags.append append('fetched') ^ ------------------------------------------------------------ magnum/magpy.pyx:6:10: Call with wrong number of arguments (expected 2, got 1) The same code would work great with cython 0.13 and it would work with 0.14.1 if I remove the infer_types=True line. regards -- S?bastien Sabl? From stefan_ml at behnel.de Thu Jan 13 15:53:24 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 15:53:24 +0100 Subject: [Cython] Regression in cython trunk In-Reply-To: <4D2EF3DC.4050005@users.sourceforge.net> References: <4D2EF3DC.4050005@users.sourceforge.net> Message-ID: <4D2F11E4.6040001@behnel.de> S?bastien Sabl?, 13.01.2011 13:45: > Thanks for Cython it is a great tool that I use everyday in my work. > > I wanted to test cython 0.14.1 before its release and I found a > regression related to type inference compared to 0.13 that impacts my > project. > > The following code will not correctly compile: > > #cython: infer_types=True > > cdef get_field_flags(): > flags = [] > append = flags.append > append('fetched') > return flags ... with the obvious (and much faster) work-around to not use the bound method instead of calling it directly on the list object. > cythoning magnum/magpy.pyx to magnum/magpy.c > > Error compiling Cython file: > ------------------------------------------------------------ > ... > #cython: infer_types=True > > cdef get_field_flags(): > flags = [] > append = flags.append > append('fetched') > ^ > ------------------------------------------------------------ > > magnum/magpy.pyx:6:10: Call with wrong number of arguments (expected 2, > got 1) > > The same code would work great with cython 0.13 and it would work with > 0.14.1 if I remove the infer_types=True line. Thanks for the report. It's quite possible that the type inference mechanism now assumes that the bound method is actually the C-API function replacement, i.e. the C function PyList_Append(). That's the wrong thing to infer here as it cannot be made to work. Stefan From stefan_ml at behnel.de Thu Jan 13 16:32:48 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 16:32:48 +0100 Subject: [Cython] Regression in cython trunk In-Reply-To: <4D2F11E4.6040001@behnel.de> References: <4D2EF3DC.4050005@users.sourceforge.net> <4D2F11E4.6040001@behnel.de> Message-ID: <4D2F1B20.8040606@behnel.de> Stefan Behnel, 13.01.2011 15:53: > S?bastien Sabl?, 13.01.2011 13:45: >> #cython: infer_types=True >> >> cdef get_field_flags(): >> flags = [] >> append = flags.append >> append('fetched') >> ^ >> ------------------------------------------------------------ >> >> magnum/magpy.pyx:6:10: Call with wrong number of arguments (expected 2, >> got 1) >> >> The same code would work great with cython 0.13 and it would work with >> 0.14.1 if I remove the infer_types=True line. > > Thanks for the report. It's quite possible that the type inference > mechanism now assumes that the bound method is actually the C-API function > replacement, i.e. the C function PyList_Append(). That's the wrong thing to > infer here as it cannot be made to work. Here's another related problem: # cython: infer_types=True cdef int func(int x): return x+2 def inferred(): f = func 'f' is inferred as C function, not as function pointer. http://trac.cython.org/cython_trac/ticket/643 http://trac.cython.org/cython_trac/ticket/644 Stefan From sable at users.sourceforge.net Thu Jan 13 17:22:56 2011 From: sable at users.sourceforge.net (=?UTF-8?B?U8OpYmFzdGllbiBTYWJsw6k=?=) Date: Thu, 13 Jan 2011 17:22:56 +0100 Subject: [Cython] Regression in cython trunk In-Reply-To: <4D2F11E4.6040001@behnel.de> References: <4D2EF3DC.4050005@users.sourceforge.net> <4D2F11E4.6040001@behnel.de> Message-ID: <4D2F26E0.5000607@users.sourceforge.net> Le 13/01/2011 15:53, Stefan Behnel a ?crit : > > ... with the obvious (and much faster) work-around to not use the bound > method instead of calling it directly on the list object. Yes, I simplified my code in order to make a readable test case. The actual code is more complex, but the workaround would still probably make it faster with type inference. > http://trac.cython.org/cython_trac/ticket/643 > http://trac.cython.org/cython_trac/ticket/644 Thanks for opening some tickets. Also if you could create an account for me in trac, I may directly open one next time, and comment if needed. sable:fMzeuao1PSadU regards -- S?bastien Sabl? From stefan_ml at behnel.de Thu Jan 13 17:24:23 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 17:24:23 +0100 Subject: [Cython] Regression in cython trunk In-Reply-To: <4D2F1B20.8040606@behnel.de> References: <4D2EF3DC.4050005@users.sourceforge.net> <4D2F11E4.6040001@behnel.de> <4D2F1B20.8040606@behnel.de> Message-ID: <4D2F2737.5050800@behnel.de> Stefan Behnel, 13.01.2011 16:32: > Stefan Behnel, 13.01.2011 15:53: >> S?bastien Sabl?, 13.01.2011 13:45: >>> #cython: infer_types=True >>> >>> cdef get_field_flags(): >>> flags = [] >>> append = flags.append >>> append('fetched') >>> ^ >>> ------------------------------------------------------------ >>> >>> magnum/magpy.pyx:6:10: Call with wrong number of arguments (expected 2, >>> got 1) >>> >>> The same code would work great with cython 0.13 and it would work with >>> 0.14.1 if I remove the infer_types=True line. >> >> Thanks for the report. It's quite possible that the type inference >> mechanism now assumes that the bound method is actually the C-API function >> replacement, i.e. the C function PyList_Append(). That's the wrong thing to >> infer here as it cannot be made to work. > > Here's another related problem: > > # cython: infer_types=True > > cdef int func(int x): > return x+2 > > def inferred(): > f = func > > 'f' is inferred as C function, not as function pointer. > > http://trac.cython.org/cython_trac/ticket/643 > http://trac.cython.org/cython_trac/ticket/644 Fixes are here: https://github.com/cython/cython/commit/3ac5fb86833de2831e72bd368fab6b6f1fb03ec2 https://github.com/cython/cython/commit/48177ec6f603d7ae88ee6302bfe5c4c299cd350a Stefan From dalcinl at gmail.com Thu Jan 13 18:33:47 2011 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 13 Jan 2011 14:33:47 -0300 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D2ED67B.2060101@behnel.de> References: <4D2ED67B.2060101@behnel.de> Message-ID: On 13 January 2011 07:39, Stefan Behnel wrote: > Hi, > > I looked through the tickets that are scheduled for 0.14.1 but couldn't see > any that are truly critical, especially no regressions. > It would be nice to make a testsuite run in Windows (32 and 64, with MSVC and MinGW) before the release. However, I'm not able to do that until Monday... > http://trac.cython.org/cython_trac/query?status=assigned&status=new&status=reopened&order=priority&col=id&col=summary&col=status&col=type&col=priority&col=milestone&col=component&milestone=0.14.1 > > I think we fixed all known bugs that appeared with 0.14 by now, so I'd say > it's time for at least a release candidate. > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- Lisandro Dalcin --------------- CIMEC (INTEC/CONICET-UNL) Predio CONICET-Santa Fe Colectora RN 168 Km 472, Paraje El Pozo Tel: +54-342-4511594 (ext 1011) Tel/Fax: +54-342-4511169 From dwdreisigmeyer at gmail.com Thu Jan 13 20:38:37 2011 From: dwdreisigmeyer at gmail.com (David Dreisigmeyer) Date: Thu, 13 Jan 2011 14:38:37 -0500 Subject: [Cython] Gambit-C problem Message-ID: Hi everyone, I've been having problem trying to use Gambit-C scheme in Python. Everything works fine if I compile Gambit-C alone and run it. But if I try to combine it with Cython to make a extension it doesn't work. I've been working on the problem with the Gambit-C folks and was hoping there might be some suggestions on the Python side. I've attached my code -- just run make. To run the standalone created by Cython: $ ./server_setup ----------------------------------------------------------------------------------------------------------------- Here's a typical Python session: Process Python finished Python 2.7.1 (r271:86832, Dec 29 2010, 01:38:09) Type "copyright", "credits" or "license" for more information. IPython 0.10.1 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object'. ?object also works, ?? prints more. In [2]: import gambit In [3]: gambit.cleanup () Gambit-C cleanup complete In [4]: gambit.setup () Gambit-C setup successful In [5]: gambit.cleanup () Process Python exited abnormally with code 1 ----------------------------------------------------------------------------------------------------------------- This is on OS X 10.6.6 with gcc-4.2, Cython 0.14, Python 2.7.1 and Gambit-C 4.6.0 from git. To install Gambit-C see: http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Contributing_Patches_to_Gambit_Source_Code or maybe http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/ , though I'm using the above. Macports might work also. -------------- next part -------------- A non-text attachment was scrubbed... Name: gambit_c.tgz Type: application/x-gzip Size: 2580 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20110113/9b8fab6d/attachment-0001.bin From dwdreisigmeyer at gmail.com Thu Jan 13 20:45:40 2011 From: dwdreisigmeyer at gmail.com (David Dreisigmeyer) Date: Thu, 13 Jan 2011 14:45:40 -0500 Subject: [Cython] Gambit-C problem In-Reply-To: References: Message-ID: I should have mentioned that neither printf nor sleep work if, e.g., I modify the Gambit-C source code like the following (in gambit/lib/setup.c): ___EXP_FUNC(void,___cleanup) ___PVOID { /* * Only do cleanup once after successful setup. */ printf ("setup_state : %i \n" , setup_state); fflush (stdout); sleep(30); if (setup_state != 1) return; setup_state = 2; ___cleanup_mem (); ___cleanup_os (); } On Thu, Jan 13, 2011 at 2:38 PM, David Dreisigmeyer wrote: > Hi everyone, > > I've been having problem trying to use Gambit-C scheme in Python. > Everything works fine if I compile Gambit-C alone and run it. ?But if > I try to combine it with Cython to make a extension it doesn't work. > I've been working on the problem with the Gambit-C folks and was > hoping there might be some suggestions on the Python side. ?I've > attached my code -- just run make. ?To run the standalone created by > Cython: > > $ ./server_setup > > ----------------------------------------------------------------------------------------------------------------- > > Here's a typical Python session: > > Process Python finished > Python 2.7.1 (r271:86832, Dec 29 2010, 01:38:09) > Type "copyright", "credits" or "license" for more information. > > IPython 0.10.1 -- An enhanced Interactive Python. > ? ? ? ? ? -> Introduction and overview of IPython's features. > %quickref -> Quick reference. > help ? ? ?-> Python's own help system. > object? ? -> Details about 'object'. ?object also works, ?? prints more. > > In [2]: import gambit > > In [3]: gambit.cleanup () > Gambit-C cleanup complete > > In [4]: gambit.setup () > Gambit-C setup successful > > In [5]: gambit.cleanup () > > Process Python exited abnormally with code 1 > > ----------------------------------------------------------------------------------------------------------------- > > This is on OS X 10.6.6 with gcc-4.2, Cython 0.14, Python 2.7.1 and > Gambit-C 4.6.0 from git. ?To install Gambit-C see: > > http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Contributing_Patches_to_Gambit_Source_Code > > or maybe http://www.iro.umontreal.ca/~gambit/download/gambit/v4.6/prebuilt/ > , though I'm using the above. ?Macports might work also. > From stefan_ml at behnel.de Thu Jan 13 20:50:42 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 13 Jan 2011 20:50:42 +0100 Subject: [Cython] Gambit-C problem In-Reply-To: References: Message-ID: <4D2F5792.4040207@behnel.de> David Dreisigmeyer, 13.01.2011 20:38: > I've been having problem trying to use Gambit-C scheme in Python. > Everything works fine if I compile Gambit-C alone and run it. But if > I try to combine it with Cython to make a extension it doesn't work. Hi, please note that the right place for help requests regarding Cython is the cython-users mailing list, not the core developers mailing list. Stefan From dwdreisigmeyer at gmail.com Thu Jan 13 21:12:34 2011 From: dwdreisigmeyer at gmail.com (David Dreisigmeyer) Date: Thu, 13 Jan 2011 15:12:34 -0500 Subject: [Cython] Gambit-C problem In-Reply-To: <4D2F5792.4040207@behnel.de> References: <4D2F5792.4040207@behnel.de> Message-ID: Oh, sorry about that. On Thu, Jan 13, 2011 at 2:50 PM, Stefan Behnel wrote: > David Dreisigmeyer, 13.01.2011 20:38: >> I've been having problem trying to use Gambit-C scheme in Python. >> Everything works fine if I compile Gambit-C alone and run it. ?But if >> I try to combine it with Cython to make a extension it doesn't work. > > Hi, > > please note that the right place for help requests regarding Cython is the > cython-users mailing list, not the core developers mailing list. > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From vic.kelson at gmail.com Fri Jan 14 05:24:44 2011 From: vic.kelson at gmail.com (Vic Kelson) Date: Fri, 14 Jan 2011 04:24:44 +0000 (UTC) Subject: [Cython] Invitation to connect on LinkedIn Message-ID: <324141747.12436275.1294979084885.JavaMail.app@ela4-bed36.prod> LinkedIn ------------ I'd like to add you to my professional network on LinkedIn. - Vic Vic Kelson Presdent, County Council at Monroe County, IN Bloomington, Indiana Area Confirm that you know Vic Kelson https://www.linkedin.com/e/3krvph-giwl40uq-51/isd/2142104884/z3IbrJO1/ -- (c) 2010, LinkedIn Corporation -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/acef496d/attachment.htm From stefan_ml at behnel.de Fri Jan 14 08:53:55 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 14 Jan 2011 08:53:55 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> Message-ID: <4D300113.8020408@behnel.de> mark florisson, 13.01.2011 12:05: > In any case, there are virtually no changes when the --gdb > flag is inactive. These last additions would be very welcome [...] Ok, here's a general statement. I personally consider the debugging support new, experimental and not a critical feature for compiler or language. You are the current maintainer of that part anyway, so I'm fine with merging the changes for 0.14.1, even unseen, as long as it doesn't break anything that's more important. And, no, that's not a green card for back doors. ;-) Stefan From robertwb at math.washington.edu Fri Jan 14 09:02:38 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 00:02:38 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D300113.8020408@behnel.de> References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> Message-ID: On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: > mark florisson, 13.01.2011 12:05: >> In any case, there are virtually no changes when the --gdb >> flag is inactive. These last additions would be very welcome [...] > > Ok, here's a general statement. I personally consider the debugging support > new, experimental and not a critical feature for compiler or language. You > are the current maintainer of that part anyway, so I'm fine with merging > the changes for 0.14.1, even unseen, as long as it doesn't break anything > that's more important. I'm actually looking at that code right now. > And, no, that's not a green card for back doors. ;-) :) - Robert From robertwb at math.washington.edu Fri Jan 14 10:05:56 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 01:05:56 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> Message-ID: On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: > On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >> mark florisson, 13.01.2011 12:05: >>> In any case, there are virtually no changes when the --gdb >>> flag is inactive. These last additions would be very welcome [...] >> >> Ok, here's a general statement. I personally consider the debugging support >> new, experimental and not a critical feature for compiler or language. You >> are the current maintainer of that part anyway, so I'm fine with merging >> the changes for 0.14.1, even unseen, as long as it doesn't break anything >> that's more important. > > I'm actually looking at that code right now. I've merged this. Note that this brings up http://trac.cython.org/cython_trac/ticket/645 . I'd say we're ready for an rc. - Robert From stefan_ml at behnel.de Fri Jan 14 12:06:56 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 14 Jan 2011 12:06:56 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> Message-ID: <4D302E50.1010503@behnel.de> Robert Bradshaw, 14.01.2011 10:05: > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >>> mark florisson, 13.01.2011 12:05: >>>> In any case, there are virtually no changes when the --gdb >>>> flag is inactive. These last additions would be very welcome [...] >>> >>> Ok, here's a general statement. I personally consider the debugging support >>> new, experimental and not a critical feature for compiler or language. You >>> are the current maintainer of that part anyway, so I'm fine with merging >>> the changes for 0.14.1, even unseen, as long as it doesn't break anything >>> that's more important. >> >> I'm actually looking at that code right now. > > I've merged this. The build complains because it cannot find a file called Cython/Debugger/do_repeat.pyx Is that a missing commit? Stefan From markflorisson88 at gmail.com Fri Jan 14 12:08:32 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 14 Jan 2011 12:08:32 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D302E50.1010503@behnel.de> References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: On 14 January 2011 12:06, Stefan Behnel wrote: > Robert Bradshaw, 14.01.2011 10:05: > > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: > >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: > >>> mark florisson, 13.01.2011 12:05: > >>>> In any case, there are virtually no changes when the --gdb > >>>> flag is inactive. These last additions would be very welcome [...] > >>> > >>> Ok, here's a general statement. I personally consider the debugging > support > >>> new, experimental and not a critical feature for compiler or language. > You > >>> are the current maintainer of that part anyway, so I'm fine with > merging > >>> the changes for 0.14.1, even unseen, as long as it doesn't break > anything > >>> that's more important. > >> > >> I'm actually looking at that code right now. > > > > I've merged this. > > The build complains because it cannot find a file called > > Cython/Debugger/do_repeat.pyx > I don't have it, how does it complain? > Is that a missing commit? > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/30a2abd7/attachment.htm From markflorisson88 at gmail.com Fri Jan 14 12:11:24 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 14 Jan 2011 12:11:24 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: On 14 January 2011 12:08, mark florisson wrote: > > > On 14 January 2011 12:06, Stefan Behnel wrote: > >> Robert Bradshaw, 14.01.2011 10:05: >> > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >> >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >> >>> mark florisson, 13.01.2011 12:05: >> >>>> In any case, there are virtually no changes when the --gdb >> >>>> flag is inactive. These last additions would be very welcome [...] >> >>> >> >>> Ok, here's a general statement. I personally consider the debugging >> support >> >>> new, experimental and not a critical feature for compiler or language. >> You >> >>> are the current maintainer of that part anyway, so I'm fine with >> merging >> >>> the changes for 0.14.1, even unseen, as long as it doesn't break >> anything >> >>> that's more important. >> >> >> >> I'm actually looking at that code right now. >> > >> > I've merged this. >> >> The build complains because it cannot find a file called >> >> Cython/Debugger/do_repeat.pyx >> > > I don't have it, how does it complain? > > >> Is that a missing commit? >> >> Stefan >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > Apparently something left that should have been removed, apologies. You can remove it from the setup.py in the list on line 104. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/857f97e2/attachment.htm From robertwb at math.washington.edu Fri Jan 14 12:11:54 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 03:11:54 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: On Fri, Jan 14, 2011 at 3:08 AM, mark florisson wrote: > > > On 14 January 2011 12:06, Stefan Behnel wrote: >> >> Robert Bradshaw, 14.01.2011 10:05: >> > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >> >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >> >>> mark florisson, 13.01.2011 12:05: >> >>>> In any case, there are virtually no changes when the --gdb >> >>>> flag is inactive. These last additions would be very welcome [...] >> >>> >> >>> Ok, here's a general statement. I personally consider the debugging >> >>> support >> >>> new, experimental and not a critical feature for compiler or language. >> >>> You >> >>> are the current maintainer of that part anyway, so I'm fine with >> >>> merging >> >>> the changes for 0.14.1, even unseen, as long as it doesn't break >> >>> anything >> >>> that's more important. >> >> >> >> I'm actually looking at that code right now. >> > >> > I've merged this. >> >> The build complains because it cannot find a file called >> >> Cython/Debugger/do_repeat.pyx > > I don't have it, how does it complain? It was added to line 103 from setup.py, but not to the repo? - Robert From markflorisson88 at gmail.com Fri Jan 14 12:12:16 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 14 Jan 2011 12:12:16 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: On 14 January 2011 12:11, mark florisson wrote: > > > On 14 January 2011 12:08, mark florisson wrote: > >> >> >> On 14 January 2011 12:06, Stefan Behnel wrote: >> >>> Robert Bradshaw, 14.01.2011 10:05: >>> > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >>> >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >>> >>> mark florisson, 13.01.2011 12:05: >>> >>>> In any case, there are virtually no changes when the --gdb >>> >>>> flag is inactive. These last additions would be very welcome [...] >>> >>> >>> >>> Ok, here's a general statement. I personally consider the debugging >>> support >>> >>> new, experimental and not a critical feature for compiler or >>> language. You >>> >>> are the current maintainer of that part anyway, so I'm fine with >>> merging >>> >>> the changes for 0.14.1, even unseen, as long as it doesn't break >>> anything >>> >>> that's more important. >>> >> >>> >> I'm actually looking at that code right now. >>> > >>> > I've merged this. >>> >>> The build complains because it cannot find a file called >>> >>> Cython/Debugger/do_repeat.pyx >>> >> >> I don't have it, how does it complain? >> >> >>> Is that a missing commit? >>> >>> Stefan >>> _______________________________________________ >>> Cython-dev mailing list >>> Cython-dev at codespeak.net >>> http://codespeak.net/mailman/listinfo/cython-dev >>> >> >> Apparently something left that should have been removed, apologies. You > can remove it from the setup.py in the list on line 104. With that change it should build and install correctly. If you want I can also push, but you'd have to pull. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/ad822135/attachment.htm From stefan_ml at behnel.de Fri Jan 14 12:12:25 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 14 Jan 2011 12:12:25 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: <4D302F99.6050700@behnel.de> mark florisson, 14.01.2011 12:08: > On 14 January 2011 12:06, Stefan Behnel wrote: > >> Robert Bradshaw, 14.01.2011 10:05: >>> On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >>>> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >>>>> mark florisson, 13.01.2011 12:05: >>>>>> In any case, there are virtually no changes when the --gdb >>>>>> flag is inactive. These last additions would be very welcome [...] >>>>> >>>>> Ok, here's a general statement. I personally consider the debugging >> support >>>>> new, experimental and not a critical feature for compiler or language. >> You >>>>> are the current maintainer of that part anyway, so I'm fine with >> merging >>>>> the changes for 0.14.1, even unseen, as long as it doesn't break >> anything >>>>> that's more important. >>>> >>>> I'm actually looking at that code right now. >>> >>> I've merged this. >> >> The build complains because it cannot find a file called >> >> Cython/Debugger/do_repeat.pyx >> > > I don't have it, how does it complain? https://sage.math.washington.edu:8091/hudson/job/cython-devel-build-py2-trunk/744/console setup.py says: compiled_modules = ["Cython.Plex.Scanners", "Cython.Plex.Actions", ..., "Cython.Runtime.refnanny", "Cython.Debugger.do_repeat",] So, you're suggesting that this line can be removed? Stefan From robertwb at math.washington.edu Fri Jan 14 12:12:44 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 03:12:44 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: > > > On 14 January 2011 12:08, mark florisson wrote: >> >> >> On 14 January 2011 12:06, Stefan Behnel wrote: >>> >>> Robert Bradshaw, 14.01.2011 10:05: >>> > On Fri, Jan 14, 2011 at 12:02 AM, Robert Bradshaw wrote: >>> >> On Thu, Jan 13, 2011 at 11:53 PM, Stefan Behnel wrote: >>> >>> mark florisson, 13.01.2011 12:05: >>> >>>> In any case, there are virtually no changes when the --gdb >>> >>>> flag is inactive. These last additions would be very welcome [...] >>> >>> >>> >>> Ok, here's a general statement. I personally consider the debugging >>> >>> support >>> >>> new, experimental and not a critical feature for compiler or >>> >>> language. You >>> >>> are the current maintainer of that part anyway, so I'm fine with >>> >>> merging >>> >>> the changes for 0.14.1, even unseen, as long as it doesn't break >>> >>> anything >>> >>> that's more important. >>> >> >>> >> I'm actually looking at that code right now. >>> > >>> > I've merged this. >>> >>> The build complains because it cannot find a file called >>> >>> Cython/Debugger/do_repeat.pyx >> >> I don't have it, how does it complain? >> >>> >>> Is that a missing commit? >>> >>> Stefan >>> _______________________________________________ >>> Cython-dev mailing list >>> Cython-dev at codespeak.net >>> http://codespeak.net/mailman/listinfo/cython-dev >> > Apparently something left that should have been removed, apologies. You can > remove it from the setup.py in the list on line 104. Done. - Robert From stefan_ml at behnel.de Fri Jan 14 12:27:41 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 14 Jan 2011 12:27:41 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> Message-ID: <4D30332D.3000704@behnel.de> Robert Bradshaw, 14.01.2011 12:12: > On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: >> On 14 January 2011 12:08, mark florisson wrote: >>> On 14 January 2011 12:06, Stefan Behnel wrote: >>>> The build complains because it cannot find a file called >>>> >>>> Cython/Debugger/do_repeat.pyx >>> >> Apparently something left that should have been removed, apologies. You can >> remove it from the setup.py in the list on line 104. > > Done. Ok, that looks better. Now that the debugger support is an official feature, should we have a Hudson job that builds Cython and all tests with gdb support enabled? Stefan From robertwb at math.washington.edu Fri Jan 14 12:31:45 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 03:31:45 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: <4D30332D.3000704@behnel.de> References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> <4D30332D.3000704@behnel.de> Message-ID: On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel wrote: > Robert Bradshaw, 14.01.2011 12:12: >> On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: >>> On 14 January 2011 12:08, mark florisson wrote: >>>> On 14 January 2011 12:06, Stefan Behnel wrote: >>>>> The build complains because it cannot find a file called >>>>> >>>>> Cython/Debugger/do_repeat.pyx >>>> >>> Apparently something left that should have been removed, apologies. You can >>> remove it from the setup.py in the list on line 104. >> >> Done. > > Ok, that looks better. > > Now that the debugger support is an official feature, should we have a > Hudson job that builds Cython and all tests with gdb support enabled? Yes, that would make sense. We'd have to build our own newer gdb of course, and a debug version of Python. - Robert From markflorisson88 at gmail.com Fri Jan 14 12:45:12 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 14 Jan 2011 12:45:12 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> <4D30332D.3000704@behnel.de> Message-ID: On 14 January 2011 12:31, Robert Bradshaw wrote: > On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel > wrote: > > Robert Bradshaw, 14.01.2011 12:12: > >> On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: > >>> On 14 January 2011 12:08, mark florisson wrote: > >>>> On 14 January 2011 12:06, Stefan Behnel wrote: > >>>>> The build complains because it cannot find a file called > >>>>> > >>>>> Cython/Debugger/do_repeat.pyx > >>>> > >>> Apparently something left that should have been removed, apologies. You > can > >>> remove it from the setup.py in the list on line 104. > >> > >> Done. > > > > Ok, that looks better. > > > > Now that the debugger support is an official feature, should we have a > > Hudson job that builds Cython and all tests with gdb support enabled? > > Yes, that would make sense. We'd have to build our own newer gdb of > course, and a debug version of Python. > > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > That would be great. In that case, is it possible that my branch will make it to Hudson? Otherwise Hudson will only be useful after merging into mainline, which is probably just a few hours or days before the release. In any case it would also help prevent for instance the issue we had now where my branch did introduce a problem just before the release (in the future I will run the full test suite instead of skipping some of them with those options to runtests.py). If that's a hassle I wouldn't mind mainline branch access either, I promise I'll be good :) The important thing is Python with debug symbols, it does not necessarily have to be a debug build. The thing is that at least many Linux distros ship debug symbols with the debug build (usually Python is compiled with -g, but then stripped). This is for instance the case in Ubuntu (and probably Debian). However, in Fedora for instance even the debug build is shipped without symbols, and they need to be installed separately. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/b916e7d7/attachment-0001.htm From stefan_ml at behnel.de Fri Jan 14 12:50:16 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 14 Jan 2011 12:50:16 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> <4D30332D.3000704@behnel.de> Message-ID: <4D303878.8090308@behnel.de> mark florisson, 14.01.2011 12:45: > is it possible that my branch will make > it to Hudson? [...] If that's a hassle I wouldn't mind mainline branch > access either, I promise I'll be good :) I'm ok with giving you write access to the main branch, so that you can merge your changes over yourself. Stefan From robertwb at math.washington.edu Fri Jan 14 19:06:03 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 14 Jan 2011 10:06:03 -0800 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> <4D30332D.3000704@behnel.de> Message-ID: On Fri, Jan 14, 2011 at 3:45 AM, mark florisson wrote: > > > On 14 January 2011 12:31, Robert Bradshaw > wrote: >> >> On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel >> wrote: >> > Robert Bradshaw, 14.01.2011 12:12: >> >> On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: >> >>> On 14 January 2011 12:08, mark florisson wrote: >> >>>> On 14 January 2011 12:06, Stefan Behnel wrote: >> >>>>> The build complains because it cannot find a file called >> >>>>> >> >>>>> Cython/Debugger/do_repeat.pyx >> >>>> >> >>> Apparently something left that should have been removed, apologies. >> >>> You can >> >>> remove it from the setup.py in the list on line 104. >> >> >> >> Done. >> > >> > Ok, that looks better. >> > >> > Now that the debugger support is an official feature, should we have a >> > Hudson job that builds Cython and all tests with gdb support enabled? >> >> Yes, that would make sense. We'd have to build our own newer gdb of >> course, and a debug version of Python. >> >> - Robert >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev > > That would be great. In that case, is it possible that my branch will make > it to Hudson? Otherwise Hudson will only be useful after merging into > mainline, which is probably just a few hours or days before the release. In > any case it would also help prevent for instance the issue we had now where > my branch did introduce a problem just before the release (in the future I > will run the full test suite instead of skipping some of them with those > options to runtests.py). If that's a hassle I wouldn't mind mainline branch > access either, I promise I'll be good :) I've read enough of your code that I think that'd be fine. I think it makes sense to have a separate branch for the debugger stuff so things could be caught before merging. > The important thing is Python with debug symbols, it does not necessarily > have to be a debug build. The thing is that at least many Linux distros ship > debug symbols with the debug build (usually Python is compiled with -g, but > then stripped). This is for instance the case in Ubuntu (and probably > Debian). However, in Fedora for instance even the debug build is shipped > without symbols, and they need to be installed separately. We build our own Pythons, so could include debug symbols in them. Other than that, is the only requirement to have a working gdb >= 7.2 in the path somewhere? - Robert From markflorisson88 at gmail.com Fri Jan 14 19:10:30 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 14 Jan 2011 19:10:30 +0100 Subject: [Cython] Cython 0.14.1? In-Reply-To: References: <4D2ED67B.2060101@behnel.de> <4D2EDA41.4000109@student.matnat.uio.no> <4D300113.8020408@behnel.de> <4D302E50.1010503@behnel.de> <4D30332D.3000704@behnel.de> Message-ID: On 14 January 2011 19:06, Robert Bradshaw wrote: > On Fri, Jan 14, 2011 at 3:45 AM, mark florisson > wrote: > > > > > > On 14 January 2011 12:31, Robert Bradshaw > > wrote: > >> > >> On Fri, Jan 14, 2011 at 3:27 AM, Stefan Behnel > >> wrote: > >> > Robert Bradshaw, 14.01.2011 12:12: > >> >> On Fri, Jan 14, 2011 at 3:11 AM, mark florisson wrote: > >> >>> On 14 January 2011 12:08, mark florisson wrote: > >> >>>> On 14 January 2011 12:06, Stefan Behnel wrote: > >> >>>>> The build complains because it cannot find a file called > >> >>>>> > >> >>>>> Cython/Debugger/do_repeat.pyx > >> >>>> > >> >>> Apparently something left that should have been removed, apologies. > >> >>> You can > >> >>> remove it from the setup.py in the list on line 104. > >> >> > >> >> Done. > >> > > >> > Ok, that looks better. > >> > > >> > Now that the debugger support is an official feature, should we have a > >> > Hudson job that builds Cython and all tests with gdb support enabled? > >> > >> Yes, that would make sense. We'd have to build our own newer gdb of > >> course, and a debug version of Python. > >> > >> - Robert > >> _______________________________________________ > >> Cython-dev mailing list > >> Cython-dev at codespeak.net > >> http://codespeak.net/mailman/listinfo/cython-dev > > > > That would be great. In that case, is it possible that my branch will > make > > it to Hudson? Otherwise Hudson will only be useful after merging into > > mainline, which is probably just a few hours or days before the release. > In > > any case it would also help prevent for instance the issue we had now > where > > my branch did introduce a problem just before the release (in the future > I > > will run the full test suite instead of skipping some of them with those > > options to runtests.py). If that's a hassle I wouldn't mind mainline > branch > > access either, I promise I'll be good :) > > I've read enough of your code that I think that'd be fine. I think it > makes sense to have a separate branch for the debugger stuff so things > could be caught before merging. > > > The important thing is Python with debug symbols, it does not necessarily > > have to be a debug build. The thing is that at least many Linux distros > ship > > debug symbols with the debug build (usually Python is compiled with -g, > but > > then stripped). This is for instance the case in Ubuntu (and probably > > Debian). However, in Fedora for instance even the debug build is shipped > > without symbols, and they need to be installed separately. > > We build our own Pythons, so could include debug symbols in them. > Other than that, is the only requirement to have a working gdb >= 7.2 > in the path somewhere? Yes. And gdb must be build with Python support, but I believe that that's the default. > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110114/eb3772f0/attachment.htm From vitja.makarov at gmail.com Sat Jan 15 15:02:09 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 15 Jan 2011 17:02:09 +0300 Subject: [Cython] string literal parsing problem Message-ID: Hi! Looking into pyregr test log, I found that this code crashes cython compiler: print('\uXX') Here is traceback: /home/vitja/python/2.7/bin/python ../cython.py -v x11.py -o x11.c Compiling /home/vitja/work/cython-vitek.git/zzz/x11.py Traceback (most recent call last): File "../cython.py", line 8, in main(command_line = 1) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 806, in main result = compile(sources, options) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 781, in compile return compile_multiple(source, options) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 753, in compile_multiple result = run_pipeline(source, options) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 617, in run_pipeline err, enddata = context.run_pipeline(pipeline, source) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 244, in run_pipeline data = phase(data) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 567, in parse tree = context.parse(source_desc, scope, pxd = 0, full_module_name = full_module_name) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Main.py", line 506, in parse tree = Parsing.p_module(s, pxd, full_module_name) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 2810, in p_module body = p_statement_list(s, Ctx(level = level), first_statement = 1) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1773, in p_statement_list stats.append(p_statement(s, ctx, first_statement = first_statement)) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1766, in p_statement s, ctx, first_statement = first_statement) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1627, in p_simple_statement_list stat = p_simple_statement(s, first_statement = first_statement) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1597, in p_simple_statement node = p_print_statement(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1093, in p_print_statement args.append(p_test(s)) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 123, in p_test expr = p_or_test(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 144, in p_or_test return p_rassoc_binop_expr(s, ('or',), p_and_test) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 147, in p_rassoc_binop_expr n1 = p_subexpr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 160, in p_and_test return p_rassoc_binop_expr(s, ('and',), p_not_test) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 147, in p_rassoc_binop_expr n1 = p_subexpr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 170, in p_not_test return p_comparison(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 176, in p_comparison n1 = p_starred_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 200, in p_starred_expr expr = p_bit_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 242, in p_bit_expr return p_binop_expr(s, ('|',), p_xor_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 247, in p_xor_expr return p_binop_expr(s, ('^',), p_and_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 252, in p_and_expr return p_binop_expr(s, ('&',), p_shift_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 257, in p_shift_expr return p_binop_expr(s, ('<<', '>>'), p_arith_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 262, in p_arith_expr return p_binop_expr(s, ('+', '-'), p_term) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 267, in p_term return p_binop_expr(s, ('*', '/', '%', '//'), p_factor) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 273, in p_factor return _p_factor(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 292, in _p_factor return p_power(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 354, in p_power n1 = p_atom(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 577, in p_atom result = p_testlist_comp(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 1012, in p_testlist_comp expr = p_test_or_starred_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 191, in p_test_or_starred_expr return p_test(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 123, in p_test expr = p_or_test(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 144, in p_or_test return p_rassoc_binop_expr(s, ('or',), p_and_test) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 147, in p_rassoc_binop_expr n1 = p_subexpr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 160, in p_and_test return p_rassoc_binop_expr(s, ('and',), p_not_test) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 147, in p_rassoc_binop_expr n1 = p_subexpr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 170, in p_not_test return p_comparison(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 176, in p_comparison n1 = p_starred_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 200, in p_starred_expr expr = p_bit_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 242, in p_bit_expr return p_binop_expr(s, ('|',), p_xor_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 247, in p_xor_expr return p_binop_expr(s, ('^',), p_and_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 252, in p_and_expr return p_binop_expr(s, ('&',), p_shift_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 257, in p_shift_expr return p_binop_expr(s, ('<<', '>>'), p_arith_expr) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 262, in p_arith_expr return p_binop_expr(s, ('+', '-'), p_term) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 267, in p_term return p_binop_expr(s, ('*', '/', '%', '//'), p_factor) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 78, in p_binop_expr n1 = p_sub_expr(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 273, in p_factor return _p_factor(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 292, in _p_factor return p_power(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 354, in p_power n1 = p_atom(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 600, in p_atom kind, bytes_value, unicode_value = p_cat_string_literal(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 682, in p_cat_string_literal kind, bytes_value, unicode_value = p_string_literal(s) File "/home/vitja/work/cython-vitek.git/Cython/Compiler/Parsing.py", line 788, in p_string_literal chrval = int(systr[2:], 16) ValueError: invalid literal for int() with base 16: '' -- vitja. From stefan_ml at behnel.de Sat Jan 15 17:09:47 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 17:09:47 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: Message-ID: <4D31C6CB.8030901@behnel.de> Vitja Makarov, 15.01.2011 15:02: > Looking into pyregr test log, I found that this code crashes cython compiler: > > print('\uXX') > > Here is traceback: > > File "Cython/Compiler/Parsing.py", line 788, in p_string_literal > chrval = int(systr[2:], 16) > ValueError: invalid literal for int() with base 16: '' Hmm, right, the scanner notices that '\uXX' is not a valid Unicode escape sequence and reads it as '\u' + 'XX'. Good catch, I'll fix it. http://trac.cython.org/cython_trac/ticket/647 Stefan From vitja.makarov at gmail.com Sat Jan 15 17:21:57 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 15 Jan 2011 19:21:57 +0300 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31C6CB.8030901@behnel.de> References: <4D31C6CB.8030901@behnel.de> Message-ID: 2011/1/15 Stefan Behnel : > Vitja Makarov, 15.01.2011 15:02: >> Looking into pyregr test log, I found that this code crashes cython compiler: >> >> print('\uXX') >> >> Here is traceback: >> >> ? ?File "Cython/Compiler/Parsing.py", line 788, in p_string_literal >> ? ? ?chrval = int(systr[2:], 16) >> ValueError: invalid literal for int() with base 16: '' > > Hmm, right, the scanner notices that '\uXX' is not a valid Unicode escape > sequence and reads it as '\u' + 'XX'. > > Good catch, I'll fix it. > > http://trac.cython.org/cython_trac/ticket/647 > Please notice that '\u' is valid string but not unicode string, so it's valid in py2 and not py3. -- vitja. From stefan_ml at behnel.de Sat Jan 15 17:29:45 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 17:29:45 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> Message-ID: <4D31CB79.5070203@behnel.de> Vitja Makarov, 15.01.2011 17:21: > 2011/1/15 Stefan Behnel: >> Vitja Makarov, 15.01.2011 15:02: >>> Looking into pyregr test log, I found that this code crashes cython compiler: >>> >>> print('\uXX') >>> >>> Here is traceback: >>> >>> File "Cython/Compiler/Parsing.py", line 788, in p_string_literal >>> chrval = int(systr[2:], 16) >>> ValueError: invalid literal for int() with base 16: '' >> >> Hmm, right, the scanner notices that '\uXX' is not a valid Unicode escape >> sequence and reads it as '\u' + 'XX'. >> >> Good catch, I'll fix it. >> >> http://trac.cython.org/cython_trac/ticket/647 The same applies to hex sequences, BTW. > Please notice that '\u' is valid string but not unicode string, I know, I've written (and rewritten) most of that code. ;-) > so it's valid in py2 and not py3. Nope, it's valid in byte strings but not in unicode strings. Py2/Py3 is not an issue here. Invalid hex sequences should trigger an error in both cases. Stefan From vitja.makarov at gmail.com Sat Jan 15 17:44:36 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 15 Jan 2011 19:44:36 +0300 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31CB79.5070203@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> Message-ID: 2011/1/15 Stefan Behnel : > Vitja Makarov, 15.01.2011 17:21: >> 2011/1/15 Stefan Behnel: >>> Vitja Makarov, 15.01.2011 15:02: >>>> Looking into pyregr test log, I found that this code crashes cython compiler: >>>> >>>> print('\uXX') >>>> >>>> Here is traceback: >>>> >>>> ? ? File "Cython/Compiler/Parsing.py", line 788, in p_string_literal >>>> ? ? ? chrval = int(systr[2:], 16) >>>> ValueError: invalid literal for int() with base 16: '' >>> >>> Hmm, right, the scanner notices that '\uXX' is not a valid Unicode escape >>> sequence and reads it as '\u' + 'XX'. >>> >>> Good catch, I'll fix it. >>> >>> http://trac.cython.org/cython_trac/ticket/647 > > The same applies to hex sequences, BTW. > > >> Please notice that '\u' is valid string but not unicode string, > > I know, I've written (and rewritten) most of that code. ;-) > > >> so it's valid in py2 and not py3. > > Nope, it's valid in byte strings but not in unicode strings. Py2/Py3 is not > an issue here. Invalid hex sequences should trigger an error in both cases. > When I say about py3 I mean that strings are unicode by default. How about this kind of errors: Error converting Pyrex file to C: ------------------------------------------------------------ ... print "\xff" ^ ------------------------------------------------------------ /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a byte string or unicode string explicitly, or adjust the source code encoding. I think it should be ok in py2 mode and give error in py3 -- vitja. From stefan_ml at behnel.de Sat Jan 15 17:48:44 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 17:48:44 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31CB79.5070203@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> Message-ID: <4D31CFEC.5030704@behnel.de> Stefan Behnel, 15.01.2011 17:29: > Vitja Makarov, 15.01.2011 17:21: >> Please notice that '\u' is valid string but not unicode string, >> so it's valid in py2 and not py3. > > Nope, it's valid in byte strings but not in unicode strings. Py2/Py3 is not > an issue here. Hmm, actually, you are right. Py2/Py3 *is* an issue here, because unprefixed strings in Py2 code become unicode strings in Py3. So, if we find an invalid unicode escape sequence, the string must become a plain byte string, and the parsed unicode string must be dropped. I think a Py3 incompatibility warning would be good in that case. Stefan From stefan_ml at behnel.de Sat Jan 15 17:52:36 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 17:52:36 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> Message-ID: <4D31D0D4.708@behnel.de> Vitja Makarov, 15.01.2011 17:44: > print "\xff" > > /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a > byte string or unicode string explicitly, or adjust the source code > encoding. That will go away with the fix. Invalid hex escapes are illegal for both byte strings and unicode strings. Stefan From stefan_ml at behnel.de Sat Jan 15 17:54:15 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 17:54:15 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31CFEC.5030704@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31CFEC.5030704@behnel.de> Message-ID: <4D31D137.7070505@behnel.de> Stefan Behnel, 15.01.2011 17:48: > Stefan Behnel, 15.01.2011 17:29: >> Vitja Makarov, 15.01.2011 17:21: >>> Please notice that '\u' is valid string but not unicode string, >>> so it's valid in py2 and not py3. >> >> Nope, it's valid in byte strings but not in unicode strings. Py2/Py3 is not >> an issue here. > > Hmm, actually, you are right. Py2/Py3 *is* an issue here, because > unprefixed strings in Py2 code become unicode strings in Py3. So, if we > find an invalid unicode escape sequence, the string must become a plain > byte string, and the parsed unicode string must be dropped. > > I think a Py3 incompatibility warning would be good in that case. Actually, no, I think an error is perfectly ok. If a byte string is wanted, use a byte string in the first place. Stefan From stefan_ml at behnel.de Sat Jan 15 18:00:33 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 18:00:33 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31D0D4.708@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D0D4.708@behnel.de> Message-ID: <4D31D2B1.1020703@behnel.de> Stefan Behnel, 15.01.2011 17:52: > Vitja Makarov, 15.01.2011 17:44: >> print "\xff" >> >> /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a >> byte string or unicode string explicitly, or adjust the source code >> encoding. > > That will go away with the fix. Invalid hex escapes are illegal for both > byte strings and unicode strings. Argl. I misread '\xff' as '\xXX', sorry. I'll give it a second try. Stefan From stefan_ml at behnel.de Sat Jan 15 18:23:02 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 18:23:02 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> Message-ID: <4D31D7F6.7010004@behnel.de> Vitja Makarov, 15.01.2011 17:44: > When I say about py3 I mean that strings are unicode by default. For this code '\u' Cython now gives you an error just like you'd get when pasting the above into Python 3. That's how 'str' works in Cython. > How about this kind of errors: > > Error converting Pyrex file to C: > ------------------------------------------------------------ > ... > print "\xff" > ^ > ------------------------------------------------------------ > > /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a > byte string or unicode string explicitly, or adjust the source code > encoding. This error has been gone for good for quite a while now. Where did you see it? Stefan From vitja.makarov at gmail.com Sat Jan 15 18:46:42 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 15 Jan 2011 20:46:42 +0300 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31D7F6.7010004@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> Message-ID: 2011/1/15 Stefan Behnel : > Vitja Makarov, 15.01.2011 17:44: >> When I say about py3 I mean that strings are unicode by default. > > For this code > > ? '\u' > > Cython now gives you an error just like you'd get when pasting the above > into Python 3. That's how 'str' works in Cython. > That's ok to me, but will break some pyregr tests. What will it print for '\u1234'? Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> '\u' '\\u' >>> '\u1234' '\\u1234' >>> I think that '\u' should be translated into '\\u' for python2 > >> How about this kind of errors: >> >> Error converting Pyrex file to C: >> ------------------------------------------------------------ >> ... >> print "\xff" >> ? ? ? ^ >> ------------------------------------------------------------ >> >> /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a >> byte string or unicode string explicitly, or adjust the source code >> encoding. > > This error has been gone for good for quite a while now. Where did you see it? > Hmm, it's still there: vitja at vitja-laptop:~/work/cython.git$ cat foo.pyx print '\xFF' vitja at vitja-laptop:~/work/cython.git$ python cython.py foo.pyx Error compiling Cython file: ------------------------------------------------------------ ... print '\xFF' ^ ------------------------------------------------------------ foo.pyx:1:6: Decoding unprefixed string literal from 'UTF-8' failed. Consider usinga byte string or unicode string explicitly, or adjust the source code encoding. -- vitja. From stefan_ml at behnel.de Sat Jan 15 19:13:46 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 19:13:46 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> Message-ID: <4D31E3DA.505@behnel.de> Vitja Makarov, 15.01.2011 18:46: > 2011/1/15 Stefan Behnel: >> Vitja Makarov, 15.01.2011 17:44: >>> When I say about py3 I mean that strings are unicode by default. >> >> For this code >> >> '\u' >> >> Cython now gives you an error just like you'd get when pasting the above >> into Python 3. That's how 'str' works in Cython. > > That's ok to me, but will break some pyregr tests. I know, but that's what was decided (as the result of a huge amount of discussion). It's there to make the life of the users easier. > What will it print for '\u1234'? > > Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) > [GCC 4.4.5] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> '\u' > '\\u' > >>> '\u1234' > '\\u1234' > >>> > > I think that '\u' should be translated into '\\u' for python2 That's what it does, yes. This works because we actually parse unprefixed strings in parallel as byte strings and unicode strings. However, now that I tried it, I actually get the same result in Py3, although it should have parsed the string correctly. Not sure if we discussed this problem before, but it looks like a bug to me. >>> How about this kind of errors: >>> >>> Error converting Pyrex file to C: >>> ------------------------------------------------------------ >>> ... >>> print "\xff" >>> ^ >>> ------------------------------------------------------------ >>> >>> /tmp/foo.pyx:1:6: String decoding as 'UTF-8' failed. Consider using a >>> byte string or unicode string explicitly, or adjust the source code >>> encoding. >> >> This error has been gone for good for quite a while now. Where did you see it? > > Hmm, it's still there: > > vitja at vitja-laptop:~/work/cython.git$ cat foo.pyx > print '\xFF' > vitja at vitja-laptop:~/work/cython.git$ python cython.py foo.pyx > > Error compiling Cython file: > ------------------------------------------------------------ > ... > print '\xFF' > ^ > ------------------------------------------------------------ > > foo.pyx:1:6: Decoding unprefixed string literal from 'UTF-8' failed. > Consider usinga byte string or unicode string explicitly, or adjust > the source code encoding. Weird, I didn't get that. Anyway, different error message, different place to look then. This is related to the above problem. If both string representations get written into the C file correctly, this is no longer necessary. Stefan From stefan_ml at behnel.de Sat Jan 15 19:20:41 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 19:20:41 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31E3DA.505@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> Message-ID: <4D31E579.6000103@behnel.de> Stefan Behnel, 15.01.2011 19:13: > Vitja Makarov, 15.01.2011 18:46: >> What will it print for '\u1234'? >> >> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >> [GCC 4.4.5] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> '\u' >> '\\u' >>>>> '\u1234' >> '\\u1234' >>>>> >> >> I think that '\u' should be translated into '\\u' for python2 > > That's what it does, yes. This works because we actually parse unprefixed > strings in parallel as byte strings and unicode strings. > > However, now that I tried it, I actually get the same result in Py3, > although it should have parsed the string correctly. Not sure if we > discussed this problem before, but it looks like a bug to me. Thinking about this some more, it's inconsistent either way. 1) If the literal string semantics should be fixed at compile time, you shouldn't get a unicode string in Python 3 in the first place. 2) If the literal should become a byte string in Py2 and a unicode string in Py3, then the unicode string should be what you you'd get if you ran your code in Py3, i.e. the unescaped unicode literal. Given that 1) is out of discussion, 2) should be fixed, IMHO. Stefan From vitja.makarov at gmail.com Sat Jan 15 19:29:44 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 15 Jan 2011 21:29:44 +0300 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31E579.6000103@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> Message-ID: 2011/1/15 Stefan Behnel : > Stefan Behnel, 15.01.2011 19:13: >> Vitja Makarov, 15.01.2011 18:46: >>> What will it print for '\u1234'? >>> >>> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >>> [GCC 4.4.5] on linux2 >>> Type "help", "copyright", "credits" or "license" for more information. >>>>>> '\u' >>> '\\u' >>>>>> '\u1234' >>> '\\u1234' >>>>>> >>> >>> I think that '\u' should be translated into '\\u' for python2 >> >> That's what it does, yes. This works because we actually parse unprefixed >> strings in parallel as byte strings and unicode strings. >> >> However, now that I tried it, I actually get the same result in Py3, >> although it should have parsed the string correctly. Not sure if we >> discussed this problem before, but it looks like a bug to me. > > Thinking about this some more, it's inconsistent either way. > > 1) If the literal string semantics should be fixed at compile time, you > shouldn't get a unicode string in Python 3 in the first place. > > 2) If the literal should become a byte string in Py2 and a unicode string > in Py3, then the unicode string should be what you you'd get if you ran > your code in Py3, i.e. the unescaped unicode literal. > > Given that 1) is out of discussion, 2) should be fixed, IMHO. > Can't we rely on -[23] cython switch? In -2 mode strings are always byte string and -3 always unicode? About '\xFF' error take a look at https://sage.math.washington.edu:8091/hudson/view/cython-devel/job/cython-devel-tests-pyregr-py27-c/294/console There are lots of similar errors reported. test_audioop.py: Error compiling Cython file: ------------------------------------------------------------ ... def test_lin2adpcm(self): # Very cursory test self.assertEqual(audioop.lin2adpcm('\0\0\0\0', 1, None), ('\0\0', (0,0))) def test_lin2alaw(self): self.assertEqual(audioop.lin2alaw(data[0], 1), '\xd5\xc5\xf5') -- vitja. From stefan_ml at behnel.de Sat Jan 15 19:55:46 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 19:55:46 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> Message-ID: <4D31EDB2.2080003@behnel.de> Vitja Makarov, 15.01.2011 19:29: > 2011/1/15 Stefan Behnel: >> Stefan Behnel, 15.01.2011 19:13: >>> Vitja Makarov, 15.01.2011 18:46: >>>> What will it print for '\u1234'? >>>> >>>> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >>>> [GCC 4.4.5] on linux2 >>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>> '\u' >>>> '\\u' >>>>>>> '\u1234' >>>> '\\u1234' >>>>>>> >>>> >>>> I think that '\u' should be translated into '\\u' for python2 >>> >>> That's what it does, yes. This works because we actually parse unprefixed >>> strings in parallel as byte strings and unicode strings. >>> >>> However, now that I tried it, I actually get the same result in Py3, >>> although it should have parsed the string correctly. Not sure if we >>> discussed this problem before, but it looks like a bug to me. >> >> Thinking about this some more, it's inconsistent either way. >> >> 1) If the literal string semantics should be fixed at compile time, you >> shouldn't get a unicode string in Python 3 in the first place. >> >> 2) If the literal should become a byte string in Py2 and a unicode string >> in Py3, then the unicode string should be what you you'd get if you ran >> your code in Py3, i.e. the unescaped unicode literal. >> >> Given that 1) is out of discussion, 2) should be fixed, IMHO. > > Can't we rely on -[23] cython switch? > > In -2 mode strings are always byte string and -3 always unicode? With -3, unprefixed strings *are* unicode strings. As for -2, there isn't currently any change in behaviour when you use that switch, and I feel reluctant to change that. For one, I doubt that anyone would seriously use it. The problem that unicode escapes in unprefixed strings behave differently in Python 2 and Python 3 is unlikely to create problems in real world code, i.e. outside of CPython's regression test suite. > self.assertEqual(audioop.lin2alaw(data[0], 1), '\xd5\xc5\xf5') That's a different problem. You will notice that this code has been fixed to use the 'b' prefix in the Py3 test suite. This is a problem that cannot be solved automatically. For Python 2 code, the compiler cannot know if the user intended an unprefixed literal to be a (binary) byte string or a unicode (text) string. Only a human brain can disambiguate the code here. Remember that Python 2 will also try to decode the above binary bytes literal if it happens to be concatenated with a unicode string for some reason. String handling is structurally hard to get right in Python 2, we have to live with that (and hope that Py2 will die out soon). I think it's a great feature of Cython that it fails fast and thus tells you that your code is ambiguous and requires changes to work in Python 3. It perfectly found the problems in the above code, for one. Stefan From robertwb at math.washington.edu Sat Jan 15 21:26:49 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 15 Jan 2011 12:26:49 -0800 Subject: [Cython] string literal parsing problem In-Reply-To: <4D31EDB2.2080003@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> <4D31EDB2.2080003@behnel.de> Message-ID: On Sat, Jan 15, 2011 at 10:55 AM, Stefan Behnel wrote: > Vitja Makarov, 15.01.2011 19:29: >> 2011/1/15 Stefan Behnel: >>> Stefan Behnel, 15.01.2011 19:13: >>>> Vitja Makarov, 15.01.2011 18:46: >>>>> What will it print for '\u1234'? >>>>> >>>>> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >>>>> [GCC 4.4.5] on linux2 >>>>> Type "help", "copyright", "credits" or "license" for more information. >>>>>>>> '\u' >>>>> '\\u' >>>>>>>> '\u1234' >>>>> '\\u1234' >>>>>>>> >>>>> >>>>> I think that '\u' should be translated into '\\u' for python2 >>>> >>>> That's what it does, yes. This works because we actually parse unprefixed >>>> strings in parallel as byte strings and unicode strings. >>>> >>>> However, now that I tried it, I actually get the same result in Py3, >>>> although it should have parsed the string correctly. Not sure if we >>>> discussed this problem before, but it looks like a bug to me. >>> >>> Thinking about this some more, it's inconsistent either way. >>> >>> 1) If the literal string semantics should be fixed at compile time, you >>> shouldn't get a unicode string in Python 3 in the first place. >>> >>> 2) If the literal should become a byte string in Py2 and a unicode string >>> in Py3, then the unicode string should be what you you'd get if you ran >>> your code in Py3, i.e. the unescaped unicode literal. >>> >>> Given that 1) is out of discussion, 2) should be fixed, IMHO. >> >> Can't we rely on -[23] cython switch? >> >> In -2 mode strings are always byte string and -3 always unicode? > > With -3, unprefixed strings *are* unicode strings. > > As for -2, there isn't currently any change in behaviour when you use that > switch, and I feel reluctant to change that. For one, I doubt that anyone > would seriously use it. The problem that unicode escapes in unprefixed > strings behave differently in Python 2 and Python 3 is unlikely to create > problems in real world code, i.e. outside of CPython's regression test suite. > > >> ? ? ? ? ?self.assertEqual(audioop.lin2alaw(data[0], 1), '\xd5\xc5\xf5') > > That's a different problem. You will notice that this code has been fixed > to use the 'b' prefix in the Py3 test suite. > > This is a problem that cannot be solved automatically. For Python 2 code, > the compiler cannot know if the user intended an unprefixed literal to be a > (binary) byte string or a unicode (text) string. Only a human brain can > disambiguate the code here. Remember that Python 2 will also try to decode > the above binary bytes literal if it happens to be concatenated with a > unicode string for some reason. String handling is structurally hard to get > right in Python 2, we have to live with that (and hope that Py2 will die > out soon). I wouldn't count on it. > I think it's a great feature of Cython that it fails fast and thus tells > you that your code is ambiguous and requires changes to work in Python 3. > It perfectly found the problems in the above code, for one. Whether it's the -2 flag, or something else, we should at least have a mode that handles things exactly as they would be handled in Python 2. Otherwise people won't be able to just compile their existing code without worrying about subtle issues like this. Of course, a compile time error is much more acceptable than different runtime behavior. - Robert From stefan_ml at behnel.de Sat Jan 15 22:02:29 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 15 Jan 2011 22:02:29 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> <4D31EDB2.2080003@behnel.de> Message-ID: <4D320B65.3010102@behnel.de> Robert Bradshaw, 15.01.2011 21:26: > Whether it's the -2 flag, or something else, we should at least have a > mode that handles things exactly as they would be handled in Python 2. > Otherwise people won't be able to just compile their existing code > without worrying about subtle issues like this. Hmm. I wouldn't mind having a mode that compiles Python 2 code and fails fast on compilation under Python 3 with a C "#error" - if someone has enough interest in this to implement it. I certainly don't. While I agree that these issues are subtle, they certainly aren't common enough to really worry about them. Broken code is best worked around by fixing the code. Even the Python 2.x line hasn't always bowed to the holy cow of backwards compatibility. Anyway, given that "-3" doesn't prevent code from working in Python 2, the "-2" flag doesn't seem like a good match. It should be something clear as in "--python2-only". Stefan From robertwb at math.washington.edu Sat Jan 15 22:51:07 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 15 Jan 2011 13:51:07 -0800 Subject: [Cython] string literal parsing problem In-Reply-To: <4D320B65.3010102@behnel.de> References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> <4D31EDB2.2080003@behnel.de> <4D320B65.3010102@behnel.de> Message-ID: On Sat, Jan 15, 2011 at 1:02 PM, Stefan Behnel wrote: > Robert Bradshaw, 15.01.2011 21:26: >> Whether it's the -2 flag, or something else, we should at least have a >> mode that handles things exactly as they would be handled in Python 2. >> Otherwise people won't be able to just compile their existing code >> without worrying about subtle issues like this. > > Hmm. I wouldn't mind having a mode that compiles Python 2 code and fails > fast on compilation under Python 3 with a C "#error" - if someone has > enough interest in this to implement it. I certainly don't. That's an interesting idea. > While I agree > that these issues are subtle, they certainly aren't common enough to really > worry about them. Broken code is best worked around by fixing the code. > Even the Python 2.x line hasn't always bowed to the holy cow of backwards > compatibility. > > Anyway, given that "-3" doesn't prevent code from working in Python 2, the > "-2" flag doesn't seem like a good match. It should be something clear as > in "--python2-only". I think the primary motivation of the -2 flag is so that, eventually, we can make -3 the default without providing a recourse for people who don't want to change their code right away. - Robert From stefan_ml at behnel.de Sun Jan 16 10:33:43 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 16 Jan 2011 10:33:43 +0100 Subject: [Cython] string literal parsing problem In-Reply-To: References: <4D31C6CB.8030901@behnel.de> <4D31CB79.5070203@behnel.de> <4D31D7F6.7010004@behnel.de> <4D31E3DA.505@behnel.de> <4D31E579.6000103@behnel.de> <4D31EDB2.2080003@behnel.de> <4D320B65.3010102@behnel.de> Message-ID: <4D32BB77.1070007@behnel.de> Robert Bradshaw, 15.01.2011 22:51: > I think the primary motivation of the -2 flag is so that, eventually, > we can make -3 the default without providing a recourse for people who > don't want to change their code right away. Exactly. Stefan From stefan_ml at behnel.de Sun Jan 16 11:07:08 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 16 Jan 2011 11:07:08 +0100 Subject: [Cython] Stable ABI mode? Message-ID: <4D32C34C.2010500@behnel.de> Hi, Python 3.2 comes with a stable ABI for extension modules. http://www.python.org/dev/peps/pep-0384/ It would be nice if Cython provided an option to restrict the C-API usage to what the ABI considers fixed and stable. That would disable excesive optimisations like list.pop(), but also disallow several other assumptions that we use all over the place in the generated C code. I'd figure it would be a somewhat involved change, though... Stefan From wjp at usecode.org Mon Jan 17 15:04:09 2011 From: wjp at usecode.org (Willem Jan Palenstijn) Date: Mon, 17 Jan 2011 14:04:09 +0000 Subject: [Cython] decorators and arithmetic operators Message-ID: <20110117140409.GB18240@usecode.org> Hi, When using a decorator on an arithmetic operator such as __mod__ in a cdef class, the method __mod__ is properly replaced, but the corresponding operator doesn't seem to be. See below for an example. Am I doing something wrong, or is this indeed not supported? It would be nice if cython could raise a warning or error in this situation. (Or even better if it would override the operator, but I don't know how feasible that would be given python's special treatment of these arithmetic slots.) Thanks, Willem Jan class DEC: def __init__(self, f): self._f = f def __call__(self, x): print "DEC.__call__ called" cdef class X: @DEC def __mod__(self, x): print "X.__mod__ called" class Y: @DEC def __mod__(self, x): print "Y.__mod__ called" > a = X() > b = Y() > a % 2 X.__mod__ called > b % 2 DEC.__call__ called > a.__mod__(2) DEC.__call__ called > b.__mod__(2) DEC.__call__ called From stefan_ml at behnel.de Mon Jan 17 15:22:19 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Mon, 17 Jan 2011 15:22:19 +0100 Subject: [Cython] decorators and arithmetic operators In-Reply-To: <20110117140409.GB18240@usecode.org> References: <20110117140409.GB18240@usecode.org> Message-ID: <4D34509B.7080401@behnel.de> Willem Jan Palenstijn, 17.01.2011 15:04: > When using a decorator on an arithmetic operator such as __mod__ in a cdef > class, the method __mod__ is properly replaced, but the corresponding > operator doesn't seem to be. See below for an example. > > Am I doing something wrong, or is this indeed not supported? It's not supported. For cdef classes, Cython generates a C function of the appropriate signature and stuffs it into the C slot of the Python type. Since we can't know what the decorator actually does with the function, we can't just call it and expect that the return value is compatible with that slot. It should be possible to fake this feature, i.e. to actually generate a wrapper function for the slot instead that simply calls whatever the decorator actually returned. However, that requires a dedicated implementation and nothing has been done in that direction. I created a ticket for this: http://trac.cython.org/cython_trac/ticket/648 > It would be nice if cython could raise a warning or error in this situation. Absolutely. http://trac.cython.org/cython_trac/ticket/649 Stefan From robertwb at math.washington.edu Mon Jan 17 19:32:33 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Mon, 17 Jan 2011 10:32:33 -0800 Subject: [Cython] Stable ABI mode? In-Reply-To: <4D32C34C.2010500@behnel.de> References: <4D32C34C.2010500@behnel.de> Message-ID: On Sun, Jan 16, 2011 at 2:07 AM, Stefan Behnel wrote: > Hi, > > Python 3.2 comes with a stable ABI for extension modules. > > http://www.python.org/dev/peps/pep-0384/ I hadn't seen that, interesting. I can see how this would be useful for Windows users. > It would be nice if Cython provided an option to restrict the C-API usage > to what the ABI considers fixed and stable. That would disable excesive > optimisations like list.pop(), but also disallow several other assumptions > that we use all over the place in the generated C code. > > I'd figure it would be a somewhat involved change, though... It would be nice to have the option, but I'm not sure how restrictive it would be. For example, would we even be able to construct tracebacks for exception propagation? It's probably worth making an attempt though to see how hard it would be (and, if necessary, perhaps we could push to extend the ABI). - Robert From stefan_ml at behnel.de Tue Jan 18 08:46:14 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 18 Jan 2011 08:46:14 +0100 Subject: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?] Message-ID: <4D354546.1090009@behnel.de> Hi Terry, thanks a lot for this huge list of feedback. I'm forwarding it to the two Cython mailing lists here. We're about to release 0.14.1 real soon, so this is the perfect time to go through the documentation and fix it up. Thanks! Stefan -------- Original-Nachricht -------- Betreff: Re: [OT] Python like lanugages [was Re: After C++, what with Python?] Datum: Mon, 17 Jan 2011 16:38:34 -0500 Von: Terry Reedy An: Stefan Behnel > So seriously need to take a look at Cython. > > http://cython.org OK. Some comments... http://docs.cython.org/src/quickstart/build.html Building a Cython module using distutils This section jumps from that topic to Sage notebook use. It seems that a section heading is missing. http://docs.cython.org/src/quickstart/cythonize.html Typing Variables Simply compiling this in Cython merely gives a 35% speedup. With what arguments? Suggest inserting "for integrate_f(0,2,.1)" or whatever. Typing Functions cdef double f(double) except? -2: return x**2-x I presume '...f(double x)...' "The except? -2 means that an error will be checked for if -2 is returned (though the ? indicates that -2 may also be used as a valid return value)." Huh? the minimum value is -.25 at .5, so why '-2' versus -100000 or any other impossible return? Speedup: 150 times over pure Python. I presume this refers back to non-cdefed integrate_f. Perhaps change this to "Integrate_f(...) is now 150 times as fast as the original pure Python version. http://docs.cython.org/src/tutorial/external.html It is perfectly OK do from math import sin to use Python?s sin() function. . However, calling C?s own sin() function is substantially faster, especially in tight loops. This and some of the following seem to be missing a bit of context. My suggested expansion. Suppose in the previous integration example we change f to return sin(x*x) instead of x**2-x. We could do from math import Python's sin() function. However, this reintroduces a slow Python call within the main for loop. Calling C?s own sin() is substantially faster. http://docs.cython.org/src/tutorial/clibraries.html Using C libraries As seen for the C string decoding functions above, ??? This is the beginning of the second sentence and the only thing 'above' is the integrate example and C sin function. it is actually trivial to call C functions directly in the code. The following describes what needs to be done to use an external C library in Cython code. This seems like it is drawing a false distinction, in that calling the C sin function *is* use of the external math library where it resides. (Or am I missing an important distinction?) Thus is seems to unnecessarily disconnect what the reader already knows from the following much more complicated example. Could cdef extern from "math.h": double sin(double) be put in a separate .pxd file, that is then imported? If so, I suggest showing that variation of the easy example and then go to the new and much harder example. cimport python_exc This is, I believe, the first mention of 'python_exc' and there is no explanation thereafter. My guess by analogy: it is a Cython-supplied .pxd (like cqueue.pxd) that (re)defines the c-api to builtin Python exceptions. (Reading further I see 'python_exc.PyErr_NoMemory()', so my guess is not completely right.) PYTHONPATH=. python -c 'import queue.Queue as Q; Q()' Is this missing a ; or something? ...the pop() method is almost identical. if cqueue.queue_is_empty(self._c_queue): raise IndexError("Queue is empty") If the queue had one item (0) before the pop, would not this be true *after* the pop, and thus the exception erroneous? Unlike with peeking, I think you need to unconditionally check before the pop. def __nonzero__(self): Adding '# __bool__ in Python 3' would be nice. The following listing shows the complete implementation that uses cpdef methods where possible. This feature is obviously not available for the extend() method, as the method signature is incompatible with Python argument types. I would like the example to show how to deal with this to make the class completely usable from Python. Could not one rename 'extend' as 'extend_c' and define extend with Python types. Would cpdef extend(self, it): for i in it: self.append(i) work? Would append raise an error for non ints. Or how about cpdef extend(self, it): for i in it: if isinstance(i,int) and lo <= i <= hi: if not cqueue.queue_push_tail(self._c_queue, value): python_exc.PyErr_NoMemory() else: raise ValueError('bad value') with 'lo' and 'hi' replaced with appropriate expressions. http://docs.cython.org/src/tutorial/cdef_classes.html Extension types Cython supports a second kind of class: extension types, We have, of course, just seen one -- cdef class Queue: Since the argument is typed, we need to check whether it is None. I do not get why the special concern for None. It seems to me that any object other than a Function would (likely) have an incompatible internal structure. In other words, why is the check not if not isinstance(f, Function): raise ValueError("f must be a Function instance") A few other details confuse me, but enough for now. My main interest at the moment is whether Cython is a viable third method (versus swig and ctypes) for wrapping C library code for access from Python. It seems to sit in between somewhat. -- Terry Jan Reedy From robertwb at math.washington.edu Tue Jan 18 09:56:42 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 18 Jan 2011 00:56:42 -0800 Subject: [Cython] Cython 0.14.1 release candidate Message-ID: A release candidate is up at http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there shouldn't be many issues, there's just a pile of bugfixes on top of 0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1 - Robert From stefan_ml at behnel.de Tue Jan 18 10:13:54 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 18 Jan 2011 10:13:54 +0100 Subject: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?] In-Reply-To: <4D34B6DA.7000904@udel.edu> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> Message-ID: <4D3559D2.3070805@behnel.de> Terry Reedy, 17.01.2011 22:38: > A few other details confuse me, but enough for now. It's helpful to be reminded from time to time that the documentation shows the usual "features" of an underfinanced OpenSource project. Part of it was inherited from the original Pyrex documentation and suffers from bit rot, and some other parts were copied over from the Wiki or from talk notes and continue to be badly integrated with the rest. Maybe we should call out a docathon on the cython-users list to see if we can't get some of our happy users to give something back by fixing up the obvious problems in the documentation. Some have done so in the past already, even without being asked. > My main interest at the > moment is whether Cython is a viable third method (versus swig and ctypes) > for wrapping C library code for access from Python. It seems to sit in > between somewhat. Regarding a comparison with SWIG, you might find this interesting: http://www.mail-archive.com/cython-dev at codespeak.net/msg01354.html I do not consider SWIG a real alternative, except when it can play its single joker, i.e. you want to generate and maintain a substantial set of identical wrappers for different languages. You might want to do that if you are the author of the library you wrap, but it's a lot less likely that you want to do it if you are a user of the library. I consider ctypes a viable alternative with two main advantages: a) it's part of CPython, which b) makes it plain Python to work with. Even PyPy supports it (IIRC), and Jython has at least been thinking about it for a while. Cython will always depend on CPython (*), on a C compiler, and on the ability to install and use binary extension modules. However, once you cross that barrier, the main advantages of Cython strike immediately: it has a much more natural way to deal with external C code than ctypes (assuming the same level of C knowledge that you also need for ctypes), and it's substantially faster. Stefan (*) or at least on its C-API - there's also work being done on a port to IronPython From wstein at gmail.com Tue Jan 18 10:46:20 2011 From: wstein at gmail.com (William Stein) Date: Tue, 18 Jan 2011 01:46:20 -0800 Subject: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?] In-Reply-To: <4D3559D2.3070805@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> Message-ID: On Tue, Jan 18, 2011 at 1:13 AM, Stefan Behnel wrote: > Terry Reedy, 17.01.2011 22:38: >> A few other details confuse me, but enough for now. > > It's helpful to be reminded from time to time that the documentation shows > the usual "features" of an underfinanced OpenSource project. Part of it was > inherited from the original Pyrex documentation and suffers from bit rot, > and some other parts were copied over from the Wiki or from talk notes and > continue to be badly integrated with the rest. I recently got some new nontrivial NSF funding for a Cython workshop. A "docathon" would be a great activity at such a workshop. When some Cython developers are ready to all get flown somewhere cool to work together on Cython for a few days, all expenses paid, let me know. I'm serious. -- William > > Maybe we should call out a docathon on the cython-users list to see if we > can't get some of our happy users to give something back by fixing up the > obvious problems in the documentation. Some have done so in the past > already, even without being asked. > > >> My main interest at the >> moment is whether Cython is a viable third method (versus swig and ctypes) >> for wrapping C library code for access from Python. It seems to sit in >> between somewhat. > > Regarding a comparison with SWIG, you might find this interesting: > > http://www.mail-archive.com/cython-dev at codespeak.net/msg01354.html > > I do not consider SWIG a real alternative, except when it can play its > single joker, i.e. you want to generate and maintain a substantial set of > identical wrappers for different languages. You might want to do that if > you are the author of the library you wrap, but it's a lot less likely that > you want to do it if you are a user of the library. > > I consider ctypes a viable alternative with two main advantages: a) it's > part of CPython, which b) makes it plain Python to work with. Even PyPy > supports it (IIRC), and Jython has at least been thinking about it for a while. > > Cython will always depend on CPython (*), on a C compiler, and on the > ability to install and use binary extension modules. However, once you > cross that barrier, the main advantages of Cython strike immediately: it > has a much more natural way to deal with external C code than ctypes > (assuming the same level of C knowledge that you also need for ctypes), and > it's substantially faster. > > Stefan > > > (*) or at least on its C-API - there's also work being done on a port to > IronPython > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -- William Stein Professor of Mathematics University of Washington http://wstein.org From stefan_ml at behnel.de Tue Jan 18 11:31:22 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 18 Jan 2011 11:31:22 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> Message-ID: <4D356BFA.1050606@behnel.de> William Stein, 18.01.2011 10:46: > I recently got some new nontrivial NSF funding for a Cython workshop. > A "docathon" would be a great activity at such a workshop. > > When some Cython developers are ready to all get flown somewhere cool > to work together on Cython for a few days, all expenses paid, let me > know. I'm serious. Hmm, in case the funding allows non-US territory - anyone interested in coming to Munich? Maybe not the coolest place in the world but not too bad either. And the (tiny) local PUG is based in the local University, so I should be able to get some infrastructure up for a weekend and likely a couple of days around that. We could advertise a couple of public talks to increase their interest. Bonus: last I heard, it's a lot easier to get into Germany than into the US part of North America. :) Stefan From vitja.makarov at gmail.com Tue Jan 18 11:50:03 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 18 Jan 2011 13:50:03 +0300 Subject: [Cython] Cython workshop In-Reply-To: <4D356BFA.1050606@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> Message-ID: 2011/1/18 Stefan Behnel : > William Stein, 18.01.2011 10:46: >> I recently got some new nontrivial NSF funding for a Cython workshop. >> ? ?A "docathon" would be a great activity at such a workshop. >> >> When some Cython developers are ready to all get flown somewhere cool >> to work together on Cython for a few days, all expenses paid, let me >> know. ?I'm serious. > > Hmm, in case the funding allows non-US territory - anyone interested in > coming to Munich? Maybe not the coolest place in the world but not too bad > either. And the (tiny) local PUG is based in the local University, so I > should be able to get some infrastructure up for a weekend and likely a > couple of days around that. We could advertise a couple of public talks to > increase their interest. > > Bonus: last I heard, it's a lot easier to get into Germany than into the US > part of North America. :) > +1 for Munich -- vitja. From stefan_ml at behnel.de Tue Jan 18 15:44:07 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 18 Jan 2011 15:44:07 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D356BFA.1050606@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> Message-ID: <4D35A737.40701@behnel.de> Stefan Behnel, 18.01.2011 11:31: > William Stein, 18.01.2011 10:46: >> I recently got some new nontrivial NSF funding for a Cython workshop. >> A "docathon" would be a great activity at such a workshop. >> >> When some Cython developers are ready to all get flown somewhere cool >> to work together on Cython for a few days, all expenses paid, let me >> know. I'm serious. > > Hmm, in case the funding allows non-US territory - anyone interested in > coming to Munich? Maybe not the coolest place in the world but not too bad > either. Just in case: the Wikipedia page has a couple of pretty representative pictures (at least of what M?nchen wants to show you). http://en.wikipedia.org/wiki/Munich (but please don't ask me to organise the trip to Neuschwanstein for you ;) Stefan From markflorisson88 at gmail.com Tue Jan 18 16:07:33 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 16:07:33 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D35A737.40701@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> <4D35A737.40701@behnel.de> Message-ID: On 18 January 2011 15:44, Stefan Behnel wrote: > Stefan Behnel, 18.01.2011 11:31: > > William Stein, 18.01.2011 10:46: > >> I recently got some new nontrivial NSF funding for a Cython workshop. > >> A "docathon" would be a great activity at such a workshop. > >> > >> When some Cython developers are ready to all get flown somewhere cool > >> to work together on Cython for a few days, all expenses paid, let me > >> know. I'm serious. > > > > Hmm, in case the funding allows non-US territory - anyone interested in > > coming to Munich? Maybe not the coolest place in the world but not too > bad > > either. > > Just in case: the Wikipedia page has a couple of pretty representative > pictures (at least of what M?nchen wants to show you). > > http://en.wikipedia.org/wiki/Munich Looks nice :) Do funds also extend to the debugger guy? > > (but please don't ask me to organise the trip to Neuschwanstein for you ;) > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/ae30ad33/attachment.htm From stefan_ml at behnel.de Tue Jan 18 16:21:00 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 18 Jan 2011 16:21:00 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> <4D35A737.40701@behnel.de> Message-ID: <4D35AFDC.9090000@behnel.de> mark florisson, 18.01.2011 16:07: > On 18 January 2011 15:44, Stefan Behnel wrote: >> Stefan Behnel, 18.01.2011 11:31: >>> William Stein, 18.01.2011 10:46: >>>> I recently got some new nontrivial NSF funding for a Cython workshop. >>>> A "docathon" would be a great activity at such a workshop. >>>> >>>> When some Cython developers are ready to all get flown somewhere cool >>>> to work together on Cython for a few days, all expenses paid, let me >>>> know. I'm serious. >>> >>> Hmm, in case the funding allows non-US territory - anyone interested in >>> coming to Munich? Maybe not the coolest place in the world but not too >> bad >>> either. >> >> Just in case: the Wikipedia page has a couple of pretty representative >> pictures (at least of what M?nchen wants to show you). >> >> http://en.wikipedia.org/wiki/Munich > > Looks nice :) Do funds also extend to the debugger guy? I do hope so. You sure want to give a tutorial, right? Stefan From markflorisson88 at gmail.com Tue Jan 18 16:25:43 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 16:25:43 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D35AFDC.9090000@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> <4D35A737.40701@behnel.de> <4D35AFDC.9090000@behnel.de> Message-ID: On 18 January 2011 16:21, Stefan Behnel wrote: > mark florisson, 18.01.2011 16:07: > > On 18 January 2011 15:44, Stefan Behnel wrote: > >> Stefan Behnel, 18.01.2011 11:31: > >>> William Stein, 18.01.2011 10:46: > >>>> I recently got some new nontrivial NSF funding for a Cython workshop. > >>>> A "docathon" would be a great activity at such a workshop. > >>>> > >>>> When some Cython developers are ready to all get flown somewhere cool > >>>> to work together on Cython for a few days, all expenses paid, let me > >>>> know. I'm serious. > >>> > >>> Hmm, in case the funding allows non-US territory - anyone interested in > >>> coming to Munich? Maybe not the coolest place in the world but not too > >> bad > >>> either. > >> > >> Just in case: the Wikipedia page has a couple of pretty representative > >> pictures (at least of what M?nchen wants to show you). > >> > >> http://en.wikipedia.org/wiki/Munich > > > > Looks nice :) Do funds also extend to the debugger guy? > > I do hope so. You sure want to give a tutorial, right? > Definitely, I think that would be nice! :) > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/d56bbe9c/attachment.htm From arfrever.fta at gmail.com Tue Jan 18 18:50:17 2011 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Tue, 18 Jan 2011 18:50:17 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: <201101181850.18448.Arfrever.FTA@gmail.com> Test failure with Python 2.6 and 2.7: ====================================================================== FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/TestLibCython.py", line 204, in test_all self.assertEquals(0, self.p.wait(), errmsg) AssertionError: ****************************** v INSIDE GDB v ****************************** Python was not compiled with debug symbols (or it was stripped). Some functionality may not work (properly). /var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/test_libcython_in_gdb.py:428: UserWarning: Unable to run tests, Python was not compiled with debugging information. Either compile python with -g or get a debug build (configure with --with-pydebug). warnings.warn(msg) ****************************** ^ INSIDE GDB ^ ****************************** ====================================================================== Test suite fails completely with Python 3.1 and 3.2: Python 3.1.3+ (release31-maint:88040, Jan 16 2011, 21:40:20) [GCC 4.4.5] Running tests against Cython 0.14.1rc1 Traceback (most recent call last): File "runtests.py", line 1239, in main() File "runtests.py", line 1169, in main collect_unittests(UNITTEST_ROOT, UNITTEST_MODULE + ".", test_suite, selectors) File "runtests.py", line 713, in collect_unittests module = __import__(modulename) File "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", line 25, in with open(codefile) as f: IOError: [Errno 2] No such file or directory: '/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/codefile' -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. Url : http://codespeak.net/pipermail/cython-dev/attachments/20110118/ad5100d9/attachment.pgp From markflorisson88 at gmail.com Tue Jan 18 18:54:03 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 18:54:03 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <201101181850.18448.Arfrever.FTA@gmail.com> References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: On 18 January 2011 18:50, Arfrever Frehtes Taifersar Arahesis < arfrever.fta at gmail.com> wrote: > Test failure with Python 2.6 and 2.7: > > ====================================================================== > FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/TestLibCython.py", > line 204, in test_all > self.assertEquals(0, self.p.wait(), errmsg) > AssertionError: > ****************************** v INSIDE GDB v > ****************************** > Python was not compiled with debug symbols (or it was stripped). Some > functionality may not work (properly). > /var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/test_libcython_in_gdb.py:428: > UserWarning: Unable to run tests, Python was not compiled with debugging > information. Either compile python with -g or get a debug build (configure > with --with-pydebug). > warnings.warn(msg) > ****************************** ^ INSIDE GDB ^ > ****************************** > > ====================================================================== > > Test suite fails completely with Python 3.1 and 3.2: > > Python 3.1.3+ (release31-maint:88040, Jan 16 2011, 21:40:20) > [GCC 4.4.5] > > Running tests against Cython 0.14.1rc1 > > Traceback (most recent call last): > File "runtests.py", line 1239, in > main() > File "runtests.py", line 1169, in main > collect_unittests(UNITTEST_ROOT, UNITTEST_MODULE + ".", test_suite, > selectors) > File "runtests.py", line 713, in collect_unittests > module = __import__(modulename) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", > line 25, in > with open(codefile) as f: > IOError: [Errno 2] No such file or directory: > '/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/codefile' > > -- > Arfrever Frehtes Taifersar Arahesis > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > > Yes, you need to run those tests with a python build for which debug symbols are available. Should I turn this into a warning instead of an error? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/babc0238/attachment.htm From markflorisson88 at gmail.com Tue Jan 18 19:00:50 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 19:00:50 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: On 18 January 2011 18:54, mark florisson wrote: > > > On 18 January 2011 18:50, Arfrever Frehtes Taifersar Arahesis < > arfrever.fta at gmail.com> wrote: > >> Test failure with Python 2.6 and 2.7: >> >> ====================================================================== >> FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> File >> "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/TestLibCython.py", >> line 204, in test_all >> self.assertEquals(0, self.p.wait(), errmsg) >> AssertionError: >> ****************************** v INSIDE GDB v >> ****************************** >> Python was not compiled with debug symbols (or it was stripped). Some >> functionality may not work (properly). >> /var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/test_libcython_in_gdb.py:428: >> UserWarning: Unable to run tests, Python was not compiled with debugging >> information. Either compile python with -g or get a debug build (configure >> with --with-pydebug). >> warnings.warn(msg) >> ****************************** ^ INSIDE GDB ^ >> ****************************** >> >> ====================================================================== >> >> Test suite fails completely with Python 3.1 and 3.2: >> >> Python 3.1.3+ (release31-maint:88040, Jan 16 2011, 21:40:20) >> [GCC 4.4.5] >> >> Running tests against Cython 0.14.1rc1 >> >> Traceback (most recent call last): >> File "runtests.py", line 1239, in >> main() >> File "runtests.py", line 1169, in main >> collect_unittests(UNITTEST_ROOT, UNITTEST_MODULE + ".", test_suite, >> selectors) >> File "runtests.py", line 713, in collect_unittests >> module = __import__(modulename) >> File >> "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", >> line 25, in >> with open(codefile) as f: >> IOError: [Errno 2] No such file or directory: >> '/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/codefile' >> > For this I think I will need to change it to a python module (codefile.py) to have setup.py include it in the installation. I will push a fix shortly. > -- >> Arfrever Frehtes Taifersar Arahesis >> >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> >> Yes, you need to run those tests with a python build for which debug > symbols are available. > > Should I turn this into a warning instead of an error? > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/9745bc45/attachment.htm From markflorisson88 at gmail.com Tue Jan 18 19:07:01 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 19:07:01 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: On an unrelated side note, should we modify the testrunner to continue when importing a test module fails and have it print a traceback instead? -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/8d1980b6/attachment.htm From arfrever.fta at gmail.com Tue Jan 18 19:09:56 2011 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Tue, 18 Jan 2011 19:09:56 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: <201101181909.57479.Arfrever.FTA@gmail.com> 2011-01-18 18:54:03 mark florisson napisa?(a): > Yes, you need to run those tests with a python build for which debug > symbols are available. > > Should I turn this into a warning instead of an error? This test should be automatically skipped when debug symbols are unavailable. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. Url : http://codespeak.net/pipermail/cython-dev/attachments/20110118/3bc240a2/attachment.pgp From markflorisson88 at gmail.com Tue Jan 18 19:40:47 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 18 Jan 2011 19:40:47 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: On 18 January 2011 19:00, mark florisson wrote: > > > On 18 January 2011 18:54, mark florisson wrote: > >> >> >> On 18 January 2011 18:50, Arfrever Frehtes Taifersar Arahesis < >> arfrever.fta at gmail.com> wrote: >> >>> Test failure with Python 2.6 and 2.7: >>> >>> ====================================================================== >>> FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) >>> ---------------------------------------------------------------------- >>> Traceback (most recent call last): >>> File >>> "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/TestLibCython.py", >>> line 204, in test_all >>> self.assertEquals(0, self.p.wait(), errmsg) >>> AssertionError: >>> ****************************** v INSIDE GDB v >>> ****************************** >>> Python was not compiled with debug symbols (or it was stripped). Some >>> functionality may not work (properly). >>> /var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/Cython/Debugger/Tests/test_libcython_in_gdb.py:428: >>> UserWarning: Unable to run tests, Python was not compiled with debugging >>> information. Either compile python with -g or get a debug build (configure >>> with --with-pydebug). >>> warnings.warn(msg) >>> ****************************** ^ INSIDE GDB ^ >>> ****************************** >>> >>> ====================================================================== >>> >>> Test suite fails completely with Python 3.1 and 3.2: >>> >>> Python 3.1.3+ (release31-maint:88040, Jan 16 2011, 21:40:20) >>> [GCC 4.4.5] >>> >>> Running tests against Cython 0.14.1rc1 >>> >>> Traceback (most recent call last): >>> File "runtests.py", line 1239, in >>> main() >>> File "runtests.py", line 1169, in main >>> collect_unittests(UNITTEST_ROOT, UNITTEST_MODULE + ".", test_suite, >>> selectors) >>> File "runtests.py", line 713, in collect_unittests >>> module = __import__(modulename) >>> File >>> "/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", >>> line 25, in >>> with open(codefile) as f: >>> IOError: [Errno 2] No such file or directory: >>> '/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/codefile' >>> >> For this I think I will need to change it to a python module (codefile.py) > to have setup.py include it in the installation. I will push a fix shortly. > MANIFEST.in worked fine too, apparently runtests.py has its own MANIFEST.in file. It seems I can't push to https://github.com/cython/cython.git yet, though. > -- >>> Arfrever Frehtes Taifersar Arahesis >>> >>> _______________________________________________ >>> Cython-dev mailing list >>> Cython-dev at codespeak.net >>> http://codespeak.net/mailman/listinfo/cython-dev >>> >>> Yes, you need to run those tests with a python build for which debug >> symbols are available. >> >> Should I turn this into a warning instead of an error? >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110118/30eba1fd/attachment.htm From robertwb at math.washington.edu Tue Jan 18 20:14:35 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 18 Jan 2011 11:14:35 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: On Tue, Jan 18, 2011 at 10:07 AM, mark florisson wrote: > On an unrelated side note, should we modify the testrunner to continue when > importing a test module fails and have it print a traceback instead? Yes, that would make the most sense. - Robert From robertwb at math.washington.edu Tue Jan 18 20:13:52 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 18 Jan 2011 11:13:52 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <201101181909.57479.Arfrever.FTA@gmail.com> References: <201101181850.18448.Arfrever.FTA@gmail.com> <201101181909.57479.Arfrever.FTA@gmail.com> Message-ID: On Tue, Jan 18, 2011 at 10:09 AM, Arfrever Frehtes Taifersar Arahesis wrote: > 2011-01-18 18:54:03 mark florisson napisa?(a): >> Yes, you need to run those tests with a python build for which debug >> symbols are available. >> >> Should I turn this into a warning instead of an error? > > This test should be automatically skipped when debug symbols are unavailable. +1 Mark, you should be able to push now. - Robert From robertwb at math.washington.edu Tue Jan 18 20:21:09 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 18 Jan 2011 11:21:09 -0800 Subject: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?] In-Reply-To: References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> Message-ID: On Tue, Jan 18, 2011 at 1:46 AM, William Stein wrote: > On Tue, Jan 18, 2011 at 1:13 AM, Stefan Behnel wrote: >> Terry Reedy, 17.01.2011 22:38: >>> A few other details confuse me, but enough for now. >> >> It's helpful to be reminded from time to time that the documentation shows >> the usual "features" of an underfinanced OpenSource project. Part of it was >> inherited from the original Pyrex documentation and suffers from bit rot, >> and some other parts were copied over from the Wiki or from talk notes and >> continue to be badly integrated with the rest. > > I recently got some new nontrivial NSF funding for a Cython workshop. > ?A "docathon" would be a great activity at such a workshop. > > When some Cython developers are ready to all get flown somewhere cool > to work together on Cython for a few days, all expenses paid, let me > know. ?I'm serious. +1, docs would be a good topic. That and, once generators are in, a push for 1.0. - Robert From robertwb at math.washington.edu Tue Jan 18 21:50:05 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 18 Jan 2011 12:50:05 -0800 Subject: [Cython] Cython workshop In-Reply-To: <4D356BFA.1050606@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> Message-ID: On Tue, Jan 18, 2011 at 2:31 AM, Stefan Behnel wrote: > William Stein, 18.01.2011 10:46: >> I recently got some new nontrivial NSF funding for a Cython workshop. >> ? ?A "docathon" would be a great activity at such a workshop. >> >> When some Cython developers are ready to all get flown somewhere cool >> to work together on Cython for a few days, all expenses paid, let me >> know. ?I'm serious. > > Hmm, in case the funding allows non-US territory - anyone interested in > coming to Munich? Maybe not the coolest place in the world but not too bad > either. And the (tiny) local PUG is based in the local University, so I > should be able to get some infrastructure up for a weekend and likely a > couple of days around that. We could advertise a couple of public talks to > increase their interest. Certainly organizing it where someone is local simplifies the logistics, thanks for volunteering. > Bonus: last I heard, it's a lot easier to get into Germany than into the US > part of North America. :) Most certainly true for an EU citizen such as yourself, and several other countries. My last trip through German customs was...bearable--I'd be more than willing to do it again to maximize participation, and Munich looks like a nice place. The other question is one of expense--how many people would we have to ship that direction vs. this. Anyone interested in coming, please reply (on or off list) with your preferences and what you think you could contribute. - Robert From ondrej at certik.cz Tue Jan 18 22:20:01 2011 From: ondrej at certik.cz (Ondrej Certik) Date: Tue, 18 Jan 2011 13:20:01 -0800 Subject: [Cython] [OT] Python like lanugages [was Re: After C++, what with Python?] In-Reply-To: References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> Message-ID: On Tue, Jan 18, 2011 at 1:46 AM, William Stein wrote: > On Tue, Jan 18, 2011 at 1:13 AM, Stefan Behnel wrote: >> Terry Reedy, 17.01.2011 22:38: >>> A few other details confuse me, but enough for now. >> >> It's helpful to be reminded from time to time that the documentation shows >> the usual "features" of an underfinanced OpenSource project. Part of it was >> inherited from the original Pyrex documentation and suffers from bit rot, >> and some other parts were copied over from the Wiki or from talk notes and >> continue to be badly integrated with the rest. > > I recently got some new nontrivial NSF funding for a Cython workshop. > ?A "docathon" would be a great activity at such a workshop. > > When some Cython developers are ready to all get flown somewhere cool > to work together on Cython for a few days, all expenses paid, let me > know. ?I'm serious. I probably won't have time to fly somewhere for a few days, but I'll at least try to join remotely and submit a few patches for docs. Ondrej From dagss at student.matnat.uio.no Tue Jan 18 22:44:31 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 18 Jan 2011 22:44:31 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> Message-ID: <4D3609BF.1080601@student.matnat.uio.no> On 01/18/2011 09:50 PM, Robert Bradshaw wrote: > On Tue, Jan 18, 2011 at 2:31 AM, Stefan Behnel wrote: > >> William Stein, 18.01.2011 10:46: >> >>> I recently got some new nontrivial NSF funding for a Cython workshop. >>> A "docathon" would be a great activity at such a workshop. >>> >>> When some Cython developers are ready to all get flown somewhere cool >>> to work together on Cython for a few days, all expenses paid, let me >>> know. I'm serious. >>> >> Hmm, in case the funding allows non-US territory - anyone interested in >> coming to Munich? Maybe not the coolest place in the world but not too bad >> either. And the (tiny) local PUG is based in the local University, so I >> should be able to get some infrastructure up for a weekend and likely a >> couple of days around that. We could advertise a couple of public talks to >> increase their interest. >> > Certainly organizing it where someone is local simplifies the > logistics, thanks for volunteering. > > >> Bonus: last I heard, it's a lot easier to get into Germany than into the US >> part of North America. :) >> > Most certainly true for an EU citizen such as yourself, and several > other countries. My last trip through German customs > was...bearable--I'd be more than willing to do it again to maximize > participation, and Munich looks like a nice place. The other question > is one of expense--how many people would we have to ship that > direction vs. this. > > Anyone interested in coming, please reply (on or off list) with your > preferences and what you think you could contribute. > I'd probably come to Munich; it's obviously a lot less stressful than going to the US. I agree that we should try to minimize travel distance and time zone shifts over all participants, so we should gather a list of possible attendees (and their location!) before settling anything else IMO. But Munich would be great for me. Dag Sverre From stefan_ml at behnel.de Wed Jan 19 09:02:18 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 19 Jan 2011 09:02:18 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <201101181850.18448.Arfrever.FTA@gmail.com> References: <201101181850.18448.Arfrever.FTA@gmail.com> Message-ID: <4D369A8A.8060008@behnel.de> Arfrever Frehtes Taifersar Arahesis, 18.01.2011 18:50: > IOError: [Errno 2] No such file or directory: '/var/tmp/portage/dev-python/cython-0.14.1_rc1/work/Cython-0.14.1rc1/BUILD/Cy3/Cython/Debugger/Tests/codefile' I had added the file to setup.py's package_data when I noticed that it was missing, but forgot to add it to MANIFEST.in as well. Mark has fixed that now. Stefan From stefan_ml at behnel.de Thu Jan 20 12:06:10 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 12:06:10 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D356BFA.1050606@behnel.de> References: <86a904f8-05b0-435f-81d6-7450650f1679@glegroupsg2000goo.googlegroups.com> <4d329899$0$43988$742ec2ed@news.sonic.net> <7x8vylfcvz.fsf@ruckus.brouhaha.com> <4d32c7cf$0$29983$c3e8da3$5496439d@news.astraweb.com> <4d3425fe$0$29983$c3e8da3$5496439d@news.astraweb.com> <4D34B6DA.7000904@udel.edu> <4D3559D2.3070805@behnel.de> <4D356BFA.1050606@behnel.de> Message-ID: <4D381722.3070609@behnel.de> Stefan Behnel, 18.01.2011 11:31: > William Stein, 18.01.2011 10:46: >> I recently got some new nontrivial NSF funding for a Cython workshop. >> A "docathon" would be a great activity at such a workshop. >> >> When some Cython developers are ready to all get flown somewhere cool >> to work together on Cython for a few days, all expenses paid, let me >> know. I'm serious. > > Hmm, in case the funding allows non-US territory - anyone interested in > coming to Munich? Maybe not the coolest place in the world but not too bad > either. And the (tiny) local PUG is based in the local University, so I > should be able to get some infrastructure up for a weekend and likely a > couple of days around that. We could advertise a couple of public talks to > increase their interest. Quick follow-up on this: I asked hand-waving and it seems like we could get a place to gather at the University for basically as many days as we like if we can manage to organise it before the end of April (lectures restart on May 2nd). Stefan From dagss at student.matnat.uio.no Thu Jan 20 12:26:06 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 20 Jan 2011 12:26:06 +0100 Subject: [Cython] Cython workshop Message-ID: <4D381BCE.30300@student.matnat.uio.no> Starting new thread in case somebody did not see the hijacking. I've created this wiki page: http://wiki.cython.org/workshop1 If you're interested in coming to a Cython workshop, please fill in your details in the table to help decide when and where a workshop should be held. Dag Sverre From stefan_ml at behnel.de Thu Jan 20 15:26:47 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 15:26:47 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D381BCE.30300@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> Message-ID: <4D384627.7030805@behnel.de> Dag Sverre Seljebotn, 20.01.2011 12:26: > I've created this wiki page: > > http://wiki.cython.org/workshop1 I restricted access to that page to authenticated users as we are collecting personal information there. Stefan From vitja.makarov at gmail.com Thu Jan 20 15:29:12 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 20 Jan 2011 17:29:12 +0300 Subject: [Cython] Cython workshop In-Reply-To: <4D384627.7030805@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> Message-ID: 2011/1/20 Stefan Behnel : > Dag Sverre Seljebotn, 20.01.2011 12:26: >> I've created this wiki page: >> >> http://wiki.cython.org/workshop1 > > I restricted access to that page to authenticated users as we are > collecting personal information there. > Can you create an account for me, please? -- vitja. From dagss at student.matnat.uio.no Thu Jan 20 15:34:19 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 20 Jan 2011 15:34:19 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D384627.7030805@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> Message-ID: <4D3847EB.900@student.matnat.uio.no> On 01/20/2011 03:26 PM, Stefan Behnel wrote: > Dag Sverre Seljebotn, 20.01.2011 12:26: > >> I've created this wiki page: >> >> http://wiki.cython.org/workshop1 >> > I restricted access to that page to authenticated users as we are > collecting personal information there. > I think that is counterproductive, because we want to develop that page into the "poster" for the workshop that anyone can visit to see if they're interested in coming. How if we drop listing people's location and instead say where they prefer the conference to be held? And people don't have to list when they're available if they don't want to. Dag Sverre From dagss at student.matnat.uio.no Thu Jan 20 15:36:34 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 20 Jan 2011 15:36:34 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3847EB.900@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> Message-ID: <4D384872.2080908@student.matnat.uio.no> On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: > On 01/20/2011 03:26 PM, Stefan Behnel wrote: > >> Dag Sverre Seljebotn, 20.01.2011 12:26: >> >> >>> I've created this wiki page: >>> >>> http://wiki.cython.org/workshop1 >>> >>> >> I restricted access to that page to authenticated users as we are >> collecting personal information there. >> >> > I think that is counterproductive, because we want to develop that page > into the "poster" for the workshop that anyone can visit to see if > they're interested in coming. > > How if we drop listing people's location and instead say where they > prefer the conference to be held? And people don't have to list when > they're available if they don't want to. > And I guess we'd need to recreate the page to delete editing history, if it's your location that's at stake. Dag Sverre From markflorisson88 at gmail.com Thu Jan 20 15:39:11 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Thu, 20 Jan 2011 15:39:11 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D384872.2080908@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> Message-ID: On 20 January 2011 15:36, Dag Sverre Seljebotn wrote: > On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: > > On 01/20/2011 03:26 PM, Stefan Behnel wrote: > > > >> Dag Sverre Seljebotn, 20.01.2011 12:26: > >> > >> > >>> I've created this wiki page: > >>> > >>> http://wiki.cython.org/workshop1 > >>> > >>> > >> I restricted access to that page to authenticated users as we are > >> collecting personal information there. > >> > >> > > I think that is counterproductive, because we want to develop that page > > into the "poster" for the workshop that anyone can visit to see if > > they're interested in coming. > > > > How if we drop listing people's location and instead say where they > > prefer the conference to be held? And people don't have to list when > > they're available if they don't want to. > > > > And I guess we'd need to recreate the page to delete editing history, if > it's your location that's at stake. I can only speak for myself, but I don't regard this as important personal information that should be withheld from the internet. > Dag Sverre > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110120/92f745ed/attachment.htm From stefan_ml at behnel.de Thu Jan 20 15:48:59 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 15:48:59 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D384872.2080908@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> Message-ID: <4D384B5B.8060602@behnel.de> Dag Sverre Seljebotn, 20.01.2011 15:36: > On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: >> On 01/20/2011 03:26 PM, Stefan Behnel wrote: >> >>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>> >>> >>>> I've created this wiki page: >>>> >>>> http://wiki.cython.org/workshop1 >>>> >>>> >>> I restricted access to that page to authenticated users as we are >>> collecting personal information there. >>> >> I think that is counterproductive, because we want to develop that page >> into the "poster" for the workshop that anyone can visit to see if >> they're interested in coming. Sounds like a different page to me. >> How if we drop listing people's location and instead say where they >> prefer the conference to be held? And people don't have to list when >> they're available if they don't want to. > > And I guess we'd need to recreate the page to delete editing history Not at long as it's access restricted. :) If it's the URL you want to keep, then yes. > if it's your location that's at stake. We've started putting lots of information on that page already. I think it's better to have a planning page and a separate advertising page. Stefan From stefan_ml at behnel.de Thu Jan 20 16:03:27 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 16:03:27 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> Message-ID: <4D384EBF.4030409@behnel.de> Vitja Makarov, 20.01.2011 15:29: > 2011/1/20 Stefan Behnel: >> Dag Sverre Seljebotn, 20.01.2011 12:26: >>> I've created this wiki page: >>> >>> http://wiki.cython.org/workshop1 >> >> I restricted access to that page to authenticated users as we are >> collecting personal information there. >> > > Can you create an account for me, please? Looks like you already have one. It's the same you use for trac. Stefan From dagss at student.matnat.uio.no Thu Jan 20 16:09:18 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Thu, 20 Jan 2011 16:09:18 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D384B5B.8060602@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> <4D384B5B.8060602@behnel.de> Message-ID: <4D38501E.1040703@student.matnat.uio.no> On 01/20/2011 03:48 PM, Stefan Behnel wrote: > Dag Sverre Seljebotn, 20.01.2011 15:36: > >> On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: >> >>> On 01/20/2011 03:26 PM, Stefan Behnel wrote: >>> >>> >>>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>>> >>>> >>>> >>>>> I've created this wiki page: >>>>> >>>>> http://wiki.cython.org/workshop1 >>>>> >>>>> >>>>> >>>> I restricted access to that page to authenticated users as we are >>>> collecting personal information there. >>>> >>>> >>> I think that is counterproductive, because we want to develop that page >>> into the "poster" for the workshop that anyone can visit to see if >>> they're interested in coming. >>> > Sounds like a different page to me. > For this workshop there's a very gray area between announcing and planning. E.g., here's an email I just sent to numpy-discussion. Anyone who's interested enough to read it can always register and it is just a minor annoyance, so I guess I'm fine with it. """ As we have funding for it, we're talking about organizing a Cython workshop sometimes this year (possibly in Munich, Germany, though it's not decided yet). It's still not clear how user-centric vs. developer-centric the workshop will be, or how strong a role numerical computation will have vs. more general language features. We're just getting in touch with people potentially interested in joining the workshop, and then we'll take it from there. Respond on the wiki page or on cython-dev. http://wiki.cython.org/workshop1 """ > Not at long as it's access restricted. :) > Anyone can register, can't they? So this is just for robots/search engines? > We've started putting lots of information on that page already. I think > it's better to have a planning page and a separate advertising page. > Like Mark I don't see what the "lots of information" is supposed to be, and the thought didn't strike me at all until you mentioned it. Then again, academics want their name and location spread around as much as possible anyway :-) I just followed the pattern Sage set up which is reasonably efficient, e.g. http://wiki.sagemath.org/days10 BUT, I can agree that the less privacy-conscious should yield to the ones who are more so, so I'm fine with following your lead here. Dag Sverre -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110120/f6f17079/attachment.htm From stefan_ml at behnel.de Thu Jan 20 16:21:01 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 16:21:01 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D38501E.1040703@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> <4D384B5B.8060602@behnel.de> <4D38501E.1040703@student.matnat.uio.no> Message-ID: <4D3852DD.6080304@behnel.de> Dag Sverre Seljebotn, 20.01.2011 16:09: > On 01/20/2011 03:48 PM, Stefan Behnel wrote: >> Dag Sverre Seljebotn, 20.01.2011 15:36: >>> On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: >>>> On 01/20/2011 03:26 PM, Stefan Behnel wrote: >>>> >>>>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>>>> >>>>> >>>>>> I've created this wiki page: >>>>>> >>>>>> http://wiki.cython.org/workshop1 >>>>>> >>>>>> >>>>> I restricted access to that page to authenticated users as we are >>>>> collecting personal information there. >>>>> >>>> I think that is counterproductive, because we want to develop that page >>>> into the "poster" for the workshop that anyone can visit to see if >>>> they're interested in coming. >> Sounds like a different page to me. > > For this workshop there's a very gray area between announcing and planning. > E.g., here's an email I just sent to numpy-discussion. Anyone who's > interested enough to read it can always register and it is just a minor > annoyance, so I guess I'm fine with it. > > """ > As we have funding for it, we're talking about organizing a Cython workshop > sometimes this year (possibly in Munich, Germany, though it's not decided > yet). > > It's still not clear how user-centric vs. developer-centric the workshop > will be, or how strong a role numerical computation will have vs. more > general language features. We're just getting in touch with people > potentially interested in joining the workshop, and then we'll take it from > there. > > Respond on the wiki page or on cython-dev. > > http://wiki.cython.org/workshop1 > """ > >> Not at long as it's access restricted. :) > > Anyone can register, can't they? No, they can't. You still have to send a htpasswd entry to one of us. > So this is just for robots/search engines? Most of all, yes. >> We've started putting lots of information on that page already. I think >> it's better to have a planning page and a separate advertising page. > > Like Mark I don't see what the "lots of information" is supposed to be, and > the thought didn't strike me at all until you mentioned it. Then again, > academics want their name and location spread around as much as possible > anyway :-) > > I just followed the pattern Sage set up which is reasonably efficient, e.g. > http://wiki.sagemath.org/days10 > > BUT, I can agree that the less privacy-conscious should yield to the ones > who are more so, so I'm fine with following your lead here. I've replaces the page with a new page and moved the complete (restricted) page to http://wiki.cython.org/workshop2011 Stefan From stefan_ml at behnel.de Thu Jan 20 16:45:53 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 20 Jan 2011 16:45:53 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3852DD.6080304@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> <4D384B5B.8060602@behnel.de> <4D38501E.1040703@student.matnat.uio.no> <4D3852DD.6080304@behnel.de> Message-ID: <4D3858B1.6000009@behnel.de> Stefan Behnel, 20.01.2011 16:21: > Dag Sverre Seljebotn, 20.01.2011 16:09: >> Anyone can register, can't they? > > No, they can't. You still have to send a htpasswd entry to one of us. Hmm, I guess I was wrong here. So you actually just need to create an account then. I think that's enough of a privacy 'protection'. >> So this is just for robots/search engines? > > Most of all, yes. Stefan From robertwb at math.washington.edu Thu Jan 20 20:53:26 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 20 Jan 2011 11:53:26 -0800 Subject: [Cython] Cython workshop In-Reply-To: <4D38501E.1040703@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D384627.7030805@behnel.de> <4D3847EB.900@student.matnat.uio.no> <4D384872.2080908@student.matnat.uio.no> <4D384B5B.8060602@behnel.de> <4D38501E.1040703@student.matnat.uio.no> Message-ID: On Thu, Jan 20, 2011 at 7:09 AM, Dag Sverre Seljebotn wrote: > On 01/20/2011 03:48 PM, Stefan Behnel wrote: > > Dag Sverre Seljebotn, 20.01.2011 15:36: > > On 01/20/2011 03:34 PM, Dag Sverre Seljebotn wrote: > > On 01/20/2011 03:26 PM, Stefan Behnel wrote: > > Dag Sverre Seljebotn, 20.01.2011 12:26: > > I've created this wiki page: > > http://wiki.cython.org/workshop1 > > I restricted access to that page to authenticated users as we are > collecting personal information there. Thanks for setting this up. > I think that is counterproductive, because we want to develop that page > into the "poster" for the workshop that anyone can visit to see if > they're interested in coming. > > Sounds like a different page to me. > > For this workshop there's a very gray area between announcing and planning. > E.g., here's an email I just sent to numpy-discussion. +1. There will be more formal planning and announcement once we get a sense for people's interest and who may be able to come. > Anyone who's > interested enough to read it can always register and it is just a minor > annoyance, so I guess I'm fine with it. > > Like Mark I don't see what the "lots of information" is supposed to be, and > the thought didn't strike me at all until you mentioned it. Then again, > academics want their name and location spread around as much as possible > anyway :-) That's how I feel too. In any case, as long as no one posts other people's information, there's no privacy invasion issue. People should feel free to put "contact me" in those boxes if they're sensitive about that kind of stuff. - Robert From stefan_ml at behnel.de Sat Jan 22 09:06:09 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 22 Jan 2011 09:06:09 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D381BCE.30300@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> Message-ID: <4D3A8FF1.3090103@behnel.de> Dag Sverre Seljebotn, 20.01.2011 12:26: > Starting new thread in case somebody did not see the hijacking. > > I've created this wiki page: > > http://wiki.cython.org/workshop1 > > If you're interested in coming to a Cython workshop, please fill in your > details in the table to help decide when and where a workshop should be > held. Current status so far: 4 core developers have signed in, 5 people in total, all from Europe. Robert said he'd like to join in, too. Have there been any further off-list replies so far? In case we actually want to advertise and run this before the end of April (or even March, and Vitja's visa will likely take a while to get), it would be good if the remaining core developers and other interested parties could speak up soon, say, within a week's time, so that we can decide on date and location. Stefan From vitja.makarov at gmail.com Sat Jan 22 10:47:53 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 22 Jan 2011 12:47:53 +0300 Subject: [Cython] Cython workshop In-Reply-To: <4D3A8FF1.3090103@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> Message-ID: 2011/1/22 Stefan Behnel : > Dag Sverre Seljebotn, 20.01.2011 12:26: >> Starting new thread in case somebody did not see the hijacking. >> >> I've created this wiki page: >> >> http://wiki.cython.org/workshop1 >> >> If you're interested in coming to a Cython workshop, please fill in your >> details in the table to help decide when and where a workshop should be >> held. > > Current status so far: 4 core developers have signed in, 5 people in total, > all from Europe. Robert said he'd like to join in, too. > > Have there been any further off-list replies so far? > > In case we actually want to advertise and run this before the end of April > (or even March, and Vitja's visa will likely take a while to get), it would > be good if the remaining core developers and other interested parties could > speak up soon, say, within a week's time, so that we can decide on date and > location. > My visa is valid until 5 apr, new one could take 2-3 weeks. -- vitja. From stefan_ml at behnel.de Sat Jan 22 13:38:21 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 22 Jan 2011 13:38:21 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> Message-ID: <4D3ACFBD.1090000@behnel.de> Vitja Makarov, 22.01.2011 10:47: > 2011/1/22 Stefan Behnel: >> In case we actually want to advertise and run this before the end of April >> (or even March, and Vitja's visa will likely take a while to get), it would >> be good if the remaining core developers and other interested parties could >> speak up soon, say, within a week's time, so that we can decide on date and >> location. > > My visa is valid until 5 apr, new one could take 2-3 weeks. Ok, just to throw in a couple of dates here that work well for me: March 2-6 or March 30-April 3rd. Is four days ok (assuming that people leave during the Sunday) or is it too short? Stefan From markflorisson88 at gmail.com Sat Jan 22 14:32:03 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Sat, 22 Jan 2011 14:32:03 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3ACFBD.1090000@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> <4D3ACFBD.1090000@behnel.de> Message-ID: On 22 January 2011 13:38, Stefan Behnel wrote: > Vitja Makarov, 22.01.2011 10:47: > > 2011/1/22 Stefan Behnel: > >> In case we actually want to advertise and run this before the end of > April > >> (or even March, and Vitja's visa will likely take a while to get), it > would > >> be good if the remaining core developers and other interested parties > could > >> speak up soon, say, within a week's time, so that we can decide on date > and > >> location. > > > > My visa is valid until 5 apr, new one could take 2-3 weeks. > > Ok, just to throw in a couple of dates here that work well for me: > > March 2-6 or March 30-April 3rd. > The former date is good for me, the second one is slightly terrible, I would be able to make it but I'd rather be studying for my exams that time. > Is four days ok (assuming that people leave during the Sunday) or is it too > short? I think it would be sufficient, short and effective. > > Stefan > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110122/81d1e3a9/attachment.htm From robertwb at math.washington.edu Sun Jan 23 09:53:41 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sun, 23 Jan 2011 00:53:41 -0800 Subject: [Cython] Cython workshop In-Reply-To: <4D3A8FF1.3090103@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> Message-ID: On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: > Dag Sverre Seljebotn, 20.01.2011 12:26: >> Starting new thread in case somebody did not see the hijacking. >> >> I've created this wiki page: >> >> http://wiki.cython.org/workshop1 >> >> If you're interested in coming to a Cython workshop, please fill in your >> details in the table to help decide when and where a workshop should be >> held. > > Current status so far: 4 core developers have signed in, 5 people in total, > all from Europe. Robert said he'd like to join in, too. > > Have there been any further off-list replies so far? I've spoken to two people, both of whom won't be able to do much the first half of this year. > In case we actually want to advertise and run this before the end of April > (or even March, and Vitja's visa will likely take a while to get), it would > be good if the remaining core developers and other interested parties could > speak up soon, say, within a week's time, so that we can decide on date and > location. The first half of March doesn't work for me, and April would be tricky, but I might be able to do March 30-April 3. More ideal would be May-June-July, but that conflicts directly with Stefans availability--are there any weekends in there that would work or are those months right out? I think 4 days is about the right amount of time. - Robert From dagss at student.matnat.uio.no Sun Jan 23 10:12:34 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Sun, 23 Jan 2011 10:12:34 +0100 Subject: [Cython] Cython workshop In-Reply-To: References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> Message-ID: <4D3BF102.5040507@student.matnat.uio.no> On 01/23/2011 09:53 AM, Robert Bradshaw wrote: > On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: > >> Dag Sverre Seljebotn, 20.01.2011 12:26: >> >>> Starting new thread in case somebody did not see the hijacking. >>> >>> I've created this wiki page: >>> >>> http://wiki.cython.org/workshop1 >>> >>> If you're interested in coming to a Cython workshop, please fill in your >>> details in the table to help decide when and where a workshop should be >>> held. >>> >> Current status so far: 4 core developers have signed in, 5 people in total, >> all from Europe. Robert said he'd like to join in, too. >> >> Have there been any further off-list replies so far? >> > I've spoken to two people, both of whom won't be able to do much the > first half of this year. > > >> In case we actually want to advertise and run this before the end of April >> (or even March, and Vitja's visa will likely take a while to get), it would >> be good if the remaining core developers and other interested parties could >> speak up soon, say, within a week's time, so that we can decide on date and >> location. >> > The first half of March doesn't work for me, and April would be > tricky, but I might be able to do March 30-April 3. More ideal would > be May-June-July, but that conflicts directly with Stefans > availability--are there any weekends in there that would work or are > those months right out? > > I think 4 days is about the right amount of time. > Within the months I listed there's good chances of things working for me; although April-June is better than July. March 30 - April 3 should work. Dag Sverre > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From stefan_ml at behnel.de Sun Jan 23 11:50:17 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 23 Jan 2011 11:50:17 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3BF102.5040507@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> <4D3BF102.5040507@student.matnat.uio.no> Message-ID: <4D3C07E9.3070505@behnel.de> Dag Sverre Seljebotn, 23.01.2011 10:12: > On 01/23/2011 09:53 AM, Robert Bradshaw wrote: >> On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: >> >>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>> >>>> Starting new thread in case somebody did not see the hijacking. >>>> >>>> I've created this wiki page: >>>> >>>> http://wiki.cython.org/workshop1 >>>> >>>> If you're interested in coming to a Cython workshop, please fill in your >>>> details in the table to help decide when and where a workshop should be >>>> held. >>>> >>> Current status so far: 4 core developers have signed in, 5 people in total, >>> all from Europe. Robert said he'd like to join in, too. >>> >>> Have there been any further off-list replies so far? >>> >> I've spoken to two people, both of whom won't be able to do much the >> first half of this year. Still, I think "close to now" is a very good time to do it, because 0.14.x is nearing stability and thus leaving the focus but needs a serious docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0 is getting in sight and worth discussing. Certainly enough to fill four days. >>> In case we actually want to advertise and run this before the end of April >>> (or even March, and Vitja's visa will likely take a while to get), it would >>> be good if the remaining core developers and other interested parties could >>> speak up soon, say, within a week's time, so that we can decide on date and >>> location. >>> >> The first half of March doesn't work for me, and April would be >> tricky, but I might be able to do March 30-April 3. More ideal would >> be May-June-July, but that conflicts directly with Stefans >> availability--are there any weekends in there that would work or are >> those months right out? Tricky for me. The first weekend in June might work, but I'd have to check that with my employer. End of June may work better. >> I think 4 days is about the right amount of time. > > Within the months I listed there's good chances of things working for > me; although April-June is better than July. March 30 - April 3 should work. Assuming there's never a date that won't be tricky for any of us, it seems that the first weekend in April could make it. Stefan From robertwb at math.washington.edu Sun Jan 23 12:09:47 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sun, 23 Jan 2011 03:09:47 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: On Tue, Jan 18, 2011 at 12:56 AM, Robert Bradshaw wrote: > A release candidate is up at > http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there > shouldn't be many issues, there's just a pile of bugfixes on top of > 0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1 Another candidate up at http://cython.org/release/Cython-0.14.1rc2.tar.gz - Robert From arfrever.fta at gmail.com Mon Jan 24 00:16:46 2011 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Mon, 24 Jan 2011 00:16:46 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: <201101240017.57043.Arfrever.FTA@gmail.com> Test failures with Python 2.7: ====================================================================== ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/Debugger/Tests/TestLibCython.py", line 177, in setUp python_version_number = [int(a.strip()) for a in python_version.strip('()').split(',')[:3]] ValueError: invalid literal for int() with base 10: 'sys.version_info(major=2' ====================================================================== ERROR: runTest (__main__.EndToEndTest) End-to-end basic_cythonize.srctree ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 797, in setUp os.path.join('tests', 'build', self.treefile), self.workdir) File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree f = open(tree_file) IOError: [Errno 2] No such file or directory: 'tests/build/basic_cythonize.srctree' ====================================================================== ERROR: runTest (__main__.EndToEndTest) End-to-end basic_distutils.srctree ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 797, in setUp os.path.join('tests', 'build', self.treefile), self.workdir) File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree f = open(tree_file) IOError: [Errno 2] No such file or directory: 'tests/build/basic_distutils.srctree' ====================================================================== ERROR: runTest (__main__.EndToEndTest) End-to-end cpp_cythonize.srctree ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 797, in setUp os.path.join('tests', 'build', self.treefile), self.workdir) File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree f = open(tree_file) IOError: [Errno 2] No such file or directory: 'tests/build/cpp_cythonize.srctree' ====================================================================== ERROR: runTest (__main__.EndToEndTest) End-to-end cythonize_options.srctree ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 797, in setUp os.path.join('tests', 'build', self.treefile), self.workdir) File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree f = open(tree_file) IOError: [Errno 2] No such file or directory: 'tests/build/cythonize_options.srctree' ====================================================================== ERROR: runTest (__main__.EndToEndTest) End-to-end inline_distutils.srctree ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 797, in setUp os.path.join('tests', 'build', self.treefile), self.workdir) File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree f = open(tree_file) IOError: [Errno 2] No such file or directory: 'tests/build/inline_distutils.srctree' ====================================================================== ERROR: test_embed (__main__.EmbedTest) ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 847, in setUp os.chdir(self.working_dir) OSError: [Errno 2] No such file or directory: 'Demos/embed' ====================================================================== sys.version_info.__str__() has changed in Python 2.7: $ python2.6 -c 'import sys; print(sys.version_info)' (2, 6, 6, 'final', 0) $ python2.7 -c 'import sys; print(sys.version_info)' sys.version_info(major=2, minor=7, micro=2, releaselevel='alpha', serial=0) The first test from the above list fails earlier with Python 3.1: ====================================================================== ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", line 171, in setUp python_version_script.write('python import sys; print sys.version_info\n') TypeError: must be bytes or buffer, not str ====================================================================== I'm attaching untested patch for the first failing test. I didn't have time to investigate other test failures. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: cython-TestLibCython.patch Type: text/x-patch Size: 1036 bytes Desc: not available Url : http://codespeak.net/pipermail/cython-dev/attachments/20110124/35d17d47/attachment.bin -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. Url : http://codespeak.net/pipermail/cython-dev/attachments/20110124/35d17d47/attachment.pgp From markflorisson88 at gmail.com Mon Jan 24 21:59:02 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Mon, 24 Jan 2011 21:59:02 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <201101240017.57043.Arfrever.FTA@gmail.com> References: <201101240017.57043.Arfrever.FTA@gmail.com> Message-ID: On 24 January 2011 00:16, Arfrever Frehtes Taifersar Arahesis < arfrever.fta at gmail.com> wrote: > Test failures with Python 2.7: > > ====================================================================== > ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/Debugger/Tests/TestLibCython.py", > line 177, in setUp > python_version_number = [int(a.strip()) for a in > python_version.strip('()').split(',')[:3]] > ValueError: invalid literal for int() with base 10: > 'sys.version_info(major=2' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end basic_cythonize.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 797, in setUp > os.path.join('tests', 'build', self.treefile), self.workdir) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", > line 176, in unpack_source_tree > f = open(tree_file) > IOError: [Errno 2] No such file or directory: > 'tests/build/basic_cythonize.srctree' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end basic_distutils.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 797, in setUp > os.path.join('tests', 'build', self.treefile), self.workdir) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", > line 176, in unpack_source_tree > f = open(tree_file) > IOError: [Errno 2] No such file or directory: > 'tests/build/basic_distutils.srctree' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end cpp_cythonize.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 797, in setUp > os.path.join('tests', 'build', self.treefile), self.workdir) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", > line 176, in unpack_source_tree > f = open(tree_file) > IOError: [Errno 2] No such file or directory: > 'tests/build/cpp_cythonize.srctree' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end cythonize_options.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 797, in setUp > os.path.join('tests', 'build', self.treefile), self.workdir) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", > line 176, in unpack_source_tree > f = open(tree_file) > IOError: [Errno 2] No such file or directory: > 'tests/build/cythonize_options.srctree' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end inline_distutils.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 797, in setUp > os.path.join('tests', 'build', self.treefile), self.workdir) > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", > line 176, in unpack_source_tree > f = open(tree_file) > IOError: [Errno 2] No such file or directory: > 'tests/build/inline_distutils.srctree' > > ====================================================================== > ERROR: test_embed (__main__.EmbedTest) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "runtests.py", line 847, in setUp > os.chdir(self.working_dir) > OSError: [Errno 2] No such file or directory: 'Demos/embed' > > ====================================================================== > > sys.version_info.__str__() has changed in Python 2.7: > $ python2.6 -c 'import sys; print(sys.version_info)' > (2, 6, 6, 'final', 0) > $ python2.7 -c 'import sys; print(sys.version_info)' > sys.version_info(major=2, minor=7, micro=2, releaselevel='alpha', serial=0) > > The first test from the above list fails earlier with Python 3.1: > > ====================================================================== > ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File > "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/BUILD/Cy3/Cython/Debugger/Tests/TestLibCython.py", > line 171, in setUp > python_version_script.write('python import sys; print > sys.version_info\n') > TypeError: must be bytes or buffer, not str > > ====================================================================== > > I'm attaching untested patch for the first failing test. I didn't have time > to investigate other test failures. > > -- > Arfrever Frehtes Taifersar Arahesis > > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > > Dear Arfrever, Thanks for your report, I committed your patch (with some modifications, namely to avoid running tests with Python 2.5 as the debuggee as the Py_TPFLAGS_XXX_SUBCLASS and such are new in 2.6) and will look at some other GDB test-related issues for Python 3. Hopefully someone else will have a look at the other test suites, or maybe I will if everyone is busy. Kind regards, Mark -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110124/cd5d707f/attachment.htm From vitja.makarov at gmail.com Tue Jan 25 08:15:28 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 25 Jan 2011 10:15:28 +0300 Subject: [Cython] unbound variables Message-ID: Hi! I want to write simple code to find unbound variables. As it's hard to detect them in common case it will mark them as: - bound (defenitley bound) def foo(): a = 1 print a Bound variables should be initialized to NULL - unbound (defenitley unbound) def foo(): print a a = 1 Unbound variables should be initialized to Py_None. And user may be warned (should depend on command line flags) - don't know (not sure here) def foo(x): if x: a = 1 [else: # optional a = 2] print a Algo is too stupid it don't know much about branches, so it's not sure here. This ones will be initialized to Py_None. Also I would like to check for unused variables and unreachable code (this should be removed). -- vitja. From stefan_ml at behnel.de Tue Jan 25 08:33:51 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 25 Jan 2011 08:33:51 +0100 Subject: [Cython] unbound variables In-Reply-To: References: Message-ID: <4D3E7CDF.6000202@behnel.de> Vitja Makarov, 25.01.2011 08:15: > I want to write simple code to find unbound variables. There is some code that does (part of) that already, in a transform IIRC. It's used to detect if we need to initialise a local variable to None or can safely set it to NULL. > As it's hard to detect them in common case it will mark them as: > - bound (defenitley bound) > def foo(): > a = 1 > print a > > Bound variables should be initialized to NULL > > - unbound (defenitley unbound) > def foo(): > print a > a = 1 > > Unbound variables should be initialized to Py_None. > And user may be warned (should depend on command line flags) > > - don't know (not sure here) > def foo(x): > if x: > a = 1 > [else: # optional > a = 2] > print a > > Algo is too stupid it don't know much about branches, so it's not sure here. Well, it *can't* know what will happen at runtime. That's not being stupid at all, it's just the correct way to do it. > This ones will be initialized to Py_None. To be correct, they'd have to get initialised to NULL and a check needs to get generated on read access as long as we don't know for sure if it has been initialised or not. CPython raises an exception on unbound locals, so Cython should do the same and should do it efficiently. When we get to the point that we safely know which variables are being initialised and for which only the runtime behaviour can tell us if they are or not, I think it's fine to add the little performance penalty of checking for NULL in exactly the unsafe cases. > Also I would like to check for unused variables and unreachable code > (this should be removed). Some unreachable code is getting dropped during constant folding, usually stuff like "if False: ...", but I agree that there's always more that can be done. Think of this: def x(): do_some_stuff() return # disable rest of code, e.g. during debugging unreachable_code() Cython shouldn't bother generating dead code here (even if the C compiler would drop it anyway). Knowing where a function returns and if it has trailing code with a default None return would also be good to know. It would be a step towards inferring the return type of functions automatically. Stefan From robertwb at math.washington.edu Tue Jan 25 10:00:32 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 25 Jan 2011 01:00:32 -0800 Subject: [Cython] unbound variables In-Reply-To: <4D3E7CDF.6000202@behnel.de> References: <4D3E7CDF.6000202@behnel.de> Message-ID: On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel wrote: > Vitja Makarov, 25.01.2011 08:15: >> I want to write simple code to find unbound variables. I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)? > There is some code that does (part of) that already, in a transform IIRC. > It's used to detect if we need to initialise a local variable to None or > can safely set it to NULL. > > >> As it's hard to detect them in common case it will mark them as: >> ? - bound (defenitley bound) >> def foo(): >> ? ? ?a = 1 >> ? ? ?print a >> >> Bound variables should be initialized to NULL >> >> ? - unbound (defenitley unbound) >> def foo(): >> ? ? ?print a >> ? ? ?a = 1 >> >> Unbound variables should be initialized to Py_None. >> And user may be warned (should depend on command line flags) >> >> ? - don't know (not sure here) >> def foo(x): >> ? ? ?if x: >> ? ? ? ? ?a = 1 >> ? ? ?[else: # optional >> ? ? ? ? ?a = 2] >> ? ? ?print a >> >> Algo is too stupid it don't know much about branches, so it's not sure here. > > Well, it *can't* know what will happen at runtime. That's not being stupid > at all, it's just the correct way to do it. If the else clause is there, it should be smart enough. Loops are a bit trickier. >> This ones will be initialized to Py_None. > > To be correct, they'd have to get initialised to NULL and a check needs to > get generated on read access as long as we don't know for sure if it has > been initialised or not. CPython raises an exception on unbound locals, so > Cython should do the same and should do it efficiently. When we get to the > point that we safely know which variables are being initialised and for > which only the runtime behaviour can tell us if they are or not, I think > it's fine to add the little performance penalty of checking for NULL in > exactly the unsafe cases. Note also that assignment, and possibly cleanup, needs to be made NULL-safe as well. >> Also I would like to check for unused variables We already do this (to avoid compiler warnings). >> and unreachable code >> (this should be removed). > > Some unreachable code is getting dropped during constant folding, usually > stuff like "if False: ...", but I agree that there's always more that can > be done. Think of this: > > ? ? def x(): > ? ? ? ? do_some_stuff() > > ? ? ? ? return # disable rest of code, e.g. during debugging > > ? ? ? ? unreachable_code() > > Cython shouldn't bother generating dead code here (even if the C compiler > would drop it anyway). Personally, I think it's better to focus on stuff that the C compiler can't do for us, but if it doesn't complicate things too much than it's nice to have. > Knowing where a function returns and if it has trailing code with a default > None return would also be good to know. It would be a step towards > inferring the return type of functions automatically. Yep, that would be nice. Currently we don't have much (anything?) in the way of control flow analysis. It probably wouldn't be too hard to walk the tree to discover this kind of information, recording on each NameNode as one goes along what its possible states are. - Robert From vitja.makarov at gmail.com Tue Jan 25 10:01:56 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 25 Jan 2011 12:01:56 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D3E7CDF.6000202@behnel.de> References: <4D3E7CDF.6000202@behnel.de> Message-ID: 2011/1/25 Stefan Behnel : > Vitja Makarov, 25.01.2011 08:15: >> I want to write simple code to find unbound variables. > > There is some code that does (part of) that already, in a transform IIRC. > It's used to detect if we need to initialise a local variable to None or > can safely set it to NULL. > Is it disabled by default? def foo(): a = "xxx" print a is transformed into: static PyObject *__pyx_pf_1f_0foo(PyObject *__pyx_self, CYTHON_UNUSED PyObject *unused) { PyObject *__pyx_v_a; PyObject *__pyx_r = NULL; __Pyx_RefNannyDeclareContext; __Pyx_RefNannySetupContext("foo"); __pyx_self = __pyx_self; __pyx_v_a = Py_None; __Pyx_INCREF(Py_None); __Pyx_INCREF(((PyObject *)__pyx_n_s__xxx)); __Pyx_DECREF(__pyx_v_a); __pyx_v_a = ((PyObject *)__pyx_n_s__xxx); ... } > >> As it's hard to detect them in common case it will mark them as: >> ? - bound (defenitley bound) >> def foo(): >> ? ? ?a = 1 >> ? ? ?print a >> >> Bound variables should be initialized to NULL >> >> ? - unbound (defenitley unbound) >> def foo(): >> ? ? ?print a >> ? ? ?a = 1 >> >> Unbound variables should be initialized to Py_None. >> And user may be warned (should depend on command line flags) >> >> ? - don't know (not sure here) >> def foo(x): >> ? ? ?if x: >> ? ? ? ? ?a = 1 >> ? ? ?[else: # optional >> ? ? ? ? ?a = 2] >> ? ? ?print a >> >> Algo is too stupid it don't know much about branches, so it's not sure here. > > Well, it *can't* know what will happen at runtime. That's not being stupid > at all, it's just the correct way to do it. > > Ok. But I guess that we can detect more cases in compile time. >> This ones will be initialized to Py_None. > > To be correct, they'd have to get initialised to NULL and a check needs to > get generated on read access as long as we don't know for sure if it has > been initialised or not. CPython raises an exception on unbound locals, so > Cython should do the same and should do it efficiently. When we get to the > point that we safely know which variables are being initialised and for > which only the runtime behaviour can tell us if they are or not, I think > it's fine to add the little performance penalty of checking for NULL in > exactly the unsafe cases. > Ok. At the point we aren't sure variable is intialized we can insert small check for NULL. Also this can help for local variable deletion support. > >> Also I would like to check for unused variables and unreachable code >> (this should be removed). > > Some unreachable code is getting dropped during constant folding, usually > stuff like "if False: ...", but I agree that there's always more that can > be done. Think of this: > > ? ? def x(): > ? ? ? ? do_some_stuff() > > ? ? ? ? return # disable rest of code, e.g. during debugging > > ? ? ? ? unreachable_code() > > Cython shouldn't bother generating dead code here (even if the C compiler > would drop it anyway). > That should be rather easy to do: remove all the nodes in StatList after: break, continue, return, raise, something else? > Knowing where a function returns and if it has trailing code with a default > None return would also be good to know. It would be a step towards > inferring the return type of functions automatically. > -- vitja. From stefan_ml at behnel.de Tue Jan 25 10:23:04 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 25 Jan 2011 10:23:04 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> Message-ID: <4D3E9678.9050108@behnel.de> Robert Bradshaw, 25.01.2011 10:00: > On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel wrote: >> Vitja Makarov, 25.01.2011 08:15: >>> I want to write simple code to find unbound variables. > > I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)? >[...] > It probably wouldn't be too hard to walk the tree to discover this > kind of information, recording on each NameNode as one goes along what > its possible states are. A general "NameNode dependency walk" of the tree could give us *loads* of information, also for better type inference. Knowing that a Python variable is definitely not None from a given point on, or that it has a specific type at one point and a different type at another would be really cool. Basically, we could build a dependency tree for each NameNode (at its specific point in the tree, not just through its symtab Entry) that references its assignment RHSs, but including a representation of relevant branches in the code. But I also agree that loops are evil. :-/ Stefan From stefan_ml at behnel.de Tue Jan 25 10:40:42 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 25 Jan 2011 10:40:42 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> Message-ID: <4D3E9A9A.1050103@behnel.de> Vitja Makarov, 25.01.2011 10:01: > 2011/1/25 Stefan Behnel: >> def x(): >> do_some_stuff() >> >> return # disable rest of code, e.g. during debugging >> >> unreachable_code() >> >> Cython shouldn't bother generating dead code here (even if the C compiler >> would drop it anyway). > > That should be rather easy to do: remove all the nodes in StatList > after: break, continue, return, raise, something else? Careful, this may involve recursive propagation through helper nodes. The tree isn't always as simple as that. I think an attribute "is_terminator" on Nodes might do the job. It's set to False by default and to True on all nodes you mentioned above, and is inherited by StatListNode if its last node is a terminator (while dropping its remaining child nodes at the same time) and by all helper nodes that contain StatListNodes. This could be done in analyse_types() (or maybe earlier?). Stefan From dagss at student.matnat.uio.no Tue Jan 25 10:42:41 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 25 Jan 2011 10:42:41 +0100 Subject: [Cython] unbound variables In-Reply-To: <4D3E9678.9050108@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9678.9050108@behnel.de> Message-ID: <4D3E9B11.6000800@student.matnat.uio.no> On 01/25/2011 10:23 AM, Stefan Behnel wrote: > Robert Bradshaw, 25.01.2011 10:00: > >> On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel wrote: >> >>> Vitja Makarov, 25.01.2011 08:15: >>> >>>> I want to write simple code to find unbound variables. >>>> >> I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)? >> [...] >> It probably wouldn't be too hard to walk the tree to discover this >> kind of information, recording on each NameNode as one goes along what >> its possible states are. >> > A general "NameNode dependency walk" of the tree could give us *loads* of > information, also for better type inference. Knowing that a Python variable > is definitely not None from a given point on, or that it has a specific > type at one point and a different type at another would be really cool. > > Basically, we could build a dependency tree for each NameNode (at its > specific point in the tree, not just through its symtab Entry) that > references its assignment RHSs, but including a representation of relevant > branches in the code. > > But I also agree that loops are evil. :-/ > A related problems is use of uninitialized local variables. I recently had problems with that, and gcc warnings are less than perfect in this area. I think we ideally should raise a compiler error if it can be proven that a variable is uninitialized. I'm even in favour of following Java and raise an error (or at least a warning) if it can't be proven that it is initialized; such code is usually poorly written anyway and it really doesn't hurt to initialize the variable to 0. Dag Sverre From robertwb at math.washington.edu Tue Jan 25 10:43:47 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 25 Jan 2011 01:43:47 -0800 Subject: [Cython] unbound variables In-Reply-To: <4D3E9678.9050108@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9678.9050108@behnel.de> Message-ID: On Tue, Jan 25, 2011 at 1:23 AM, Stefan Behnel wrote: > Robert Bradshaw, 25.01.2011 10:00: >> On Mon, Jan 24, 2011 at 11:33 PM, Stefan Behnel wrote: >>> Vitja Makarov, 25.01.2011 08:15: >>>> I want to write simple code to find unbound variables. >> >> I'm assuming you mean unassigned (as opposed to unbound in, e.g., a closure)? >>[...] >> It probably wouldn't be too hard to walk the tree to discover this >> kind of information, recording on each NameNode as one goes along what >> its possible states are. > > A general "NameNode dependency walk" of the tree could give us *loads* of > information, also for better type inference. Knowing that a Python variable > is definitely not None from a given point on, or that it has a specific > type at one point and a different type at another would be really cool. > > Basically, we could build a dependency tree for each NameNode (at its > specific point in the tree, not just through its symtab Entry) that > references its assignment RHSs, but including a representation of relevant > branches in the code. Yes, that's exactly what I was thinking. Each NameNode would have an associated status (which could contain arbitrary information and have merge methods) and we would walk the tree updating the status as we went, merging on the branches. - Robert From stefan_ml at behnel.de Tue Jan 25 16:47:18 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Tue, 25 Jan 2011 16:47:18 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3C07E9.3070505@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> <4D3BF102.5040507@student.matnat.uio.no> <4D3C07E9.3070505@behnel.de> Message-ID: <4D3EF086.8050107@behnel.de> Stefan Behnel, 23.01.2011 11:50: > Dag Sverre Seljebotn, 23.01.2011 10:12: >> On 01/23/2011 09:53 AM, Robert Bradshaw wrote: >>> On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: >>> >>>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>>> >>>>> Starting new thread in case somebody did not see the hijacking. >>>>> >>>>> I've created this wiki page: >>>>> >>>>> http://wiki.cython.org/workshop1 >>>>> >>>>> If you're interested in coming to a Cython workshop, please fill in your >>>>> details in the table to help decide when and where a workshop should be >>>>> held. >>>>> >>>> Current status so far: 4 core developers have signed in, 5 people in total, >>>> all from Europe. Robert said he'd like to join in, too. >>>> >>>> Have there been any further off-list replies so far? >>>> >>> I've spoken to two people, both of whom won't be able to do much the >>> first half of this year. > > Still, I think "close to now" is a very good time to do it, because 0.14.x > is nearing stability and thus leaving the focus but needs a serious > docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0 > is getting in sight and worth discussing. Certainly enough to fill four days. I added a couple of development topics that could benefit from a hacking workshop. Depending on how "nontrivial" the NFS funding is - maybe we could split it into two events? One more planning/core developer oriented workshop now and one geared towards code sprinting later during the year? Stefan From dagss at student.matnat.uio.no Tue Jan 25 17:55:36 2011 From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn) Date: Tue, 25 Jan 2011 17:55:36 +0100 Subject: [Cython] Cython workshop In-Reply-To: <4D3EF086.8050107@behnel.de> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> <4D3BF102.5040507@student.matnat.uio.no> <4D3C07E9.3070505@behnel.de> <4D3EF086.8050107@behnel.de> Message-ID: <4D3F0088.5060102@student.matnat.uio.no> On 01/25/2011 04:47 PM, Stefan Behnel wrote: > Stefan Behnel, 23.01.2011 11:50: > >> Dag Sverre Seljebotn, 23.01.2011 10:12: >> >>> On 01/23/2011 09:53 AM, Robert Bradshaw wrote: >>> >>>> On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: >>>> >>>> >>>>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>>>> >>>>> >>>>>> Starting new thread in case somebody did not see the hijacking. >>>>>> >>>>>> I've created this wiki page: >>>>>> >>>>>> http://wiki.cython.org/workshop1 >>>>>> >>>>>> If you're interested in coming to a Cython workshop, please fill in your >>>>>> details in the table to help decide when and where a workshop should be >>>>>> held. >>>>>> >>>>>> >>>>> Current status so far: 4 core developers have signed in, 5 people in total, >>>>> all from Europe. Robert said he'd like to join in, too. >>>>> >>>>> Have there been any further off-list replies so far? >>>>> >>>>> >>>> I've spoken to two people, both of whom won't be able to do much the >>>> first half of this year. >>>> >> Still, I think "close to now" is a very good time to do it, because 0.14.x >> is nearing stability and thus leaving the focus but needs a serious >> docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0 >> is getting in sight and worth discussing. Certainly enough to fill four days. >> > I added a couple of development topics that could benefit from a hacking > workshop. > > Depending on how "nontrivial" the NFS funding is - maybe we could split it > into two events? One more planning/core developer oriented workshop now and > one geared towards code sprinting later during the year? > Reading chapter 11 of the NSF proposal, at least there was two Cython workshops planned when applying for funding, so it seems likely. Of course, we want to make sure there's some ties to scientific computation. For this one, at least Pauli Virtanen, a very significant contributor to NumPy/SciPy, wants to come (assuming we have it in Munich). Me and him (and whoever else is interested) should be able to hash out the most important things regarding Cython+NumPy/SciPy. I think the "macro" ability (inline certain generators used in for-loop) is completely crucial here, and templates too. I don't worry too much with planning that part though; when me and Pauli (and whoever else is interested) gets together we'll figure it out. Dag Sverre From robertwb at math.washington.edu Tue Jan 25 20:36:17 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Tue, 25 Jan 2011 11:36:17 -0800 Subject: [Cython] Cython workshop In-Reply-To: <4D3F0088.5060102@student.matnat.uio.no> References: <4D381BCE.30300@student.matnat.uio.no> <4D3A8FF1.3090103@behnel.de> <4D3BF102.5040507@student.matnat.uio.no> <4D3C07E9.3070505@behnel.de> <4D3EF086.8050107@behnel.de> <4D3F0088.5060102@student.matnat.uio.no> Message-ID: On Tue, Jan 25, 2011 at 8:55 AM, Dag Sverre Seljebotn wrote: > On 01/25/2011 04:47 PM, Stefan Behnel wrote: >> Stefan Behnel, 23.01.2011 11:50: >> >>> Dag Sverre Seljebotn, 23.01.2011 10:12: >>> >>>> On 01/23/2011 09:53 AM, Robert Bradshaw wrote: >>>> >>>>> On Sat, Jan 22, 2011 at 12:06 AM, Stefan Behnel wrote: >>>>> >>>>> >>>>>> Dag Sverre Seljebotn, 20.01.2011 12:26: >>>>>> >>>>>> >>>>>>> Starting new thread in case somebody did not see the hijacking. >>>>>>> >>>>>>> I've created this wiki page: >>>>>>> >>>>>>> http://wiki.cython.org/workshop1 >>>>>>> >>>>>>> If you're interested in coming to a Cython workshop, please fill in your >>>>>>> details in the table to help decide when and where a workshop should be >>>>>>> held. >>>>>>> >>>>>>> >>>>>> Current status so far: 4 core developers have signed in, 5 people in total, >>>>>> all from Europe. Robert said he'd like to join in, too. >>>>>> >>>>>> Have there been any further off-list replies so far? >>>>>> >>>>>> >>>>> I've spoken to two people, both of whom won't be able to do much the >>>>> first half of this year. >>>>> >>> Still, I think "close to now" is a very good time to do it, because 0.14.x >>> is nearing stability and thus leaving the focus but needs a serious >>> docsprint, 0.15 is in the pipeline and worth putting some work into and 1.0 >>> is getting in sight and worth discussing. Certainly enough to fill four days. >>> >> I added a couple of development topics that could benefit from a hacking >> workshop. >> >> Depending on how "nontrivial" the NFS funding is - maybe we could split it >> into two events? One more planning/core developer oriented workshop now and >> one geared towards code sprinting later during the year? >> > > Reading chapter 11 of the NSF proposal, at least there was two Cython > workshops planned when applying for funding, so it seems likely. Of > course, we want to make sure there's some ties to scientific computation. > > For this one, at least Pauli Virtanen, a very significant contributor to > NumPy/SciPy, wants to come (assuming we have it in Munich). Me and him > (and whoever else is interested) should be able to hash out the most > important things regarding Cython+NumPy/SciPy. I think the "macro" > ability (inline certain generators used in for-loop) is completely > crucial here, and templates too. I don't worry too much with planning > that part though; when me and Pauli (and whoever else is interested) > gets together we'll figure it out. That sounds like it'd be worthwhile. I like the idea of trying to have two workshops, with the upcoming one more focused on planning, including NumPy/SciPy and post 1.0. - Robert From vitja.makarov at gmail.com Tue Jan 25 20:49:40 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Tue, 25 Jan 2011 22:49:40 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D3E9A9A.1050103@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> Message-ID: 2011/1/25 Stefan Behnel : > Vitja Makarov, 25.01.2011 10:01: >> 2011/1/25 Stefan Behnel: >>> ? ? ?def x(): >>> ? ? ? ? ?do_some_stuff() >>> >>> ? ? ? ? ?return # disable rest of code, e.g. during debugging >>> >>> ? ? ? ? ?unreachable_code() >>> >>> Cython shouldn't bother generating dead code here (even if the C compiler >>> would drop it anyway). >> >> That should be rather easy to do: remove all the nodes in StatList >> after: break, continue, return, raise, something else? > > Careful, this may involve recursive propagation through helper nodes. The > tree isn't always as simple as that. > > I think an attribute "is_terminator" on Nodes might do the job. It's set to > False by default and to True on all nodes you mentioned above, and is > inherited by StatListNode if its last node is a terminator (while dropping > its remaining child nodes at the same time) and by all helper nodes that > contain StatListNodes. This could be done in analyse_types() (or maybe > earlier?). > May be it could be placed in StatNodeList.analyse_flow_control()? I've tried this, and as side effect some error tests doesn't work as code after first return is removed: ====================================================================== FAIL: compiling (c) e_return ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 306, in runTest self.runCompileTest() File "runtests.py", line 310, in runCompileTest self.test_directory, self.expect_errors, self.annotate) File "runtests.py", line 449, in compile self.assertEquals(expected_error, None) AssertionError: u"8:17: Cannot assign type 'int *' to 'int'" != None ====================================================================== FAIL: compiling (c) e_while ---------------------------------------------------------------------- Traceback (most recent call last): File "runtests.py", line 306, in runTest self.runCompileTest() File "runtests.py", line 310, in runCompileTest self.test_directory, self.expect_errors, self.annotate) File "runtests.py", line 449, in compile self.assertEquals(expected_error, None) AssertionError: u'4:1: continue statement not inside loop' != None -- vitja. From vitja.makarov at gmail.com Tue Jan 25 22:18:05 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Wed, 26 Jan 2011 00:18:05 +0300 Subject: [Cython] debuger test failure Message-ID: Recently playing with cython I found this issue: latest cython from github, ubuntu 10.10 vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp --no-annotate libc Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5] Running tests against Cython 0.14.1rc2 test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: codefile.pyx:25:6: 'eggs' redeclared FAIL compiling (c) libc_all ... ok compiling (c) libc_errno ... ok compiling (c) libc_math ... ok compiling (c) libc_signal ... ok ====================================================================== FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/vitja/work/cython.git/Cython/Debugger/Tests/TestLibCython.py", line 219, in test_all self.assertEquals(0, self.p.wait(), errmsg) AssertionError: ****************************** v INSIDE GDB v ****************************** F...Function "__pyx_pf_8codefile_5outer_inner" not defined. .................. ====================================================================== FAIL: test_backtrace (Cython.Debugger.Tests.test_libcython_in_gdb.TestBacktrace) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", line 38, in wrapper return func(self, *args, **kwargs) File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", line 300, in test_backtrace assert re.search(r'\#0 *0x.* in main\(\)', result), result AssertionError: #0 0x00000000004172a0 in _start() #1 0x00007ffff69dac90 in __libc_start_main() #2 0x00000000004176c0 in Py_Main() #3 0x00000000004c7fe0 in PyRun_SimpleStringFlags() #4 0x00000000004c7990 in PyRun_StringFlags() #5 0x0000000000000000 in run_mod() #6 0x00000000004a6c70 in PyEval_EvalCode() #7 0x00000000004a62c0 in PyEval_EvalCodeEx() #8 0x00000000004a0040 in () at :1 #9 0x000000000049f010 in PyEval_CallObjectWithKeywords() #10 0x000000000041c990 in PyObject_Call() #11 0x000000000049eb70 in builtin___import__() #12 0x00000000004bd0f0 in PyImport_ImportModuleLevel() #13 0x00000000004bc9e0 in import_module_level() #14 0x00000000004bc480 in load_next() #15 0x00000000004bc170 in import_submodule() #16 0x00000000004bdc80 in _PyImport_LoadDynamicModule() #17 0x00007ffff63f3eb6 in codefile() at /tmp/tmpzs9Dzt/codefile.pyx:46 46 spam() #18 0x000000000041c990 in PyObject_Call() #19 0x00007ffff63f2d7c in spam() at /tmp/tmpzs9Dzt/codefile.pyx:22 22 os.path.join("foo", "bar") #20 0x000000000041c990 in PyObject_Call() #21 0x0000000000535aa0 in function_call() #22 0x00000000004a62c0 in PyEval_EvalCodeEx() #23 0x00000000004a0040 in join() at /usr/lib/python2.6/posixpath.py:59 59 def join(a, *p): ---------------------------------------------------------------------- Ran 22 tests in 8.739s FAILED (failures=1) ..... ---------------------------------------------------------------------- Ran 5 tests in 1.641s OK close failed in file object destructor: IOError: [Errno 9] \u041d\u0435\u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0439 \u0434\u0435\u0441\u043a\u0440\u0438\u043f\u0442\u043e\u0440 \u0444\u0430\u0439\u043b\u0430 ****************************** ^ INSIDE GDB ^ ****************************** ---------------------------------------------------------------------- Ran 5 tests in 14.643s FAILED (failures=1) ALL DONE -- vitja. From markflorisson88 at gmail.com Tue Jan 25 22:38:03 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 25 Jan 2011 22:38:03 +0100 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On 25 January 2011 22:18, Vitja Makarov wrote: > Recently playing with cython I found this issue: > > latest cython from github, ubuntu 10.10 > > vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp > --no-annotate libc > Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) > [GCC 4.4.5] > > Running tests against Cython 0.14.1rc2 > > test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: > codefile.pyx:25:6: 'eggs' redeclared > FAIL > compiling (c) libc_all ... ok > compiling (c) libc_errno ... ok > compiling (c) libc_math ... ok > compiling (c) libc_signal ... ok > > ====================================================================== > FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/TestLibCython.py", > line 219, in test_all > ? ?self.assertEquals(0, self.p.wait(), errmsg) > AssertionError: > ****************************** ? v INSIDE GDB v ? ****************************** > F...Function "__pyx_pf_8codefile_5outer_inner" not defined. > .................. > ====================================================================== > FAIL: test_backtrace (Cython.Debugger.Tests.test_libcython_in_gdb.TestBacktrace) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", > line 38, in wrapper > ? ?return func(self, *args, **kwargs) > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", > line 300, in test_backtrace > ? ?assert re.search(r'\#0 *0x.* in main\(\)', result), result > AssertionError: #0 ?0x00000000004172a0 in _start() > #1 ?0x00007ffff69dac90 in __libc_start_main() > #2 ?0x00000000004176c0 in Py_Main() > #3 ?0x00000000004c7fe0 in PyRun_SimpleStringFlags() > #4 ?0x00000000004c7990 in PyRun_StringFlags() > #5 ?0x0000000000000000 in run_mod() > #6 ?0x00000000004a6c70 in PyEval_EvalCode() > #7 ?0x00000000004a62c0 in PyEval_EvalCodeEx() > #8 ?0x00000000004a0040 in () at :1 > #9 ?0x000000000049f010 in PyEval_CallObjectWithKeywords() > #10 0x000000000041c990 in PyObject_Call() > #11 0x000000000049eb70 in builtin___import__() > #12 0x00000000004bd0f0 in PyImport_ImportModuleLevel() > #13 0x00000000004bc9e0 in import_module_level() > #14 0x00000000004bc480 in load_next() > #15 0x00000000004bc170 in import_submodule() > #16 0x00000000004bdc80 in _PyImport_LoadDynamicModule() > #17 0x00007ffff63f3eb6 in codefile() at /tmp/tmpzs9Dzt/codefile.pyx:46 > ? ? ? ?46 ? ?spam() > #18 0x000000000041c990 in PyObject_Call() > #19 0x00007ffff63f2d7c in spam() at /tmp/tmpzs9Dzt/codefile.pyx:22 > ? ? ? ?22 ? ? ? ?os.path.join("foo", "bar") > #20 0x000000000041c990 in PyObject_Call() > #21 0x0000000000535aa0 in function_call() > #22 0x00000000004a62c0 in PyEval_EvalCodeEx() > #23 0x00000000004a0040 in join() at /usr/lib/python2.6/posixpath.py:59 > ? ? ? ?59 ? ?def join(a, *p): > > > ---------------------------------------------------------------------- > Ran 22 tests in 8.739s > > FAILED (failures=1) > ..... > ---------------------------------------------------------------------- > Ran 5 tests in 1.641s > > OK > close failed in file object destructor: > IOError: [Errno 9] > \u041d\u0435\u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0439 > \u0434\u0435\u0441\u043a\u0440\u0438\u043f\u0442\u043e\u0440 > \u0444\u0430\u0439\u043b\u0430 > ****************************** ? ^ INSIDE GDB ^ ? ****************************** > > ---------------------------------------------------------------------- > Ran 5 tests in 14.643s > > FAILED (failures=1) > ALL DONE > Interesting, you don't have a main function in the backtrace. I'll fix it, thanks. To be curious, what OS (and distro if applicable) are you on? > -- > vitja. > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > From markflorisson88 at gmail.com Tue Jan 25 22:44:36 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Tue, 25 Jan 2011 22:44:36 +0100 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On 25 January 2011 22:18, Vitja Makarov wrote: > Recently playing with cython I found this issue: > > latest cython from github, ubuntu 10.10 > > vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp > --no-annotate libc > Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) > [GCC 4.4.5] > > Running tests against Cython 0.14.1rc2 > > test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: > codefile.pyx:25:6: 'eggs' redeclared > FAIL > compiling (c) libc_all ... ok > compiling (c) libc_errno ... ok > compiling (c) libc_math ... ok > compiling (c) libc_signal ... ok > > ====================================================================== > FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/TestLibCython.py", > line 219, in test_all > ? ?self.assertEquals(0, self.p.wait(), errmsg) > AssertionError: > ****************************** ? v INSIDE GDB v ? ****************************** > F...Function "__pyx_pf_8codefile_5outer_inner" not defined. > .................. > ====================================================================== > FAIL: test_backtrace (Cython.Debugger.Tests.test_libcython_in_gdb.TestBacktrace) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", > line 38, in wrapper > ? ?return func(self, *args, **kwargs) > ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", > line 300, in test_backtrace > ? ?assert re.search(r'\#0 *0x.* in main\(\)', result), result > AssertionError: #0 ?0x00000000004172a0 in _start() > #1 ?0x00007ffff69dac90 in __libc_start_main() > #2 ?0x00000000004176c0 in Py_Main() > #3 ?0x00000000004c7fe0 in PyRun_SimpleStringFlags() > #4 ?0x00000000004c7990 in PyRun_StringFlags() > #5 ?0x0000000000000000 in run_mod() > #6 ?0x00000000004a6c70 in PyEval_EvalCode() > #7 ?0x00000000004a62c0 in PyEval_EvalCodeEx() > #8 ?0x00000000004a0040 in () at :1 > #9 ?0x000000000049f010 in PyEval_CallObjectWithKeywords() > #10 0x000000000041c990 in PyObject_Call() > #11 0x000000000049eb70 in builtin___import__() > #12 0x00000000004bd0f0 in PyImport_ImportModuleLevel() > #13 0x00000000004bc9e0 in import_module_level() > #14 0x00000000004bc480 in load_next() > #15 0x00000000004bc170 in import_submodule() > #16 0x00000000004bdc80 in _PyImport_LoadDynamicModule() > #17 0x00007ffff63f3eb6 in codefile() at /tmp/tmpzs9Dzt/codefile.pyx:46 > ? ? ? ?46 ? ?spam() > #18 0x000000000041c990 in PyObject_Call() > #19 0x00007ffff63f2d7c in spam() at /tmp/tmpzs9Dzt/codefile.pyx:22 > ? ? ? ?22 ? ? ? ?os.path.join("foo", "bar") > #20 0x000000000041c990 in PyObject_Call() > #21 0x0000000000535aa0 in function_call() > #22 0x00000000004a62c0 in PyEval_EvalCodeEx() > #23 0x00000000004a0040 in join() at /usr/lib/python2.6/posixpath.py:59 > ? ? ? ?59 ? ?def join(a, *p): > > > ---------------------------------------------------------------------- > Ran 22 tests in 8.739s > > FAILED (failures=1) > ..... > ---------------------------------------------------------------------- > Ran 5 tests in 1.641s > > OK > close failed in file object destructor: > IOError: [Errno 9] > \u041d\u0435\u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0439 > \u0434\u0435\u0441\u043a\u0440\u0438\u043f\u0442\u043e\u0440 > \u0444\u0430\u0439\u043b\u0430 > ****************************** ? ^ INSIDE GDB ^ ? ****************************** > > ---------------------------------------------------------------------- > Ran 5 tests in 14.643s > > FAILED (failures=1) > ALL DONE > > > -- > vitja. > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > I pushed a fix, could you retry? From vitja.makarov at gmail.com Tue Jan 25 22:46:36 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Wed, 26 Jan 2011 00:46:36 +0300 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: 2011/1/26 mark florisson : > On 25 January 2011 22:18, Vitja Makarov wrote: >> Recently playing with cython I found this issue: >> >> latest cython from github, ubuntu 10.10 >> >> vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp >> --no-annotate libc >> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >> [GCC 4.4.5] >> >> Running tests against Cython 0.14.1rc2 >> >> test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: >> codefile.pyx:25:6: 'eggs' redeclared >> FAIL >> compiling (c) libc_all ... ok >> compiling (c) libc_errno ... ok >> compiling (c) libc_math ... ok >> compiling (c) libc_signal ... ok >> >> ====================================================================== >> FAIL: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/TestLibCython.py", >> line 219, in test_all >> ? ?self.assertEquals(0, self.p.wait(), errmsg) >> AssertionError: >> ****************************** ? v INSIDE GDB v ? ****************************** >> F...Function "__pyx_pf_8codefile_5outer_inner" not defined. >> .................. >> ====================================================================== >> FAIL: test_backtrace (Cython.Debugger.Tests.test_libcython_in_gdb.TestBacktrace) >> ---------------------------------------------------------------------- >> Traceback (most recent call last): >> ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", >> line 38, in wrapper >> ? ?return func(self, *args, **kwargs) >> ?File "/home/vitja/work/cython.git/Cython/Debugger/Tests/test_libcython_in_gdb.py", >> line 300, in test_backtrace >> ? ?assert re.search(r'\#0 *0x.* in main\(\)', result), result >> AssertionError: #0 ?0x00000000004172a0 in _start() >> #1 ?0x00007ffff69dac90 in __libc_start_main() >> #2 ?0x00000000004176c0 in Py_Main() >> #3 ?0x00000000004c7fe0 in PyRun_SimpleStringFlags() >> #4 ?0x00000000004c7990 in PyRun_StringFlags() >> #5 ?0x0000000000000000 in run_mod() >> #6 ?0x00000000004a6c70 in PyEval_EvalCode() >> #7 ?0x00000000004a62c0 in PyEval_EvalCodeEx() >> #8 ?0x00000000004a0040 in () at :1 >> #9 ?0x000000000049f010 in PyEval_CallObjectWithKeywords() >> #10 0x000000000041c990 in PyObject_Call() >> #11 0x000000000049eb70 in builtin___import__() >> #12 0x00000000004bd0f0 in PyImport_ImportModuleLevel() >> #13 0x00000000004bc9e0 in import_module_level() >> #14 0x00000000004bc480 in load_next() >> #15 0x00000000004bc170 in import_submodule() >> #16 0x00000000004bdc80 in _PyImport_LoadDynamicModule() >> #17 0x00007ffff63f3eb6 in codefile() at /tmp/tmpzs9Dzt/codefile.pyx:46 >> ? ? ? ?46 ? ?spam() >> #18 0x000000000041c990 in PyObject_Call() >> #19 0x00007ffff63f2d7c in spam() at /tmp/tmpzs9Dzt/codefile.pyx:22 >> ? ? ? ?22 ? ? ? ?os.path.join("foo", "bar") >> #20 0x000000000041c990 in PyObject_Call() >> #21 0x0000000000535aa0 in function_call() >> #22 0x00000000004a62c0 in PyEval_EvalCodeEx() >> #23 0x00000000004a0040 in join() at /usr/lib/python2.6/posixpath.py:59 >> ? ? ? ?59 ? ?def join(a, *p): >> >> >> ---------------------------------------------------------------------- >> Ran 22 tests in 8.739s >> >> FAILED (failures=1) >> ..... >> ---------------------------------------------------------------------- >> Ran 5 tests in 1.641s >> >> OK >> close failed in file object destructor: >> IOError: [Errno 9] >> \u041d\u0435\u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u044b\u0439 >> \u0434\u0435\u0441\u043a\u0440\u0438\u043f\u0442\u043e\u0440 >> \u0444\u0430\u0439\u043b\u0430 >> ****************************** ? ^ INSIDE GDB ^ ? ****************************** >> >> ---------------------------------------------------------------------- >> Ran 5 tests in 14.643s >> >> FAILED (failures=1) >> ALL DONE >> >> >> -- >> vitja. >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > I pushed a fix, could you retry? > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > It's okay now. But with few warnings. vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp --no-annotate libc Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) [GCC 4.4.5] Running tests against Cython 0.14.1rc2 test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: codefile.pyx:25:6: 'eggs' redeclared ....Function "__pyx_pf_8codefile_5outer_inner" not defined. .................. ---------------------------------------------------------------------- Ran 22 tests in 8.263s OK ..... ---------------------------------------------------------------------- Ran 5 tests in 1.606s OK ok compiling (c) libc_all ... ok compiling (c) libc_errno ... ok compiling (c) libc_math ... ok compiling (c) libc_signal ... ok ---------------------------------------------------------------------- Ran 5 tests in 13.274s OK ALL DONE -- vitja. From almar.klein at gmail.com Wed Jan 26 16:18:50 2011 From: almar.klein at gmail.com (Almar Klein) Date: Wed, 26 Jan 2011 16:18:50 +0100 Subject: [Cython] Bug report: includ_dirs not handled correctly Message-ID: Hi all, First off, let me say how happy I am with Cython. It enables me to design (parts of) numpy-based algoritms that run at lightning speed! I keep being amazed at how easy it is to create code with Cython. Keep up the good work! However, I just updated to version 1.4 and got a bug. I compile my code by invoking the compiler from within Python (code added below). I use the include_dirs option to include the Numpy header files, and get this error message: error: Command "gcc -mno-cygwin -O2 -Wall -Wstrict-prototypes -I. -I"C:\Program Files (x86)\python26\lib\site-packages\numpy\core\include" -I"C:\Program Files (x86)\python26\include" -I"C:\Program Files (x86)\python26\PC" -c convolution1D_.c -o build\temp.win32-2.6\Release\convolution1d_.o -O" failed with exit status 1 Clearly, there should be a space between the "-I" and the "C:\Pro...". Thanks, Almar PS: When responding to this e-mail, please add me to the CC, since I'm not a member of this mailing list. ================== def compile_cython(fname, callerScript=None): """ compile_cython(fname, callerScript=None) Compiles the Cython file given by fname. """ # Set callerScript if None if callerScript is None: callerScript = fname # Try importing Cython, if not available, return gracefully try: from Cython.Distutils import build_ext except ImportError: print "Require Cython to compile .pyx files (www.cython.org)." return # Store interpreter state old_argv = sys.argv old_dir = os.getcwd() # Prepare state for distutils sys.argv = [callerScript] sys.argv.append('build_ext') sys.argv.append('--inplace') if sys.platform.startswith('win'): sys.argv.append('-cmingw32') # os.chdir(os.path.dirname(callerScript)) # Get modulename modNamePlus = os.path.split(fname)[1] modName = os.path.splitext(modNamePlus)[0] # Try compiling try: # Imports from distutils.core import setup from distutils.extension import Extension from numpy.distutils.misc_util import get_numpy_include_dirs # Init include dirs include_dirs = ['.'] # Get numpy headers include_dirs.extend(get_numpy_include_dirs()) # Create extension module object ext_modules = [ Extension(modName, [modNamePlus], include_dirs=include_dirs, extra_compile_args=['-O']), ] # Compile setup( cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules, ) except BaseException, why: # also catch system exit print why raise RuntimeError('SOMETHING WENT WRONG') else: print 'Successfully compiled cython file: ', modNamePlus # Put back the state sys.argv = old_argv os.chdir(old_dir) From almar.klein at gmail.com Wed Jan 26 16:52:15 2011 From: almar.klein at gmail.com (Almar Klein) Date: Wed, 26 Jan 2011 16:52:15 +0100 Subject: [Cython] Bug report: includ_dirs not handled correctly In-Reply-To: References: Message-ID: > Clearly, there should be a space between the "-I" and the "C:\Pro...". Mmm, I jumped too conclusions here, because the style seemed weird to me. While trying older versions, including the version I had before this (0.12.1), I realized that this is how it always happened. I will dig into this and see if I can find the source of my problem. Sorry for wasting your time :/ Almar From stefan_ml at behnel.de Wed Jan 26 18:13:37 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Wed, 26 Jan 2011 18:13:37 +0100 Subject: [Cython] Bug report: includ_dirs not handled correctly In-Reply-To: References: Message-ID: <4D405641.8050109@behnel.de> Almar Klein, 26.01.2011 16:52: >> Clearly, there should be a space between the "-I" and the "C:\Pro...". > > Mmm, I jumped too conclusions here, because the style seemed weird to > me. While trying older versions, including the version I had before > this (0.12.1), I realized that this is how it always happened. > > I will dig into this and see if I can find the source of my problem. There should be some more interesting compiler error output somewhere above the failure line that you posted. Stefan From robertwb at math.washington.edu Thu Jan 27 07:14:09 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 26 Jan 2011 22:14:09 -0800 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> Message-ID: On Tue, Jan 25, 2011 at 11:49 AM, Vitja Makarov wrote: > 2011/1/25 Stefan Behnel : >> Vitja Makarov, 25.01.2011 10:01: >>> 2011/1/25 Stefan Behnel: >>>> ? ? ?def x(): >>>> ? ? ? ? ?do_some_stuff() >>>> >>>> ? ? ? ? ?return # disable rest of code, e.g. during debugging >>>> >>>> ? ? ? ? ?unreachable_code() >>>> >>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>> would drop it anyway). >>> >>> That should be rather easy to do: remove all the nodes in StatList >>> after: break, continue, return, raise, something else? >> >> Careful, this may involve recursive propagation through helper nodes. The >> tree isn't always as simple as that. >> >> I think an attribute "is_terminator" on Nodes might do the job. It's set to >> False by default and to True on all nodes you mentioned above, and is >> inherited by StatListNode if its last node is a terminator (while dropping >> its remaining child nodes at the same time) and by all helper nodes that >> contain StatListNodes. This could be done in analyse_types() (or maybe >> earlier?). >> > > May be it could be placed in StatNodeList.analyse_flow_control()? This is crufty code that should probably be removed, I wouldn't count on using it too much. - Robert From stefan_ml at behnel.de Thu Jan 27 07:29:31 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 07:29:31 +0100 Subject: [Cython] Sage problem after fixing #654 Message-ID: <4D4110CB.4050507@behnel.de> Hi, ticket #654 describes a problem with the order in which function call arguments are evaluated. I fixed it and it broke Sage. The problem was that in cases where some arguments are simple and others are not, the non-simple arguments are stuffed into a temp before hand, thus being evaluated before the other arguments. This can break side-effects of simple arguments, which include C function calls. My fix was to put all arguments into temps if any of them are in temps anyway. However, there is this code in Sage (wrapper_rr.pyx/.pxd): cdef class Wrapper_rr(Wrapper): cdef int _n_args cdef mpfr_t* _args ... # offending line: mpfr_init2(self._args[i], self.domain.prec()) The signature of mpfr_init2() is ctypedef __mpfr_struct* mpfr_t void mpfr_init2 (mpfr_t x, mp_prec_t prec) Cython generates this code: mpfr_t __pyx_t_7; ... __pyx_t_7 = (((struct ...)__pyx_v_self)->_args[__pyx_v_i]); ... mpfr_init2(__pyx_t_7, __pyx_t_8); Looks reasonable at first sight. However, gcc complains about the temp assignment: sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in assignment Any idea what might be going wrong here? Stefan From stefan_ml at behnel.de Thu Jan 27 07:35:15 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 07:35:15 +0100 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D4110CB.4050507@behnel.de> References: <4D4110CB.4050507@behnel.de> Message-ID: <4D411223.2010701@behnel.de> Stefan Behnel, 27.01.2011 07:29: > ticket #654 describes a problem with the order in which function call > arguments are evaluated. I fixed it and it broke Sage. > > The problem was that in cases where some arguments are simple and others > are not, the non-simple arguments are stuffed into a temp before hand, thus > being evaluated before the other arguments. This can break side-effects of > simple arguments, which include C function calls. > > My fix was to put all arguments into temps if any of them are in temps > anyway. BTW, I wonder if a better fix would be to declare C function calls non-simple in general. Stefan From robertwb at math.washington.edu Thu Jan 27 07:39:07 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 26 Jan 2011 22:39:07 -0800 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D4110CB.4050507@behnel.de> References: <4D4110CB.4050507@behnel.de> Message-ID: On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: > Hi, > > ticket #654 describes a problem with the order in which function call > arguments are evaluated. I fixed it and it broke Sage. Thanks for fixing this, I never got around to it last night. I'm wondering how many other similar situations may be lurking around. Of course we could store every sub-expression in temporary variables, but that would be less than ideal. > The problem was that in cases where some arguments are simple and others > are not, the non-simple arguments are stuffed into a temp before hand, thus > being evaluated before the other arguments. This can break side-effects of > simple arguments, which include C function calls. > > My fix was to put all arguments into temps if any of them are in temps > anyway. However, there is this code in Sage (wrapper_rr.pyx/.pxd): > > ? cdef class Wrapper_rr(Wrapper): > ? ? ? cdef int _n_args > ? ? ? cdef mpfr_t* _args > ? ? ? ... > > ? # offending line: > ? mpfr_init2(self._args[i], self.domain.prec()) > > The signature of mpfr_init2() is > > ? ? ctypedef __mpfr_struct* mpfr_t > ? ? void mpfr_init2 (mpfr_t x, mp_prec_t prec) > > Cython generates this code: > > ? ? mpfr_t __pyx_t_7; > ? ? ... > ? ? __pyx_t_7 = (((struct ...)__pyx_v_self)->_args[__pyx_v_i]); > ? ? ... > ? ? mpfr_init2(__pyx_t_7, __pyx_t_8); > > Looks reasonable at first sight. However, gcc complains about the temp > assignment: > > ? sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in > assignment > > Any idea what might be going wrong here? mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have stack-allocated, pass-by-reference semantics.) We only need to pull non-simple arguments into temp variables. - Robert From robertwb at math.washington.edu Thu Jan 27 07:40:31 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Wed, 26 Jan 2011 22:40:31 -0800 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D411223.2010701@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D411223.2010701@behnel.de> Message-ID: On Wed, Jan 26, 2011 at 10:35 PM, Stefan Behnel wrote: > Stefan Behnel, 27.01.2011 07:29: >> ticket #654 describes a problem with the order in which function call >> arguments are evaluated. I fixed it and it broke Sage. >> >> The problem was that in cases where some arguments are simple and others >> are not, the non-simple arguments are stuffed into a temp before hand, thus >> being evaluated before the other arguments. This can break side-effects of >> simple arguments, which include C function calls. >> >> My fix was to put all arguments into temps if any of them are in temps >> anyway. > > BTW, I wonder if a better fix would be to declare C function calls > non-simple in general. If C function calls are simple, that's a bug. Simple expressions should be side-effect free by definition. - Robert From vitja.makarov at gmail.com Thu Jan 27 08:25:10 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 27 Jan 2011 10:25:10 +0300 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> Message-ID: 2011/1/27 Robert Bradshaw : > On Tue, Jan 25, 2011 at 11:49 AM, Vitja Makarov wrote: >> 2011/1/25 Stefan Behnel : >>> Vitja Makarov, 25.01.2011 10:01: >>>> 2011/1/25 Stefan Behnel: >>>>> ? ? ?def x(): >>>>> ? ? ? ? ?do_some_stuff() >>>>> >>>>> ? ? ? ? ?return # disable rest of code, e.g. during debugging >>>>> >>>>> ? ? ? ? ?unreachable_code() >>>>> >>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>> would drop it anyway). >>>> >>>> That should be rather easy to do: remove all the nodes in StatList >>>> after: break, continue, return, raise, something else? >>> >>> Careful, this may involve recursive propagation through helper nodes. The >>> tree isn't always as simple as that. >>> >>> I think an attribute "is_terminator" on Nodes might do the job. It's set to >>> False by default and to True on all nodes you mentioned above, and is >>> inherited by StatListNode if its last node is a terminator (while dropping >>> its remaining child nodes at the same time) and by all helper nodes that >>> contain StatListNodes. This could be done in analyse_types() (or maybe >>> earlier?). >>> >> >> May be it could be placed in StatNodeList.analyse_flow_control()? > > This is crufty code that should probably be removed, I wouldn't count > on using it too much. Ok. I've moved it into ParseTreeTransforms and created branch: https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 Some error tests do fail because nodes are removed and code generation time error is omited. -- vitja. From stefan_ml at behnel.de Thu Jan 27 09:40:19 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 09:40:19 +0100 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: References: <4D4110CB.4050507@behnel.de> Message-ID: <4D412F73.8080206@behnel.de> Robert Bradshaw, 27.01.2011 07:39: > On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >> ticket #654 describes a problem with the order in which function call >> arguments are evaluated. I fixed it and it broke Sage. > > Thanks for fixing this, I never got around to it last night. I'm > wondering how many other similar situations may be lurking around. I could imagine that there are cases where comparisons of coerced types run into the same issue. And other kinds of expressions may also suffer from this. > Of > course we could store every sub-expression in temporary variables, but > that would be less than ideal. We will be getting close to that, though, look at this: ----------------- cdef object X = 1 cdef redefine_global(): global X x,X = X,2 return x cdef call3(object x1, int o, object x2): return (x1, o, x2) def test_global_redefine(): """ >>> test_global_redefine() (1, 1, 2) """ return call3(X, redefine_global(), X) ----------------- The same applies to any non-local or closure variable. NameNode currently just says this: def is_simple(self): # If it's not a C variable, it'll be in a temp. return 1 True when you interpret "simple" as "side-effect free", but false if you take it to mean "has no external dependencies". Maybe we need something like "coerce_to_local()" or "coerce_to_owned()"? >> The problem was that in cases where some arguments are simple and others >> are not, the non-simple arguments are stuffed into a temp before hand, thus >> being evaluated before the other arguments. This can break side-effects of >> simple arguments, which include C function calls. >> >> My fix was to put all arguments into temps if any of them are in temps >> anyway. However, there is this code in Sage (wrapper_rr.pyx/.pxd): >> >> cdef class Wrapper_rr(Wrapper): >> cdef int _n_args >> cdef mpfr_t* _args >> ... >> >> # offending line: >> mpfr_init2(self._args[i], self.domain.prec()) >> >> The signature of mpfr_init2() is >> >> ctypedef __mpfr_struct* mpfr_t >> void mpfr_init2 (mpfr_t x, mp_prec_t prec) >> >> Cython generates this code: >> >> mpfr_t __pyx_t_7; >> ... >> __pyx_t_7 = (((struct ...)__pyx_v_self)->_args[__pyx_v_i]); >> ... >> mpfr_init2(__pyx_t_7, __pyx_t_8); >> >> Looks reasonable at first sight. However, gcc complains about the temp >> assignment: >> >> sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in >> assignment >> >> Any idea what might be going wrong here? > > mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have > stack-allocated, pass-by-reference semantics.) We only need to pull > non-simple arguments into temp variables. Ah, that would explain it. But this also means that there may be cases where we can't help breaking existing code with such a change. Stefan From stefan_ml at behnel.de Thu Jan 27 09:49:59 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 09:49:59 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> Message-ID: <4D4131B7.8020505@behnel.de> Vitja Makarov, 27.01.2011 08:25: >>> 2011/1/25 Stefan Behnel: >>>> Vitja Makarov, 25.01.2011 10:01: >>>>> 2011/1/25 Stefan Behnel: >>>>>> def x(): >>>>>> do_some_stuff() >>>>>> >>>>>> return # disable rest of code, e.g. during debugging >>>>>> >>>>>> unreachable_code() >>>>>> >>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>> would drop it anyway). >>>>> >>>>> That should be rather easy to do: remove all the nodes in StatList >>>>> after: break, continue, return, raise, something else? >>>> >>>> Careful, this may involve recursive propagation through helper nodes. The >>>> tree isn't always as simple as that. >>>> >>>> I think an attribute "is_terminator" on Nodes might do the job. It's set to >>>> False by default and to True on all nodes you mentioned above, and is >>>> inherited by StatListNode if its last node is a terminator (while dropping >>>> its remaining child nodes at the same time) and by all helper nodes that >>>> contain StatListNodes. This could be done in analyse_types() (or maybe >>>> earlier?). > > Ok. I've moved it into ParseTreeTransforms and created branch: > > https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 Interesting: + is_terminator = 0 + is_terminator = 1 + is_terminator = True Also, as I said, this is just the very first step, the StatListNode (and other nodes) should inherit the flag from their last child. > Some error tests do fail because nodes are removed and code generation > time error is omited. That should be fixable in most cases. We could also use a compiler option that disables dead code removal, and then use it in the error tests. Stefan From vitja.makarov at gmail.com Thu Jan 27 10:02:51 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 27 Jan 2011 12:02:51 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D4131B7.8020505@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> Message-ID: 2011/1/27 Stefan Behnel : > Vitja Makarov, 27.01.2011 08:25: >>>> 2011/1/25 Stefan Behnel: >>>>> Vitja Makarov, 25.01.2011 10:01: >>>>>> 2011/1/25 Stefan Behnel: >>>>>>> ? ? ? def x(): >>>>>>> ? ? ? ? ? do_some_stuff() >>>>>>> >>>>>>> ? ? ? ? ? return # disable rest of code, e.g. during debugging >>>>>>> >>>>>>> ? ? ? ? ? unreachable_code() >>>>>>> >>>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>>> would drop it anyway). >>>>>> >>>>>> That should be rather easy to do: remove all the nodes in StatList >>>>>> after: break, continue, return, raise, something else? >>>>> >>>>> Careful, this may involve recursive propagation through helper nodes. The >>>>> tree isn't always as simple as that. >>>>> >>>>> I think an attribute "is_terminator" on Nodes might do the job. It's set to >>>>> False by default and to True on all nodes you mentioned above, and is >>>>> inherited by StatListNode if its last node is a terminator (while dropping >>>>> its remaining child nodes at the same time) and by all helper nodes that >>>>> contain StatListNodes. This could be done in analyse_types() (or maybe >>>>> earlier?). >> >> Ok. I've moved it into ParseTreeTransforms and created branch: >> >> https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 > > Interesting: > > + ? ?is_terminator = 0 > + ? ?is_terminator = 1 > + ? ?is_terminator = True > Oops > Also, as I said, this is just the very first step, Sure here, just a try. > the StatListNode (and > other nodes) should inherit the flag from their last child. > This could be done simply in RemoveUnreachableCode node.is_terminator = True I don't actually understand where could be that used later? > >> Some error tests do fail because nodes are removed and code generation >> time error is omited. > > That should be fixable in most cases. We could also use a compiler option > that disables dead code removal, and then use it in the error tests. > Hmm, not sure here. I think it should be better to move checks outside code generation. -- vitja. From stefan_ml at behnel.de Thu Jan 27 10:24:51 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 10:24:51 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> Message-ID: <4D4139E3.5090307@behnel.de> Vitja Makarov, 27.01.2011 10:02: > 2011/1/27 Stefan Behnel: >> Vitja Makarov, 27.01.2011 08:25: >>>>> 2011/1/25 Stefan Behnel: >>>>>> Vitja Makarov, 25.01.2011 10:01: >>>>>>> 2011/1/25 Stefan Behnel: >>>>>>>> def x(): >>>>>>>> do_some_stuff() >>>>>>>> >>>>>>>> return # disable rest of code, e.g. during debugging >>>>>>>> >>>>>>>> unreachable_code() >>>>>>>> >>>>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>>>> would drop it anyway). >>>>>>> >>>>>>> That should be rather easy to do: remove all the nodes in StatList >>>>>>> after: break, continue, return, raise, something else? >>>>>> >>>>>> Careful, this may involve recursive propagation through helper nodes. The >>>>>> tree isn't always as simple as that. >>>>>> >>>>>> I think an attribute "is_terminator" on Nodes might do the job. It's set to >>>>>> False by default and to True on all nodes you mentioned above, and is >>>>>> inherited by StatListNode if its last node is a terminator (while dropping >>>>>> its remaining child nodes at the same time) and by all helper nodes that >>>>>> contain StatListNodes. This could be done in analyse_types() (or maybe >>>>>> earlier?). >>> >>> Ok. I've moved it into ParseTreeTransforms and created branch: >>> >>> https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 >> >> the StatListNode (and >> other nodes) should inherit the flag from their last child. > > This could be done simply in RemoveUnreachableCode > > node.is_terminator = True Sure. > I don't actually understand where could be that used later? Well, if you recursively propagate the flag through nodes that support it, you may end up in another StatListNode that strip its trailing dead code. Look through UtilNodes.py, there are a couple of nodes that can wrap Stat(List)Nodes. Maybe check Nodes.py also, not sure if there aren't any other nodes that can safely assume that a terminator at the end of their last child makes them a terminator as well. >>> Some error tests do fail because nodes are removed and code generation >>> time error is omited. >> >> That should be fixable in most cases. We could also use a compiler option >> that disables dead code removal, and then use it in the error tests. > > Hmm, not sure here. > I think it should be better to move checks outside code generation. Ah, ok, I misunderstood you then. Yes, errors should no longer occur at code generation time. Stefan From vitja.makarov at gmail.com Thu Jan 27 10:49:47 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 27 Jan 2011 12:49:47 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D4139E3.5090307@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> Message-ID: 2011/1/27 Stefan Behnel : > Vitja Makarov, 27.01.2011 10:02: >> 2011/1/27 Stefan Behnel: >>> Vitja Makarov, 27.01.2011 08:25: >>>>>> 2011/1/25 Stefan Behnel: >>>>>>> Vitja Makarov, 25.01.2011 10:01: >>>>>>>> 2011/1/25 Stefan Behnel: >>>>>>>>> ? ? ? ?def x(): >>>>>>>>> ? ? ? ? ? ?do_some_stuff() >>>>>>>>> >>>>>>>>> ? ? ? ? ? ?return # disable rest of code, e.g. during debugging >>>>>>>>> >>>>>>>>> ? ? ? ? ? ?unreachable_code() >>>>>>>>> >>>>>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>>>>> would drop it anyway). >>>>>>>> >>>>>>>> That should be rather easy to do: remove all the nodes in StatList >>>>>>>> after: break, continue, return, raise, something else? >>>>>>> >>>>>>> Careful, this may involve recursive propagation through helper nodes. The >>>>>>> tree isn't always as simple as that. >>>>>>> >>>>>>> I think an attribute "is_terminator" on Nodes might do the job. It's set to >>>>>>> False by default and to True on all nodes you mentioned above, and is >>>>>>> inherited by StatListNode if its last node is a terminator (while dropping >>>>>>> its remaining child nodes at the same time) and by all helper nodes that >>>>>>> contain StatListNodes. This could be done in analyse_types() (or maybe >>>>>>> earlier?). >>>> >>>> Ok. I've moved it into ParseTreeTransforms and created branch: >>>> >>>> https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 >>> >>> the StatListNode (and >>> other nodes) should inherit the flag from their last child. >> >> This could be done simply in RemoveUnreachableCode >> >> node.is_terminator = True > > Sure. > > >> I don't actually understand where could be that used later? > > Well, if you recursively propagate the flag through nodes that support it, > you may end up in another StatListNode that strip its trailing dead code. > Look through UtilNodes.py, there are a couple of nodes that can wrap > Stat(List)Nodes. Maybe check Nodes.py also, not sure if there aren't any > other nodes that can safely assume that a terminator at the end of their > last child makes them a terminator as well. > > return and loop-controls are very different here if cond: return 1 else: return 0 # dead code follows We can set is_terminator here if all the clauses are terminators for i in a: if i > 0: continue else break # dead code here for i in a: return # not dead for i in a: return else: return # dead It seems to me that each case should be handled its own way. We can't simply say: this node have child with is_terminator set so it's terminator too. Is there a way to write tests for warnings? If no think we should create one. >>>> Some error tests do fail because nodes are removed and code generation >>>> time error is omited. >>> >>> That should be fixable in most cases. We could also use a compiler option >>> that disables dead code removal, and then use it in the error tests. >> >> Hmm, not sure here. >> I think it should be better to move checks outside code generation. > > Ah, ok, I misunderstood you then. Yes, errors should no longer occur at > code generation time. > Think that's whole lot of work. When implementing generators I've moved error checking into ParseTreeTransforms and left 'yield not supported' in YieldExprNode. That could be later replaced with: raise InternalError -- vitja. From stefan_ml at behnel.de Thu Jan 27 10:50:43 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 10:50:43 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> Message-ID: <4D413FF3.4040104@behnel.de> Vitja Makarov, 27.01.2011 08:25: > Ok. I've moved it into ParseTreeTransforms and created branch: > > https://github.com/vitek/cython/commit/a8e957ec29f0448ee7c43bd3969012772d09b236 Hmm, currently, it runs after type analysis: _check_c_declarations, AnalyseExpressionsTransform(self), RemoveUnreachableCode(self), This /may/ make a difference, considering that there may be declarations in the dead code that may have an impact on the remaining code. However, I would prefer running the dead code removal as early as possible, even before analysing the declarations. If users rely on dead code making live code work, I'd consider that a bug. That would also simplify the removal operation, as the tree tends to be a lot simpler in earlier stages (e.g. before optimisations, WithTransform, etc.). Stefan From stefan_ml at behnel.de Thu Jan 27 11:02:55 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 11:02:55 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> Message-ID: <4D4142CF.5000500@behnel.de> Vitja Makarov, 27.01.2011 10:49: >>>>>>>>>> def x(): >>>>>>>>>> do_some_stuff() >>>>>>>>>> >>>>>>>>>> return # disable rest of code, e.g. during debugging >>>>>>>>>> >>>>>>>>>> unreachable_code() >>>>>>>>>> >>>>>>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>>>>>> would drop it anyway). >>>>>>>>> >>>>>>>>> That should be rather easy to do: remove all the nodes in StatList >>>>>>>>> after: break, continue, return, raise, something else? >>>>>>>> >>>>>>>> Careful, this may involve recursive propagation through helper nodes. The >>>>>>>> tree isn't always as simple as that. >>>>>>>> >>>>>>>> I think an attribute "is_terminator" on Nodes might do the job. >>>> the StatListNode (and >>>> other nodes) should inherit the flag from their last child. >> >>> I don't actually understand where could be that used later? >> >> Well, if you recursively propagate the flag through nodes that support it, >> you may end up in another StatListNode that strip its trailing dead code. >> Look through UtilNodes.py, there are a couple of nodes that can wrap >> Stat(List)Nodes. Maybe check Nodes.py also, not sure if there aren't any >> other nodes that can safely assume that a terminator at the end of their >> last child makes them a terminator as well. > > return and loop-controls are very different here > > if cond: > return 1 > else: > return 0 > # dead code follows > > We can set is_terminator here if all the clauses are terminators > > for i in a: > if i> 0: > continue > else > break > # dead code here > > for i in a: > return > # not dead > > for i in a: > return > else: > return > # dead > > It seems to me that each case should be handled its own way. > We can't simply say: this node have child with is_terminator set so > it's terminator too. That's what I meant, yes. You may also consider making is_terminator a method instead of a simple attribute. That would allow nodes to reimplement it (even recursively) and make decisions based on their context. > Is there a way to write tests for warnings? If no think we should create one. Not yet, it wasn't really a high priority back when I wrote the error test support. >>>>> Some error tests do fail because nodes are removed and code generation >>>>> time error is omited. >>>> >>>> That should be fixable in most cases. We could also use a compiler option >>>> that disables dead code removal, and then use it in the error tests. >>> >>> Hmm, not sure here. >>> I think it should be better to move checks outside code generation. >> >> Ah, ok, I misunderstood you then. Yes, errors should no longer occur at >> code generation time. > > Think that's whole lot of work. Steps in that direction have been taken several times already. Hopefully, they will eventually converge to the expected state. > When implementing generators I've moved error checking into ParseTreeTransforms > and left 'yield not supported' in YieldExprNode. That could be later > replaced with: raise InternalError Hmm? YieldExprNode is used and generates code, doesn't it? Or did you mean to raise InternalError on errors that should have been caught before? I don't think there's a need to retest inside of the node. If someone disables that transform, I'm fine with seeing Cython crash. Stefan From vitja.makarov at gmail.com Thu Jan 27 11:19:41 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Thu, 27 Jan 2011 13:19:41 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D4142CF.5000500@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> Message-ID: 2011/1/27 Stefan Behnel : > Vitja Makarov, 27.01.2011 10:49: >>>>>>>>>>> ? ? ? ? def x(): >>>>>>>>>>> ? ? ? ? ? ? do_some_stuff() >>>>>>>>>>> >>>>>>>>>>> ? ? ? ? ? ? return # disable rest of code, e.g. during debugging >>>>>>>>>>> >>>>>>>>>>> ? ? ? ? ? ? unreachable_code() >>>>>>>>>>> >>>>>>>>>>> Cython shouldn't bother generating dead code here (even if the C compiler >>>>>>>>>>> would drop it anyway). >>>>>>>>>> >>>>>>>>>> That should be rather easy to do: remove all the nodes in StatList >>>>>>>>>> after: break, continue, return, raise, something else? >>>>>>>>> >>>>>>>>> Careful, this may involve recursive propagation through helper nodes. The >>>>>>>>> tree isn't always as simple as that. >>>>>>>>> >>>>>>>>> I think an attribute "is_terminator" on Nodes might do the job. >>>>> the StatListNode (and >>>>> other nodes) should inherit the flag from their last child. >>> >>>> I don't actually understand where could be that used later? >>> >>> Well, if you recursively propagate the flag through nodes that support it, >>> you may end up in another StatListNode that strip its trailing dead code. >>> Look through UtilNodes.py, there are a couple of nodes that can wrap >>> Stat(List)Nodes. Maybe check Nodes.py also, not sure if there aren't any >>> other nodes that can safely assume that a terminator at the end of their >>> last child makes them a terminator as well. >> >> return and loop-controls are very different here >> >> if cond: >> ? ? return 1 >> else: >> ? ? return 0 >> # dead code follows >> >> We can set is_terminator here if all the clauses are terminators >> >> for i in a: >> ? ? ?if i> ?0: >> ? ? ? ? ? continue >> ? ? ?else >> ? ? ? ? ? break >> ? ? ?# dead code here >> >> for i in a: >> ? ? return >> # not dead >> >> for i in a: >> ? ? ?return >> else: >> ? ? ?return >> # dead >> >> It seems to me that each case should be handled its own way. >> We can't simply say: this node have child with is_terminator set so >> it's terminator too. > > That's what I meant, yes. You may also consider making is_terminator a > method instead of a simple attribute. That would allow nodes to reimplement > it (even recursively) and make decisions based on their context. > > >> Is there a way to write tests for warnings? If no think we should create one. > > Not yet, it wasn't really a high priority back when I wrote the error test > support. > I would try to handle this. > >>>>>> Some error tests do fail because nodes are removed and code generation >>>>>> time error is omited. >>>>> >>>>> That should be fixable in most cases. We could also use a compiler option >>>>> that disables dead code removal, and then use it in the error tests. >>>> >>>> Hmm, not sure here. >>>> I think it should be better to move checks outside code generation. >>> >>> Ah, ok, I misunderstood you then. Yes, errors should no longer occur at >>> code generation time. >> >> Think that's whole lot of work. > > Steps in that direction have been taken several times already. Hopefully, > they will eventually converge to the expected state. > > >> When implementing generators I've moved error checking into ParseTreeTransforms >> and left 'yield not supported' in YieldExprNode. That could be later >> replaced with: raise InternalError > > Hmm? YieldExprNode is used and generates code, doesn't it? Or did you mean > to raise InternalError on errors that should have been caught before? I > don't think there's a need to retest inside of the node. If someone > disables that transform, I'm fine with seeing Cython crash. > https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 .... def analyse_types(self, env): if not self.label_num: error(self.pos, "'yield' not supported here") .... This error message should be replaced with assertion on self.label_num or internal error. -- vitja. From almar.klein at gmail.com Thu Jan 27 11:20:11 2011 From: almar.klein at gmail.com (Almar Klein) Date: Thu, 27 Jan 2011 11:20:11 +0100 Subject: [Cython] Bug report: includ_dirs not handled correctly In-Reply-To: <4D405641.8050109@behnel.de> References: <4D405641.8050109@behnel.de> Message-ID: On 26 January 2011 18:13, Stefan Behnel wrote: > Almar Klein, 26.01.2011 16:52: >>> >>> Clearly, there should be a space between the "-I" and the "C:\Pro...". >> >> Mmm, I jumped too conclusions here, because the style seemed weird to >> me. While trying older versions, including the version I had before >> this (0.12.1), I realized that this is how it always happened. >> >> I will dig into this and see if I can find the source of my problem. > > There should be some more interesting compiler error output somewhere above > the failure line that you posted. Nothing that said "error". After updating mingw it did: "error: #if with no expression". Looking at the code at the reported line: "#if HAVE_HYPOT " Googling for this got me a recent thread of someone having the same problem: after clearing the build directory and the c-file everything worked as it should! Thanks for taking the time trying to help me out. Almar -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/cython-dev/attachments/20110127/23526815/attachment.htm From stefan_ml at behnel.de Thu Jan 27 11:25:07 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 11:25:07 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> Message-ID: <4D414803.6010108@behnel.de> Vitja Makarov, 27.01.2011 11:19: > https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 > > .... > def analyse_types(self, env): > if not self.label_num: > error(self.pos, "'yield' not supported here") > .... > > This error message should be replaced with assertion on self.label_num > or internal error. Yes, if handled by the transform already. Stefan From stefan_ml at behnel.de Thu Jan 27 21:19:53 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Thu, 27 Jan 2011 21:19:53 +0100 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D412F73.8080206@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> Message-ID: <4D41D369.3050601@behnel.de> Stefan Behnel, 27.01.2011 09:40: > Robert Bradshaw, 27.01.2011 07:39: >> On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >>> ticket #654 describes a problem with the order in which function call >>> arguments are evaluated. I fixed it and it broke Sage. >> >> Thanks for fixing this, I never got around to it last night. I'm >> wondering how many other similar situations may be lurking around. > > [...]this also means that there may be cases > where we can't help breaking existing code with such a change. My attempts to fix this properly seem to uncover brittle code in several other places. Given that the particular problem that #654 describes is less likely to break code than my current attempts to fix it, I would suggest releasing 0.14.1 without my latest changes, i.e. from Mark's last commit "fe17af96655e7ab0310a". Stefan From robertwb at math.washington.edu Thu Jan 27 22:05:09 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 27 Jan 2011 13:05:09 -0800 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D41D369.3050601@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> <4D41D369.3050601@behnel.de> Message-ID: On Thu, Jan 27, 2011 at 12:19 PM, Stefan Behnel wrote: > Stefan Behnel, 27.01.2011 09:40: >> Robert Bradshaw, 27.01.2011 07:39: >>> On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >>>> ticket #654 describes a problem with the order in which function call >>>> arguments are evaluated. I fixed it and it broke Sage. >>> >>> Thanks for fixing this, I never got around to it last night. I'm >>> wondering how many other similar situations may be lurking around. >> >> [...]this also means that there may be cases >> where we can't help breaking existing code with such a change. > > My attempts to fix this properly seem to uncover brittle code in several > other places. Given that the particular problem that #654 describes is less > likely to break code than my current attempts to fix it, I would suggest > releasing 0.14.1 without my latest changes, i.e. from Mark's last commit > "fe17af96655e7ab0310a". +1, I was actually thinking that the -devel branch is starting to diverge to far for this release. Lets move everything that's not happening off of the 0.14.1 milestone. I'll take a look at the couple of test failure reports later today, and push out another (last?) rc. - Robert From stefan_ml at behnel.de Fri Jan 28 07:13:37 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 28 Jan 2011 07:13:37 +0100 Subject: [Cython] Fwd: PSF Sprints - Call For Applications Message-ID: <4D425E91.5090804@behnel.de> Interesting. -------- Original-Message -------- Subject: PSF Sprints - Call For Applications Date: Thu, 27 Jan 2011 22:18:41 -0600 From: Brian Curtin To: python-list at python.org Hello Python Users! On behalf of the Python Software Foundation?s sponsored sprint group, I wanted to drop your group a quick note introducing us. If you?re already familiar with our sponsored sprints, you?ll be happy to know we made a few changes to help both sprint groups and Python even more. The PSF recently set aside funding to be distributed to groups who spend time contributing to the Python ecosystem, often in the form of development sprints. Our goal is to help you help Python, so whether it?s buying meals or renting meeting space for your all-day hackathon, we have a budget set aside to reimburse your expenses up to $300 USD (up from $250). If your goal is to make the Python world a better place, and you work on the problems facing Python today, we want to help you. We?re looking for groups of hackers that spend their time fixing and expanding the wide variety of Python interpreters, libraries, tools, and anything else affecting the community.We?re also looking for groups who want to help and get started but don?t have the resources to get together. Whether your group is separated by a train ride or lacking a shared space, we want to help you. On-boarding new contributors to open source Python projects is an especially important area that we?d like to work with.This means if you have a Python project and you want to sprint -- we want to help you.Some sprints we?ve sponsored include the porting of Genshi to Python 3, improvements to packaging (Distribute/distutils), and most recently, the PyPy winter sprint in Switzerland. If your group is interested in hosting a sprint, check out the full details of our call for applications at http://www.pythonsprints.com/cfa/ and contact us at sprints at python.org. Thanks for your time, and happy sprinting! Brian Curtin Jesse Noller http://www.pythonsprints.com/ -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: Nachrichtenteil als Anhang Url: http://codespeak.net/pipermail/cython-dev/attachments/20110128/da031d17/attachment.diff From vitja.makarov at gmail.com Fri Jan 28 07:19:29 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 28 Jan 2011 09:19:29 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D414803.6010108@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> Message-ID: 2011/1/27 Stefan Behnel : > Vitja Makarov, 27.01.2011 11:19: >> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >> >> .... >> ? ? ?def analyse_types(self, env): >> ? ? ? ? ?if not self.label_num: >> ? ? ? ? ? ? ?error(self.pos, "'yield' not supported here") >> .... >> >> This error message should be replaced with assertion on self.label_num >> or internal error. > > Yes, if handled by the transform already. > I tried to handle IfStatNode terminator here: https://github.com/vitek/cython/commits/uninitialized About tests I the easiest way is to add compiler directive -Werror And add it in cython header comment # cython: werror=True -- vitja. From stefan_ml at behnel.de Fri Jan 28 07:24:48 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 28 Jan 2011 07:24:48 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> Message-ID: <4D426130.6010702@behnel.de> Vitja Makarov, 28.01.2011 07:19: > 2011/1/27 Stefan Behnel: >> Vitja Makarov, 27.01.2011 11:19: >>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>> >>> .... >>> def analyse_types(self, env): >>> if not self.label_num: >>> error(self.pos, "'yield' not supported here") >>> .... >>> >>> This error message should be replaced with assertion on self.label_num >>> or internal error. >> >> Yes, if handled by the transform already. >> > > I tried to handle IfStatNode terminator here: > > https://github.com/vitek/cython/commits/uninitialized > > About tests I the easiest way is to add compiler directive -Werror We already have an "errors are fatal" option, but I like this one. > And add it in cython header comment > > # cython: werror=True Or rather "warnings_are_errors=True". Stefan From vitja.makarov at gmail.com Fri Jan 28 07:29:37 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 28 Jan 2011 09:29:37 +0300 Subject: [Cython] unbound variables In-Reply-To: <4D426130.6010702@behnel.de> References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: 2011/1/28 Stefan Behnel : > Vitja Makarov, 28.01.2011 07:19: >> 2011/1/27 Stefan Behnel: >>> Vitja Makarov, 27.01.2011 11:19: >>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>> >>>> .... >>>> ? ? ? def analyse_types(self, env): >>>> ? ? ? ? ? if not self.label_num: >>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>> .... >>>> >>>> This error message should be replaced with assertion on self.label_num >>>> or internal error. >>> >>> Yes, if handled by the transform already. >>> >> >> I tried to handle IfStatNode terminator here: >> >> https://github.com/vitek/cython/commits/uninitialized >> > > >> About tests I the easiest way is to add compiler directive -Werror > > We already have an "errors are fatal" option, but I like this one. > This one means treat warnings as errors. > >> And add it in cython header comment >> >> # cython: werror=True > > Or rather "warnings_are_errors=True". > Cython/Options.py: > 'warn': None, > 'warn.undeclared': False, Or better warn.errors But I'm wondering how to get access to env.directives inside Erros.warning -- vitja. From stefan_ml at behnel.de Fri Jan 28 07:30:21 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 28 Jan 2011 07:30:21 +0100 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> Message-ID: <4D42627D.9090909@behnel.de> Vitja Makarov, 28.01.2011 07:19: > 2011/1/27 Stefan Behnel: >> Vitja Makarov, 27.01.2011 11:19: >>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>> >>> .... >>> def analyse_types(self, env): >>> if not self.label_num: >>> error(self.pos, "'yield' not supported here") >>> .... >>> >>> This error message should be replaced with assertion on self.label_num >>> or internal error. >> >> Yes, if handled by the transform already. >> > > I tried to handle IfStatNode terminator here: > > https://github.com/vitek/cython/commits/uninitialized Yes, that looks good. Stefan From vitja.makarov at gmail.com Fri Jan 28 07:46:51 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 28 Jan 2011 09:46:51 +0300 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: 2011/1/28 Vitja Makarov : > 2011/1/28 Stefan Behnel : >> Vitja Makarov, 28.01.2011 07:19: >>> 2011/1/27 Stefan Behnel: >>>> Vitja Makarov, 27.01.2011 11:19: >>>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>>> >>>>> .... >>>>> ? ? ? def analyse_types(self, env): >>>>> ? ? ? ? ? if not self.label_num: >>>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>>> .... >>>>> >>>>> This error message should be replaced with assertion on self.label_num >>>>> or internal error. >>>> >>>> Yes, if handled by the transform already. >>>> >>> >>> I tried to handle IfStatNode terminator here: >>> >>> https://github.com/vitek/cython/commits/uninitialized >>> >> >> >>> About tests I the easiest way is to add compiler directive -Werror >> >> We already have an "errors are fatal" option, but I like this one. >> > > This one means treat warnings as errors. > >> >>> And add it in cython header comment >>> >>> # cython: werror=True >> >> Or rather "warnings_are_errors=True". > >> Cython/Options.py: >> ? ? 'warn': None, >> ? ? 'warn.undeclared': False, > > Or better warn.errors > > But I'm wondering how to get access to > ? env.directives > > inside Erros.warning > So maybe it would be better to have -Werror command line switch? -- vitja. From robertwb at math.washington.edu Fri Jan 28 08:23:15 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 27 Jan 2011 23:23:15 -0800 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov wrote: > 2011/1/28 Vitja Makarov : >> 2011/1/28 Stefan Behnel : >>> Vitja Makarov, 28.01.2011 07:19: >>>> 2011/1/27 Stefan Behnel: >>>>> Vitja Makarov, 27.01.2011 11:19: >>>>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>>>> >>>>>> .... >>>>>> ? ? ? def analyse_types(self, env): >>>>>> ? ? ? ? ? if not self.label_num: >>>>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>>>> .... >>>>>> >>>>>> This error message should be replaced with assertion on self.label_num >>>>>> or internal error. >>>>> >>>>> Yes, if handled by the transform already. >>>>> >>>> >>>> I tried to handle IfStatNode terminator here: >>>> >>>> https://github.com/vitek/cython/commits/uninitialized >>>> >>> >>> >>>> About tests I the easiest way is to add compiler directive -Werror >>> >>> We already have an "errors are fatal" option, but I like this one. >>> >> >> This one means treat warnings as errors. >> >>> >>>> And add it in cython header comment >>>> >>>> # cython: werror=True >>> >>> Or rather "warnings_are_errors=True". >> >>> Cython/Options.py: >>> ? ? 'warn': None, >>> ? ? 'warn.undeclared': False, >> >> Or better warn.errors >> >> But I'm wondering how to get access to >> ? env.directives >> >> inside Erros.warning >> > > So maybe it would be better to have -Werror command line switch? Directives can be enabled from the command line, so I'd rather it be a directive. +1 to warnings_are_errors over the more cryptic (for those not familiar with gcc) werror. - Robert From robertwb at math.washington.edu Fri Jan 28 08:40:12 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Thu, 27 Jan 2011 23:40:12 -0800 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov wrote: > It's okay now. But with few warnings. > > vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp > --no-annotate libc > Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) > [GCC 4.4.5] > > Running tests against Cython 0.14.1rc2 > > test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: > codefile.pyx:25:6: 'eggs' redeclared > ....Function "__pyx_pf_8codefile_5outer_inner" not defined. > .................. > ---------------------------------------------------------------------- > Ran 22 tests in 8.263s I would like to get rid of these warnings if possible... - Robert From vitja.makarov at gmail.com Fri Jan 28 09:20:42 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Fri, 28 Jan 2011 11:20:42 +0300 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: 2011/1/28 Robert Bradshaw : > On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov wrote: >> 2011/1/28 Vitja Makarov : >>> 2011/1/28 Stefan Behnel : >>>> Vitja Makarov, 28.01.2011 07:19: >>>>> 2011/1/27 Stefan Behnel: >>>>>> Vitja Makarov, 27.01.2011 11:19: >>>>>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>>>>> >>>>>>> .... >>>>>>> ? ? ? def analyse_types(self, env): >>>>>>> ? ? ? ? ? if not self.label_num: >>>>>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>>>>> .... >>>>>>> >>>>>>> This error message should be replaced with assertion on self.label_num >>>>>>> or internal error. >>>>>> >>>>>> Yes, if handled by the transform already. >>>>>> >>>>> >>>>> I tried to handle IfStatNode terminator here: >>>>> >>>>> https://github.com/vitek/cython/commits/uninitialized >>>>> >>>> >>>> >>>>> About tests I the easiest way is to add compiler directive -Werror >>>> >>>> We already have an "errors are fatal" option, but I like this one. >>>> >>> >>> This one means treat warnings as errors. >>> >>>> >>>>> And add it in cython header comment >>>>> >>>>> # cython: werror=True >>>> >>>> Or rather "warnings_are_errors=True". >>> >>>> Cython/Options.py: >>>> ? ? 'warn': None, >>>> ? ? 'warn.undeclared': False, >>> >>> Or better warn.errors >>> >>> But I'm wondering how to get access to >>> ? env.directives >>> >>> inside Erros.warning >>> >> >> So maybe it would be better to have -Werror command line switch? > > Directives can be enabled from the command line, so I'd rather it be a > directive. +1 to warnings_are_errors ?over the more cryptic (for those > not familiar with gcc) werror. > Command line options are stored in Cython.Options, Compiler directives in environment is it accessible from Cython.Errors? -- vitja. From markflorisson88 at gmail.com Fri Jan 28 11:35:07 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 28 Jan 2011 11:35:07 +0100 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On 28 January 2011 08:40, Robert Bradshaw wrote: > On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov wrote: >> It's okay now. But with few warnings. >> >> vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp >> --no-annotate libc >> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >> [GCC 4.4.5] >> >> Running tests against Cython 0.14.1rc2 >> >> test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: >> codefile.pyx:25:6: 'eggs' redeclared >> ....Function "__pyx_pf_8codefile_5outer_inner" not defined. >> .................. >> ---------------------------------------------------------------------- >> Ran 22 tests in 8.263s > > I would like to get rid of these warnings if possible... > > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > Right, I also recall a ticket for that. I'll have a look today. From robertwb at math.washington.edu Fri Jan 28 11:39:18 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 02:39:18 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <201101240017.57043.Arfrever.FTA@gmail.com> References: <201101240017.57043.Arfrever.FTA@gmail.com> Message-ID: On Sun, Jan 23, 2011 at 3:16 PM, Arfrever Frehtes Taifersar Arahesis wrote: > Test failures with Python 2.7: > > ====================================================================== > ERROR: test_all (Cython.Debugger.Tests.TestLibCython.TestAll) > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/Debugger/Tests/TestLibCython.py", line 177, in setUp > ? ?python_version_number = [int(a.strip()) for a in python_version.strip('()').split(',')[:3]] > ValueError: invalid literal for int() with base 10: 'sys.version_info(major=2' > > ====================================================================== > ERROR: runTest (__main__.EndToEndTest) > End-to-end basic_cythonize.srctree > ---------------------------------------------------------------------- > Traceback (most recent call last): > ?File "runtests.py", line 797, in setUp > ? ?os.path.join('tests', 'build', self.treefile), self.workdir) > ?File "/var/tmp/portage/dev-python/cython-0.14.1_rc2/work/Cython-0.14.1rc2/Cython/TestUtils.py", line 176, in unpack_source_tree > ? ?f = open(tree_file) > IOError: [Errno 2] No such file or directory: 'tests/build/basic_cythonize.srctree' I think all of these failed because of a unfixed chdir in the failed debugging test. Please try again with the latest rc. - Robert From robertwb at math.washington.edu Fri Jan 28 11:40:07 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 02:40:07 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: On Tue, Jan 18, 2011 at 12:56 AM, Robert Bradshaw wrote: > A release candidate is up at > http://cython.org/release/Cython-0.14.1rc1.tar.gz . Hopefully there > shouldn't be many issues, there's just a pile of bugfixes on top of > 0.14. I also started http://wiki.cython.org/ReleaseNotes-0.14.1 New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz - Robert From markflorisson88 at gmail.com Fri Jan 28 12:59:23 2011 From: markflorisson88 at gmail.com (mark florisson) Date: Fri, 28 Jan 2011 12:59:23 +0100 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On 28 January 2011 08:40, Robert Bradshaw wrote: > On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov wrote: >> It's okay now. But with few warnings. >> >> vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp >> --no-annotate libc >> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >> [GCC 4.4.5] >> >> Running tests against Cython 0.14.1rc2 >> >> test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: >> codefile.pyx:25:6: 'eggs' redeclared >> ....Function "__pyx_pf_8codefile_5outer_inner" not defined. >> .................. >> ---------------------------------------------------------------------- >> Ran 22 tests in 8.263s > > I would like to get rid of these warnings if possible... > > - Robert > _______________________________________________ > Cython-dev mailing list > Cython-dev at codespeak.net > http://codespeak.net/mailman/listinfo/cython-dev > I see you duplicated the checks for the gdb and python versions in a global function. Are you still working on removing the duplication? (I'm talking about the code in Cython/Debugger/Tests/TestLibCython:GdbDebuggerTestCase.setUp). From robertwb at math.washington.edu Fri Jan 28 19:24:15 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 10:24:15 -0800 Subject: [Cython] debuger test failure In-Reply-To: References: Message-ID: On Fri, Jan 28, 2011 at 3:59 AM, mark florisson wrote: > On 28 January 2011 08:40, Robert Bradshaw wrote: >> On Tue, Jan 25, 2011 at 1:46 PM, Vitja Makarov wrote: >>> It's okay now. But with few warnings. >>> >>> vitja at vitja-laptop:~/work/cython.git$ python runtests.py -vv --no-cpp >>> --no-annotate libc >>> Python 2.6.6 (r266:84292, Sep 15 2010, 16:22:56) >>> [GCC 4.4.5] >>> >>> Running tests against Cython 0.14.1rc2 >>> >>> test_all (Cython.Debugger.Tests.TestLibCython.TestAll) ... warning: >>> codefile.pyx:25:6: 'eggs' redeclared >>> ....Function "__pyx_pf_8codefile_5outer_inner" not defined. >>> .................. >>> ---------------------------------------------------------------------- >>> Ran 22 tests in 8.263s >> >> I would like to get rid of these warnings if possible... >> >> - Robert >> _______________________________________________ >> Cython-dev mailing list >> Cython-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/cython-dev >> > > I see you duplicated the checks for the gdb and python versions in a > global function. Are you still working on removing the duplication? > (I'm talking about the code in > Cython/Debugger/Tests/TestLibCython:GdbDebuggerTestCase.setUp). The global function caches its value, so it's only doing the actual check once, but we do want to guard both bodies (on the assumption that later tests could be written). BTW, thanks for writing these tests for the debugger. It's unfortunate that 7.2 is still so new (or maybe, that so many systems are so behind) but I know what a pain it is to write tests for something interactive like this, so kudos to you. - Robert From robertwb at math.washington.edu Fri Jan 28 19:28:50 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 10:28:50 -0800 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: On Fri, Jan 28, 2011 at 12:20 AM, Vitja Makarov wrote: > 2011/1/28 Robert Bradshaw : >> On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov wrote: >>> 2011/1/28 Vitja Makarov : >>>> 2011/1/28 Stefan Behnel : >>>>> Vitja Makarov, 28.01.2011 07:19: >>>>>> 2011/1/27 Stefan Behnel: >>>>>>> Vitja Makarov, 27.01.2011 11:19: >>>>>>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>>>>>> >>>>>>>> .... >>>>>>>> ? ? ? def analyse_types(self, env): >>>>>>>> ? ? ? ? ? if not self.label_num: >>>>>>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>>>>>> .... >>>>>>>> >>>>>>>> This error message should be replaced with assertion on self.label_num >>>>>>>> or internal error. >>>>>>> >>>>>>> Yes, if handled by the transform already. >>>>>>> >>>>>> >>>>>> I tried to handle IfStatNode terminator here: >>>>>> >>>>>> https://github.com/vitek/cython/commits/uninitialized >>>>>> >>>>> >>>>> >>>>>> About tests I the easiest way is to add compiler directive -Werror >>>>> >>>>> We already have an "errors are fatal" option, but I like this one. >>>>> >>>> >>>> This one means treat warnings as errors. >>>> >>>>> >>>>>> And add it in cython header comment >>>>>> >>>>>> # cython: werror=True >>>>> >>>>> Or rather "warnings_are_errors=True". >>>> >>>>> Cython/Options.py: >>>>> ? ? 'warn': None, >>>>> ? ? 'warn.undeclared': False, >>>> >>>> Or better warn.errors >>>> >>>> But I'm wondering how to get access to >>>> ? env.directives >>>> >>>> inside Erros.warning >>>> >>> >>> So maybe it would be better to have -Werror command line switch? >> >> Directives can be enabled from the command line, so I'd rather it be a >> directive. +1 to warnings_are_errors ?over the more cryptic (for those >> not familiar with gcc) werror. >> > > Command line options are stored in Cython.Options, > Compiler directives in environment is it accessible from Cython.Errors? That is a good point. No, Cython.Errors is "global" wheras directives (may) have smaller scope. As it's not anything that influences the behavior of the program itself, I'm less adamant that there be a way to express it in the source code file. - Robert From stefan_ml at behnel.de Fri Jan 28 22:01:02 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Fri, 28 Jan 2011 22:01:02 +0100 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D412F73.8080206@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> Message-ID: <4D432E8E.3010707@behnel.de> Stefan Behnel, 27.01.2011 09:40: > Robert Bradshaw, 27.01.2011 07:39: >> On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >>> there is this code in Sage (wrapper_rr.pyx/.pxd): >>> >>> cdef class Wrapper_rr(Wrapper): >>> cdef int _n_args >>> cdef mpfr_t* _args >>> ... >>> >>> # offending line: >>> mpfr_init2(self._args[i], self.domain.prec()) >>> >>> The signature of mpfr_init2() is >>> >>> ctypedef __mpfr_struct* mpfr_t >>> void mpfr_init2 (mpfr_t x, mp_prec_t prec) >>> [...] >>> sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in >>> assignment >>> >>> Any idea what might be going wrong here? >> >> mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have >> stack-allocated, pass-by-reference semantics.) We only need to pull >> non-simple arguments into temp variables. > > Ah, that would explain it. But this also means that there may be cases > where we can't help breaking existing code with such a change. Ok, this particular code really can't be worked around by Cython. From the POV of the compiler, the second function argument (a Python function call result) may potentially have side effects on the first one, so both must be coerced to a temp in the right order to be correct. 2732 /* "sage/ext/interpreters/wrapper_rr.pyx":56 2733 * if self._args == NULL: raise MemoryError 2734 * for i in range(count): 2735 * mpfr_init2(self._args[i], self.domain.prec()) # <<<<< 2736 * val = args['constants'] 2737 * self._n_constants = len(val) 2738 */ 2739 __pyx_t_8 = (((struct __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr *)__pyx_v_self)->_args[__pyx_v_i]); 2740 __pyx_t_2 = PyObject_GetAttr(((PyObject *)((struct __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr *)__pyx_v_self)->domain), __pyx_n_s__prec); if /*...*/ 2741 __Pyx_GOTREF(__pyx_t_2); 2742 __pyx_t_3 = PyObject_Call(__pyx_t_2, ((PyObject *)__pyx_empty_tuple), NULL); if /*...*/ 2743 __Pyx_GOTREF(__pyx_t_3); 2744 __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; 2745 __pyx_t_9 = __Pyx_PyInt_from_py_mp_prec_t(__pyx_t_3); if /*...*/ 2746 __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; 2747 mpfr_init2(__pyx_t_8, __pyx_t_9); And given that Cython cannot know that the pointer is actually not a pointer, it generates the expected code here which only the C compiler can detect as invalid. This needs fixing in Sage in one way or another. Stefan From robertwb at math.washington.edu Fri Jan 28 22:49:02 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 13:49:02 -0800 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D432E8E.3010707@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> <4D432E8E.3010707@behnel.de> Message-ID: On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote: > Stefan Behnel, 27.01.2011 09:40: >> Robert Bradshaw, 27.01.2011 07:39: >>> On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >>>> there is this code in Sage (wrapper_rr.pyx/.pxd): >>>> >>>> ? ? cdef class Wrapper_rr(Wrapper): >>>> ? ? ? ? cdef int _n_args >>>> ? ? ? ? cdef mpfr_t* _args >>>> ? ? ? ? ... >>>> >>>> ? ? # offending line: >>>> ? ? mpfr_init2(self._args[i], self.domain.prec()) >>>> >>>> The signature of mpfr_init2() is >>>> >>>> ? ? ? ctypedef __mpfr_struct* mpfr_t >>>> ? ? ? void mpfr_init2 (mpfr_t x, mp_prec_t prec) >>>> ?[...] >>>> ? ? sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in >>>> assignment >>>> >>>> Any idea what might be going wrong here? >>> >>> mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have >>> stack-allocated, pass-by-reference semantics.) We only need to pull >>> non-simple arguments into temp variables. >> >> Ah, that would explain it. But this also means that there may be cases >> where we can't help breaking existing code with such a change. > > Ok, this particular code really can't be worked around by Cython. From the > POV of the compiler, the second function argument (a Python function call > result) may potentially have side effects on the first one, so both must be > coerced to a temp in the right order to be correct. > > ?2732 ? ? /* "sage/ext/interpreters/wrapper_rr.pyx":56 > ?2733 ?* ? ? ? ? if self._args == NULL: raise MemoryError > ?2734 ?* ? ? ? ? for i in range(count): > ?2735 ?* ? ? ? ? ? ? mpfr_init2(self._args[i], self.domain.prec()) # <<<<< > ?2736 ?* ? ? ? ? val = args['constants'] > ?2737 ?* ? ? ? ? self._n_constants = len(val) > ?2738 ?*/ > > ?2739 ? ? __pyx_t_8 = (((struct > __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr > *)__pyx_v_self)->_args[__pyx_v_i]); > > ?2740 ? ? __pyx_t_2 = PyObject_GetAttr(((PyObject *)((struct > __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr > *)__pyx_v_self)->domain), __pyx_n_s__prec); if /*...*/ > > ?2741 ? ? __Pyx_GOTREF(__pyx_t_2); > ?2742 ? ? __pyx_t_3 = PyObject_Call(__pyx_t_2, ((PyObject > *)__pyx_empty_tuple), NULL); if /*...*/ > > ?2743 ? ? __Pyx_GOTREF(__pyx_t_3); > ?2744 ? ? __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; > ?2745 ? ? __pyx_t_9 = __Pyx_PyInt_from_py_mp_prec_t(__pyx_t_3); if /*...*/ > ?2746 ? ? __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; > ?2747 ? ? mpfr_init2(__pyx_t_8, __pyx_t_9); > > > And given that Cython cannot know that the pointer is actually not a > pointer, it generates the expected code here which only the C compiler can > detect as invalid. > This needs fixing in Sage in one way or another. Cython didn't used to support tyepdef'd arrays, so that's why it was declared as a pointer, but yes, that should be fixed. I'll do that in our copy, and it will go out with the 0.14.1 fixes (even though we don't need it yet). There's still the case of, say, mpfr_init2(self._args[f(i)], self.domain.prec()) where f is a cdef int -> int function. Do we handle that? - Robert From arfrever.fta at gmail.com Fri Jan 28 22:59:22 2011 From: arfrever.fta at gmail.com (Arfrever Frehtes Taifersar Arahesis) Date: Fri, 28 Jan 2011 22:59:22 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: <201101282259.55705.Arfrever.FTA@gmail.com> 2011-01-28 11:40:07 Robert Bradshaw napisa?(a): > New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz Now only Numpy-related tests fail (ticket #630), which is not a regression. -- Arfrever Frehtes Taifersar Arahesis -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: This is a digitally signed message part. Url : http://codespeak.net/pipermail/cython-dev/attachments/20110128/249bb4a5/attachment.pgp From dominic.sacre at gmx.de Sat Jan 29 00:38:44 2011 From: dominic.sacre at gmx.de (=?UTF-8?Q?Dominic_Sacr=C3=A9?=) Date: Sat, 29 Jan 2011 00:38:44 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw wrote: > New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz Is it just me, or do the release candidates break isinstance() when testing against a tuple of multiple types? When I do isinstance(x, (Foo, Bar)) a Foo object will be recognized, but a Bar object will not. The same code used to work fine in older versions, including 0.14. Dominic From robertwb at math.washington.edu Sat Jan 29 01:55:51 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Fri, 28 Jan 2011 16:55:51 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: On Fri, Jan 28, 2011 at 3:38 PM, Dominic Sacr? wrote: > On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw > wrote: >> New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz > > Is it just me, or do the release candidates break isinstance() when > testing against a tuple of multiple types? When I do isinstance(x, > (Foo, Bar)) a Foo object will be recognized, but a Bar object will > not. The same code used to work fine in older versions, including > 0.14. Thanks for the report. I did fix one isinstance bug, perhaps I introduced another one (though I thought I tested this...). I'll make sure this works before release. - Robert From stefan_ml at behnel.de Sat Jan 29 06:14:51 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 29 Jan 2011 06:14:51 +0100 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> <4D432E8E.3010707@behnel.de> Message-ID: <4D43A24B.1020901@behnel.de> Robert Bradshaw, 28.01.2011 22:49: > On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote: >> Stefan Behnel, 27.01.2011 09:40: >>> Robert Bradshaw, 27.01.2011 07:39: >>>> On Wed, Jan 26, 2011 at 10:29 PM, Stefan Behnel wrote: >>>>> there is this code in Sage (wrapper_rr.pyx/.pxd): >>>>> >>>>> cdef class Wrapper_rr(Wrapper): >>>>> cdef int _n_args >>>>> cdef mpfr_t* _args >>>>> ... >>>>> >>>>> # offending line: >>>>> mpfr_init2(self._args[i], self.domain.prec()) >>>>> >>>>> The signature of mpfr_init2() is >>>>> >>>>> ctypedef __mpfr_struct* mpfr_t >>>>> void mpfr_init2 (mpfr_t x, mp_prec_t prec) >>>>> [...] >>>>> sage/ext/interpreters/wrapper_rr.c:2737: error: incompatible types in >>>>> assignment >>>>> >>>>> Any idea what might be going wrong here? >>>> >>>> mpfr_t is actually an __mpfr_struct[1]. (It's a clever hack to have >>>> stack-allocated, pass-by-reference semantics.) We only need to pull >>>> non-simple arguments into temp variables. >>> >>> Ah, that would explain it. But this also means that there may be cases >>> where we can't help breaking existing code with such a change. >> >> Ok, this particular code really can't be worked around by Cython. From the >> POV of the compiler, the second function argument (a Python function call >> result) may potentially have side effects on the first one, so both must be >> coerced to a temp in the right order to be correct. >> >> 2732 /* "sage/ext/interpreters/wrapper_rr.pyx":56 >> 2733 * if self._args == NULL: raise MemoryError >> 2734 * for i in range(count): >> 2735 * mpfr_init2(self._args[i], self.domain.prec()) #<<<<< >> 2736 * val = args['constants'] >> 2737 * self._n_constants = len(val) >> 2738 */ >> >> 2739 __pyx_t_8 = (((struct >> __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr >> *)__pyx_v_self)->_args[__pyx_v_i]); >> >> 2740 __pyx_t_2 = PyObject_GetAttr(((PyObject *)((struct >> __pyx_obj_4sage_3ext_12interpreters_10wrapper_rr_Wrapper_rr >> *)__pyx_v_self)->domain), __pyx_n_s__prec); if /*...*/ >> >> 2741 __Pyx_GOTREF(__pyx_t_2); >> 2742 __pyx_t_3 = PyObject_Call(__pyx_t_2, ((PyObject >> *)__pyx_empty_tuple), NULL); if /*...*/ >> >> 2743 __Pyx_GOTREF(__pyx_t_3); >> 2744 __Pyx_DECREF(__pyx_t_2); __pyx_t_2 = 0; >> 2745 __pyx_t_9 = __Pyx_PyInt_from_py_mp_prec_t(__pyx_t_3); if /*...*/ >> 2746 __Pyx_DECREF(__pyx_t_3); __pyx_t_3 = 0; >> 2747 mpfr_init2(__pyx_t_8, __pyx_t_9); >> >> >> And given that Cython cannot know that the pointer is actually not a >> pointer, it generates the expected code here which only the C compiler can >> detect as invalid. >> This needs fixing in Sage in one way or another. > > Cython didn't used to support tyepdef'd arrays, so that's why it was > declared as a pointer, but yes, that should be fixed. I'll do that in > our copy, and it will go out with the 0.14.1 fixes (even though we > don't need it yet). There's still the case of, say, > > mpfr_init2(self._args[f(i)], self.domain.prec()) > > where f is a cdef int -> int function. Do we handle that? Sure. The problem isn't (any longer) that we *forget* to put things into a temp, the problem is that we try to put things into a temp now that we cannot store away. I've been dropping some cases in my latest commits where the temp copying mechanism was triggered needlessly. It's basically a matter of properly implementing is_simple() for an ExprNode in order to decide when a non-temp result is side-effect free. I'm still not sure if a coerce_to_owned() method would be worthwhile. As long as SimpleCallNode is the only place where this is actually needed, it's fine to leave the code there. Can anyone think of other places where the order of C value evaluation matters? Tuples and other container literals may qualify, but given that they use only Python values, I don't see it being a problem there. Stefan From stefan_ml at behnel.de Sat Jan 29 06:27:34 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 29 Jan 2011 06:27:34 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: <4D43A546.7040407@behnel.de> Dominic Sacr?, 29.01.2011 00:38: > On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw wrote: >> New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz > > Is it just me, or do the release candidates break isinstance() when > testing against a tuple of multiple types? When I do isinstance(x, > (Foo, Bar)) a Foo object will be recognized, but a Bar object will > not. The same code used to work fine in older versions, including > 0.14. Yes, looks like a bug: ------------------ cdef class B: pass cdef class C: pass def test_custom_tuple(obj): """ >>> test_custom_tuple(A()) True >>> test_custom_tuple(B()) True >>> test_custom_tuple(C()) False """ return isinstance(obj, (A,B)) ------------------ Fails for B(), the second type check is simply dropped. I'm looking into it. Stefan From stefan_ml at behnel.de Sat Jan 29 06:39:11 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 29 Jan 2011 06:39:11 +0100 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: References: Message-ID: <4D43A7FF.10106@behnel.de> Robert Bradshaw, 29.01.2011 01:55: > On Fri, Jan 28, 2011 at 3:38 PM, Dominic Sacr? wrote: >> On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw wrote: >>> New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz >> >> Is it just me, or do the release candidates break isinstance() when >> testing against a tuple of multiple types? When I do isinstance(x, >> (Foo, Bar)) a Foo object will be recognized, but a Bar object will >> not. The same code used to work fine in older versions, including >> 0.14. > > Thanks for the report. I did fix one isinstance bug, perhaps I > introduced another one (though I thought I tested this...). I'll make > sure this works before release. My bad. There was code in Optimize.py, line 2003: if type_check_function not in tests: tests.append(type_check_function) test_nodes.append(...) Basically, it executes each type check function only once, but regardless of the type it is testing against. Works well for builtin types, doesn't work for extension types. https://github.com/cython/cython/commit/c14533e4a00e789df8d800fa9f4cc099faabb67e Hmm, I'm not sure how to merge commits over git branches with hg-git... Stefan From vitja.makarov at gmail.com Sat Jan 29 08:35:24 2011 From: vitja.makarov at gmail.com (Vitja Makarov) Date: Sat, 29 Jan 2011 10:35:24 +0300 Subject: [Cython] unbound variables In-Reply-To: References: <4D3E7CDF.6000202@behnel.de> <4D3E9A9A.1050103@behnel.de> <4D4131B7.8020505@behnel.de> <4D4139E3.5090307@behnel.de> <4D4142CF.5000500@behnel.de> <4D414803.6010108@behnel.de> <4D426130.6010702@behnel.de> Message-ID: 2011/1/28 Robert Bradshaw : > On Fri, Jan 28, 2011 at 12:20 AM, Vitja Makarov wrote: >> 2011/1/28 Robert Bradshaw : >>> On Thu, Jan 27, 2011 at 10:46 PM, Vitja Makarov wrote: >>>> 2011/1/28 Vitja Makarov : >>>>> 2011/1/28 Stefan Behnel : >>>>>> Vitja Makarov, 28.01.2011 07:19: >>>>>>> 2011/1/27 Stefan Behnel: >>>>>>>> Vitja Makarov, 27.01.2011 11:19: >>>>>>>>> https://github.com/vitek/cython/blob/master/Cython/Compiler/ExprNodes.py#L4995 >>>>>>>>> >>>>>>>>> .... >>>>>>>>> ? ? ? def analyse_types(self, env): >>>>>>>>> ? ? ? ? ? if not self.label_num: >>>>>>>>> ? ? ? ? ? ? ? error(self.pos, "'yield' not supported here") >>>>>>>>> .... >>>>>>>>> >>>>>>>>> This error message should be replaced with assertion on self.label_num >>>>>>>>> or internal error. >>>>>>>> >>>>>>>> Yes, if handled by the transform already. >>>>>>>> >>>>>>> >>>>>>> I tried to handle IfStatNode terminator here: >>>>>>> >>>>>>> https://github.com/vitek/cython/commits/uninitialized >>>>>>> >>>>>> >>>>>> >>>>>>> About tests I the easiest way is to add compiler directive -Werror >>>>>> >>>>>> We already have an "errors are fatal" option, but I like this one. >>>>>> >>>>> >>>>> This one means treat warnings as errors. >>>>> >>>>>> >>>>>>> And add it in cython header comment >>>>>>> >>>>>>> # cython: werror=True >>>>>> >>>>>> Or rather "warnings_are_errors=True". >>>>> >>>>>> Cython/Options.py: >>>>>> ? ? 'warn': None, >>>>>> ? ? 'warn.undeclared': False, >>>>> >>>>> Or better warn.errors >>>>> >>>>> But I'm wondering how to get access to >>>>> ? env.directives >>>>> >>>>> inside Erros.warning >>>>> >>>> >>>> So maybe it would be better to have -Werror command line switch? >>> >>> Directives can be enabled from the command line, so I'd rather it be a >>> directive. +1 to warnings_are_errors ?over the more cryptic (for those >>> not familiar with gcc) werror. >>> >> >> Command line options are stored in Cython.Options, >> Compiler directives in environment is it accessible from Cython.Errors? > > That is a good point. No, Cython.Errors is "global" wheras directives > (may) have smaller scope. As it's not anything that influences the > behavior of the program itself, I'm less adamant that there be a way > to express it in the source code file. > I've added support for --warning-errors and -Werror command line flags Also test runner now supports warnings. I found that warnings() sometimes could be called with no position, so this warning will still be warnings with -Werror flag. https://github.com/vitek/cython/commits/uninitialized -- vitja. From stefan_ml at behnel.de Sat Jan 29 08:37:57 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 29 Jan 2011 08:37:57 +0100 Subject: [Cython] Switching from Py_UNICODE to Py_UCS4 Message-ID: <4D43C3D5.8070506@behnel.de> Hi, there is a recent discussion on python-dev about a new memory layout for the unicode type in CPython 3.3(?), proposed by Martin von L?wis (so it's serious ;) http://comments.gmane.org/gmane.comp.python.devel/120784 If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit unsigned int), which I had completely lost from sight. It's public and undocumented and has been there basically forever, but it's a much nicer type to support than Py_UNICODE, which changes size based on build time options. Py_UCS4 is capable of representing any Unicode code point on any platform. So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4 internally (without breaking user code which can continue to use either of the two explicitly). This means that loops over unicode objects will infer Py_UCS4 as loop variable, as would indexing. It would basically become the native C type that 1 character unicode strings would coerce to and from. Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value is too large in the given CPython runtime, as would write access to unicode objects (in case anyone really does that) outside of the platform specific Py_UNICODE value range. Writing to unicode buffers will be dangerous and tricky anyway if the above PEP gets accepted. One open question that I see is whether we should handle surrogate pairs automatically. They are basically a split of large Unicode code point values (>65535) into two code points in specific ranges that are safe to detect. So we could allow a 2 'character' surrogate pair in a unicode string to coerce to one Py_UCS4 character and coerce that back into a surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that this would only work for single characters, not for looping or indexing (without the PEP, that is). So it's somewhat inconsistent. It would work well for literals, though. Also, we'd have to support it for 'in' tests, as a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the character is in the string. Comments? Stefan From robertwb at math.washington.edu Sat Jan 29 09:52:23 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 29 Jan 2011 00:52:23 -0800 Subject: [Cython] Cython 0.14.1 release candidate In-Reply-To: <4D43A7FF.10106@behnel.de> References: <4D43A7FF.10106@behnel.de> Message-ID: On Fri, Jan 28, 2011 at 9:39 PM, Stefan Behnel wrote: > Robert Bradshaw, 29.01.2011 01:55: >> On Fri, Jan 28, 2011 at 3:38 PM, Dominic Sacr? wrote: >>> On Fri, Jan 28, 2011 at 11:40 AM, Robert Bradshaw wrote: >>>> New candidate up at http://cython.org/release/Cython-0.14.1rc3.tar.gz >>> >>> Is it just me, or do the release candidates break isinstance() when >>> testing against a tuple of multiple types? When I do isinstance(x, >>> (Foo, Bar)) a Foo object will be recognized, but a Bar object will >>> not. The same code used to work fine in older versions, including >>> 0.14. >> >> Thanks for the report. I did fix one isinstance bug, perhaps I >> introduced another one (though I thought I tested this...). I'll make >> sure this works before release. > > My bad. There was code in Optimize.py, line 2003: > > ? ? ? ? ? ? if type_check_function not in tests: > ? ? ? ? ? ? ? ? tests.append(type_check_function) > ? ? ? ? ? ? ? ? test_nodes.append(...) > > Basically, it executes each type check function only once, but regardless > of the type it is testing against. Works well for builtin types, doesn't > work for extension types. > > https://github.com/cython/cython/commit/c14533e4a00e789df8d800fa9f4cc099faabb67e > > Hmm, I'm not sure how to merge commits over git branches with hg-git... I've cherry-picked it in. - Robert From robertwb at math.washington.edu Sat Jan 29 10:01:46 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 29 Jan 2011 01:01:46 -0800 Subject: [Cython] Switching from Py_UNICODE to Py_UCS4 In-Reply-To: <4D43C3D5.8070506@behnel.de> References: <4D43C3D5.8070506@behnel.de> Message-ID: On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel wrote: > Hi, > > there is a recent discussion on python-dev about a new memory layout for > the unicode type in CPython 3.3(?), proposed by Martin von L?wis (so it's > serious ;) > > http://comments.gmane.org/gmane.comp.python.devel/120784 That's an interesting PEP, I like it. > If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit > unsigned int), which I had completely lost from sight. It's public and > undocumented and has been there basically forever, but it's a much nicer > type to support than Py_UNICODE, which changes size based on build time > options. Py_UCS4 is capable of representing any Unicode code point on any > platform. > > So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4 > internally (without breaking user code which can continue to use either of > the two explicitly). This means that loops over unicode objects will infer > Py_UCS4 as loop variable, as would indexing. It would basically become the > native C type that 1 character unicode strings would coerce to and from. > Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value > is too large in the given CPython runtime, as would write access to unicode > objects (in case anyone really does that) outside of the platform specific > Py_UNICODE value range. Writing to unicode buffers will be dangerous and > tricky anyway if the above PEP gets accepted. I am a bit concerned about the performance overhead of the Py_UCS4 to Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but maybe that's both uncommon and negligible. > One open question that I see is whether we should handle surrogate pairs > automatically. They are basically a split of large Unicode code point > values (>65535) into two code points in specific ranges that are safe to > detect. So we could allow a 2 'character' surrogate pair in a unicode > string to coerce to one Py_UCS4 character and coerce that back into a > surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that > this would only work for single characters, not for looping or indexing > (without the PEP, that is). So it's somewhat inconsistent. It would work > well for literals, though. Also, we'd have to support it for 'in' tests, as > a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the > character is in the string. > > Comments? No, I don't think we should handle surrogate pairs automatically, at least without making it optional--this could be a significant performance impact with little benefit for most users. Using these higher characters is rare, but using them on a non USS4 build is probably even rarer. Also, this would be inconsistant with python-level slicing, indexing, and range, right? - Robert From robertwb at math.washington.edu Sat Jan 29 11:24:52 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sat, 29 Jan 2011 02:24:52 -0800 Subject: [Cython] Sage problem after fixing #654 In-Reply-To: <4D43A24B.1020901@behnel.de> References: <4D4110CB.4050507@behnel.de> <4D412F73.8080206@behnel.de> <4D432E8E.3010707@behnel.de> <4D43A24B.1020901@behnel.de> Message-ID: On Fri, Jan 28, 2011 at 9:14 PM, Stefan Behnel wrote: > Robert Bradshaw, 28.01.2011 22:49: >> On Fri, Jan 28, 2011 at 1:01 PM, Stefan Behnel wrote: >>> And given that Cython cannot know that the pointer is actually not a >>> pointer, it generates the expected code here which only the C compiler can >>> detect as invalid. >>> This needs fixing in Sage in one way or another. >> >> Cython didn't used to support tyepdef'd arrays, so that's why it was >> declared as a pointer, but yes, that should be fixed. I'll do that in >> our copy, and it will go out with the 0.14.1 fixes (even though we >> don't need it yet). There's still the case of, say, >> >> ? ? ?mpfr_init2(self._args[f(i)], self.domain.prec()) >> >> where f is a cdef int -> ?int function. Do we handle that? > > Sure. The problem isn't (any longer) that we *forget* to put things into a > temp, the problem is that we try to put things into a temp now that we > cannot store away. > > I've been dropping some cases in my latest commits where the temp copying > mechanism was triggered needlessly. It's basically a matter of properly > implementing is_simple() for an ExprNode in order to decide when a non-temp > result is side-effect free. > > I'm still not sure if a coerce_to_owned() method would be worthwhile. As > long as SimpleCallNode is the only place where this is actually needed, > it's fine to leave the code there. Can anyone think of other places where > the order of C value evaluation matters? Tuples and other container > literals may qualify, but given that they use only Python values, I don't > see it being a problem there. I'm not sure we handle the optimization of, e.g, "a in (x, y, z)" There might be others as well. Fixing Sage is turning out to be non-trivial...there are many places that assume mpz_t is (compatible with) a void*. We also have the problem that Cython doesn't let you take the address of an array, as it is a non-lvalue, but that's legal in C (and used in Sage). - Robert From stefan_ml at behnel.de Sat Jan 29 11:35:35 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sat, 29 Jan 2011 11:35:35 +0100 Subject: [Cython] Switching from Py_UNICODE to Py_UCS4 In-Reply-To: References: <4D43C3D5.8070506@behnel.de> Message-ID: <4D43ED77.1070201@behnel.de> Robert Bradshaw, 29.01.2011 10:01: > On Fri, Jan 28, 2011 at 11:37 PM, Stefan Behnel wrote: >> there is a recent discussion on python-dev about a new memory layout for >> the unicode type in CPython 3.3(?), proposed by Martin von L?wis (so it's >> serious ;) >> >> http://comments.gmane.org/gmane.comp.python.devel/120784 > > That's an interesting PEP, I like it. Yep, after some discussion, I started liking it too. Even if it means I'll have to touch a lot of code in Cython again. ;) >> If nothing else, it gave me a new view on Py_UCS4 (basically a 32bit >> unsigned int), which I had completely lost from sight. It's public and >> undocumented and has been there basically forever, but it's a much nicer >> type to support than Py_UNICODE, which changes size based on build time >> options. Py_UCS4 is capable of representing any Unicode code point on any >> platform. >> >> So, I'm proposing to switch from the current Py_UNICODE support to Py_UCS4 >> internally (without breaking user code which can continue to use either of >> the two explicitly). This means that loops over unicode objects will infer >> Py_UCS4 as loop variable, as would indexing. It would basically become the >> native C type that 1 character unicode strings would coerce to and from. >> Coercion from Py_UCS4 to Py_UNICODE would raise an exception if the value >> is too large in the given CPython runtime, as would write access to unicode >> objects (in case anyone really does that) outside of the platform specific >> Py_UNICODE value range. Writing to unicode buffers will be dangerous and >> tricky anyway if the above PEP gets accepted. > > I am a bit concerned about the performance overhead of the Py_UCS4 to > Py_UNICODE coercion (e.g. if constructing a Py_UNICODE* by hand), but > maybe that's both uncommon and negligible. I think so. If users deal with Py_UNICODE explicitly, they'll likely type their respective variables anyway, so that there won't be an intermediate step through Py_UCS4. And on 32bit Unicode builds this isn't an issue at all. >> One open question that I see is whether we should handle surrogate pairs >> automatically. They are basically a split of large Unicode code point >> values (>65535) into two code points in specific ranges that are safe to >> detect. So we could allow a 2 'character' surrogate pair in a unicode >> string to coerce to one Py_UCS4 character and coerce that back into a >> surrogate pair string if the runtime uses 16 bit for Py_UNICODE. Note that >> this would only work for single characters, not for looping or indexing >> (without the PEP, that is). So it's somewhat inconsistent. It would work >> well for literals, though. Also, we'd have to support it for 'in' tests, as >> a Py_UCS4 value may simply not be in a Py_UNICODE buffer, even though the >> character is in the string. > > No, I don't think we should handle surrogate pairs automatically, at > least without making it optional--this could be a significant > performance impact with little benefit for most users. Using these > higher characters is rare, but using them on a non USS4 build is > probably even rarer. Well, basically they are the only way to use 'wide' Unicode characters on 16bit Unicode builds. I think a unicode string of length 2 should be able to coerce into a Py_UCS4 value at runtime instead of raising the current exception because it's too long. For the opposite direction, integer to unicode string, you already get a string of length 2 on narrow builds, that's how unichr()/chr() work in Python 2/3. So, in a way, it's actually more consistent with how narrow builds work today. The only reason this isn't currently working in Cython is that Py_UNICODE is too small on narrow builds to represent the larger Unicode code points. If we switched to Py_UCS4, the problem would go away in narrow builds now and code could be written today that would easily continue to work efficiently in a post-PEP CPython as it wouldn't rely on the deprecated (and then inefficient) Py_UNICODE type anymore. What about supporting surrogate pairs in 'in' tests only on narrow platforms? I mean, we could simply duplicate the search code for that, depending on how large the code point value really is at runtime. That code will become a lot more involved anyway when the PEP gets implemented. > Also, this would be inconsistant with > python-level slicing, indexing, and range, right? Yes, it does not match well with slicing and indexing. That's the problem with narrow builds in both CPython and Cython. Only the PEP can fix that by basically dropping the restrictions of a narrow build. Stefan From robertwb at math.washington.edu Sun Jan 30 11:45:00 2011 From: robertwb at math.washington.edu (Robert Bradshaw) Date: Sun, 30 Jan 2011 02:45:00 -0800 Subject: [Cython] Fwd: sage.math sysadmin In-Reply-To: <1a91a357-a669-48c2-882b-9f0fc330afa7@w7g2000pre.googlegroups.com> References: <1a91a357-a669-48c2-882b-9f0fc330afa7@w7g2000pre.googlegroups.com> Message-ID: This is relevant to us, as we use the sage.math infrastructure. Thanks Joanna, and William, for supporting and letting us use your equipment. - Robert ---------- Forwarded message ---------- From: Joanna Date: Sat, Jan 29, 2011 at 4:25 PM Subject: sage.math sysadmin To: "sage.math users" Hello. I am William's new sysadmin for the sage.math cluster. I have a couple projects to start out with on the cluster (woohoo, expanded offsite backups), but I'll also try to pick up requests from this list as they happen. You can also just email me if you notice something that needs attention and might not interest everyone else. Thanks! -- Joanna -- You received this message because you are subscribed to the Google Groups "sage.math users" group. To post to this group, send email to sagemath-users at googlegroups.com To unsubscribe from this group, send email to sagemath-users+unsubscribe at googlegroups.com For more options, visit this group at http://groups.google.com/group/sagemath-users From stefan_ml at behnel.de Sun Jan 30 13:38:15 2011 From: stefan_ml at behnel.de (Stefan Behnel) Date: Sun, 30 Jan 2011 13:38:15 +0100 Subject: [Cython] Fwd: sage.math sysadmin In-Reply-To: References: <1a91a357-a669-48c2-882b-9f0fc330afa7@w7g2000pre.googlegroups.com> Message-ID: <4D455BB7.7080905@behnel.de> Robert Bradshaw, 30.01.2011 11:45: > This is relevant to us, as we use the sage.math infrastructure. Thanks > Joanna, and William, for supporting and letting us use your equipment. Big +1 from me. It continues to be a great help. Stefan