From p.giarrusso at gmail.com  Thu Jan  1 06:38:39 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Thu, 1 Jan 2009 06:38:39 +0100
Subject: [pypy-dev] [Fwd: [Fwd: Re: Threaded interpretation]]
In-Reply-To: <495AA2C0.9050105@gmail.com>
References: <495AA2C0.9050105@gmail.com>
Message-ID: <a4d0c86c0812312138p4742f398mf2017fc5af282ad9@mail.gmail.com>

On Tue, Dec 30, 2008 at 23:37, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Hi,
> Antoine Pitrou told me that his mail got rejected by the mailing list, so I'm
> forwarding it.

> -------- Message transf?r? --------
> De: Antoine Pitrou <solipsis at pitrou.net>
> ?: pypy-dev at codespeak.net
> Sujet: Re: Threaded interpretation
> Date: Fri, 26 Dec 2008 21:16:36 +0000 (UTC)
>
> Hi people,
>
> By reading this thread I had the idea to write a threaded interpretation patch
> for py3k. Speedup on pybench and pystone is 15-20%.
> http://bugs.python.org/issue4753

> Regards

> Antoine.

That's nice!

I didn't have time until now (Erasmus life is pretty demanding), but
it was nice anyway, also because the code is a bit different, and a
bit better than mine (I didn't realize I could fetch the operand after
dispatch instead of before, and this makes a difference, because after
the jump the result of HAS_ARG() becomes known at compile time).

Regards
-- 
Paolo Giarrusso

From arigo at tunes.org  Fri Jan  2 14:23:38 2009
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 2 Jan 2009 14:23:38 +0100
Subject: [pypy-dev] Threaded interpretation (was: Re: compiler
	optimizations: collecting ideas)
In-Reply-To: <a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>
References: <a4d0c86c0812202206l6d05b74fq8b9dbe6b175a2f66@mail.gmail.com>
	<494EC7EE.7000205@gmail.com>
	<a4d0c86c0812222116o458f9fcchd0543c871c6c1af0@mail.gmail.com>
	<20081223150113.GA1522@code0.codespeak.net>
	<a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>
Message-ID: <20090102132338.GA17873@code0.codespeak.net>

Hi Paolo,

On Thu, Dec 25, 2008 at 12:42:18AM +0100, Paolo Giarrusso wrote:
> If I'll want to try something without refcounting, I'll guess I'd turn
> to PyPy, but don't hold your breath for that. The fact that indirect
> threading didn't work, that you're 1.5-2x slower than CPython, and
> that you store locals in frame objects, they all show that the
> abstraction overhead of the interpret is too high.

True, but a 1.5x slowdown is not a big deal on many application; the
blocker is mostly elsewhere.  And on the other hand, we've been working
on the JIT generator -- since a while now, so I cannot make any promise
-- and the goal is to turn this slowish interpreter into a good JITting
one "for free".  As far as I know, this cannot be done so easily if you
start from a lower-level interpreter like CPython.  This is our "real"
goal somehow, or one of our real goals: a JITting virtual machine for
any language that we care to write an interpreter for.

> 3) still, I do believe that working on it was interesting to get
> experience about how to optimize an interpreter.

Sure, it makes some sense if you think about it in this way.  I doesn't
if you think about "I want to give the world a fast Python interpreter",
but you corrected me already: this is not your goal, so it's fine :-)

> And the original idea was to show that real multithreading (without a
> global interpreter lock) cannot be done in Python just because of the
> big design mistakes of CPython.

I disagree on this point.  Jython shows that without redesign of the
language, they can implement a real multithreading model without a GIL.
About PyPy, the lesson that we learned is different: it's that
implementing a GIL-free model requires a bit of tweaking of the whole
interpreter -- as opposed to doing it in a nicely separable manner.  So
far we only implemented threads-with-a-GIL, which can be done in this
separable way (i.e. without changing a single line of most of the
interpreter).  Given enough interest we can implement full GIL-free
multithreading the slow way: by going over most features of the
interpreter and thinking.  Note that it would also be possible to do
that in CPython (but it would be even more work).

It's just that doing so is not really "in the spirit" of PyPy :-)
In our style, we would rather start by thinking harder about some
unpolished ideas that float around about doing it in a separable
manner anyway -- but this has not been tried at all.


A bientot,

Armin.

From p.giarrusso at gmail.com  Sat Jan  3 05:59:57 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sat, 3 Jan 2009 05:59:57 +0100
Subject: [pypy-dev] Threaded interpretation (was: Re: compiler
	optimizations: collecting ideas)
In-Reply-To: <20090102132338.GA17873@code0.codespeak.net>
References: <a4d0c86c0812202206l6d05b74fq8b9dbe6b175a2f66@mail.gmail.com>
	<494EC7EE.7000205@gmail.com>
	<a4d0c86c0812222116o458f9fcchd0543c871c6c1af0@mail.gmail.com>
	<20081223150113.GA1522@code0.codespeak.net>
	<a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>
	<20090102132338.GA17873@code0.codespeak.net>
Message-ID: <a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>

On Fri, Jan 2, 2009 at 14:23, Armin Rigo <arigo at tunes.org> wrote:
> Hi Paolo,

> On Thu, Dec 25, 2008 at 12:42:18AM +0100, Paolo Giarrusso wrote:
>> If I'll want to try something without refcounting, I'll guess I'd turn
>> to PyPy, but don't hold your breath for that. The fact that indirect
>> threading didn't work, that you're 1.5-2x slower than CPython, and
>> that you store locals in frame objects, they all show that the
>> abstraction overhead of the interpret is too high.

> True, but a 1.5x slowdown is not a big deal on many application; the
> blocker is mostly elsewhere.

Let's say 1.5x * 2x = 3x, since CPython is not as fast as it could be,
because of refcounting for instance. The 2x is taken from PyVM
performance reports (see http://www.python.org/dev/implementations/).
And for PyPy a slower interpreter means a slower JIT output, isn't it?
See below.

Also, according to CS literature, interpreter performance makes more
difference than JIT. According to the paper about efficient
interpreters I'm mentioning endless times, an inefficient interpreter
is 1000x slower than C, while an efficient one is only 10x slower.
Python is not that slow, but what you wrote about Psyco seems to imply
that there is still lots of room for improvement.

"You might get 10x to 100x speed-ups. It is theoretically possible to
actually speed up this kind of code up to the performance of C
itself."

At some point, I should repeat these comparison with the OcaML interpreter:
  http://mail.python.org/pipermail/python-list/2000-December/063212.html
to see how faster it got :-).

> And on the other hand, we've been working
> on the JIT generator -- since a while now, so I cannot make any promise
> -- and the goal is to turn this slowish interpreter into a good JITting
> one "for free".  As far as I know, this cannot be done so easily if you
> start from a lower-level interpreter like CPython.  This is our "real"
> goal somehow, or one of our real goals: a JITting virtual machine for
> any language that we care to write an interpreter for.

That's amazing - I knew that, but your explanation is much nicer,
since it gives the emphasis to the right thing.
>From a native interpreter you can only do code copying, which removes
dispatch costs but is still suboptimal.

And well, the current overhead can be removed with further
improvements on the interpreter and/or the RPython translator. And it
is important, for two reasons:
1) an interpreter is still needed to profile the code; experience with
Java showed that only having a JIT gives too much latency, and people
expect even less latency from Python.
2) partially different translators are used I guess, but since the
compiler and the interpreter are built from the same source, part of
the 3x slowdown would have effect also on the JIT-ed code.

In any case, even if it may be unclear, I do like your plans.
Believe me, lack of time until now is the only thing which prevented
me from actually joining you and coding.

>> And the original idea was to show that real multithreading (without a
>> global interpreter lock) cannot be done in Python just because of the
>> big design mistakes of CPython.

> I disagree on this point.  Jython shows that without redesign of the
> language, they can implement a real multithreading model without a GIL.

Ok, first I have to rephrase what I said. Python in itself is a Good
Thing. BUT, there are some big design mistakes about some semantic
details of __del__, which make a difference for code using files
without calling close() for instance. To be polite, those semantics
are totally brain-dead, unless one worships reference counting and
considers GC as the devil.

Having clarified that, I can state that indeed, Jython may only show
the opposite of what you say; indeed I like to say that they didn't
bother to be bug-to-bug compatible with Python _specification bugs_,
and that's why it seems they proved your point.

Need I point out that reference cycles can't be reclaimed in Python if
any involved object implements __del__?
I know the Python gc module allows to fix that, but in really clumsy
ways. I was going to bash Python specs a lot more, but I'll spare your
time for now :-).

Regards
-- 
Paolo Giarrusso

From p.giarrusso at gmail.com  Sat Jan  3 07:18:22 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sat, 3 Jan 2009 07:18:22 +0100
Subject: [pypy-dev] GIL removal in PyPy (was: Re: Threaded interpretation)
Message-ID: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>

On Fri, Jan 2, 2009 at 14:23, Armin Rigo <arigo at tunes.org> wrote:

> About PyPy, the lesson that we learned is different: it's that
> implementing a GIL-free model requires a bit of tweaking of the whole
> interpreter -- as opposed to doing it in a nicely separable manner.  So
> far we only implemented threads-with-a-GIL, which can be done in this
> separable way (i.e. without changing a single line of most of the
> interpreter).

> Given enough interest we can implement full GIL-free
> multithreading the slow way: by going over most features of the
> interpreter and thinking.

That'll take more time now than it would have taken to do it in the first place.

IMHO, introducing fine-grained locking is one of the things costing x
if done from the beginning and costing 10x to do afterwards. Switching
from refcounting to GC is another of those things (the factor here is
maybe even higher), and your EU report on GC acknowledges this when
talking about CPython's choice of refcounting. The location of all
pointers on the C stack would have to be registered to convert
CPython.
So, I can understand what you mean by "nicely separable".

> Note that it would also be possible to do
> that in CPython (but it would be even more work).

That has been proven to be false :-).
It has been tried in 2004 (look for "Python free threading", I guess),
but free (i.e. GIL-less) threading requires atomic manipulation of
reference counts, and that gave a 2x overall slowdown; nowadays, the
relative impact would be even higher, just because . The reaction from
Guido van Rossum and/or the list (IIRC) was "ok, it can't be done,
we're not gonna have real multithreading, and there are better
programming models anyway".
There was, luckily, somebody on the mailing list who said "maybe we
should drop refcounting", but people didn't listen for some reason.

> It's just that doing so is not really "in the spirit" of PyPy :-)
> In our style, we would rather start by thinking harder about some
> unpolished ideas that float around about doing it in a separable
> manner anyway -- but this has not been tried at all.

That's interesting, can I get some pointers to these ideas?
I've heard about some work on this on the AOP front, but to me it
looks like their work is still not relevant, except for explaining us
that locking is a crosscutting concern, so I'd be interested about
something that could actually work.

And an _interpreter_ is not special at all for this, I think, it is
just another concurrent application (a compiler might be a different
case) - if you want to keep it simple, making everything in the data
space a monitor is the best bet. The problem is, that anything simple
will probably perform horribly and need to be optimized. And for me,
it's easier to write code in a much more sensible way from day one,
than optimizing it afterward.

=== Why AOP is broken for locking ===
I'm simply too used to advanced (and even not advanced) locking
pattern that can't be expressed in AOP. Given the following
independent snippets in an application:

LOCK(obj)
obj.a()
obj.b()
UNLOCK(obj)

LOCK(obj)
obj.b()
obj.c()
UNLOCK(obj)

how can an aspect introduce the calls to locks if they are omitted
from the source? I can't add an advice on a() or b(), the a()-b()
sequence must be atomic. And often I can't advice the containing
methods, unless it begins and ends with a locking call; obj might not
even be available at method entry. The only solution is to split each
critical sections out of the containing method into another one, but
that's not cheaper than AOP usage.

But the real challenge is how to implement, for instance, the
Collection.addAll(Collection b) Java method, if it has to lock both
dictionaries. Thread A might call a.addAll(b) while thread B calls
b.addAll(a) and any thread might attempt any concurrent modification
on any of a and b. So, which is the correct locking order to avoid
deadlock?

If I had really to face this problem (this particular example is
partially unrealistic - locking for addAll should be a responsibility
of the caller, probably), the only solution is to find some total
order on all collections and lock first the smaller object. In C, I
would compare their pointers, here I'd have to find some trick (like
comparing their identityHashCode() in Java).

I've been used to similar issues while coding in the Linux kernel, and
that's why I think that the locking concern should not be separated
from the application code. Unless one uses message passing, but I
can't see any way to implement multithreading by using message passing
that is not horribly inefficient.

The first Google hit on "AOP locking" is this article:
http://www.voelter.de/data/articles/aop/aop.html
and the example it makes is just too simple and unrealistic - that
pattern might apply to a minority of real-world cases. The author
seems unaware of real issues in concurrent code.

Moreover, I'm concerned about the quality of the article. When
discussing the "classic" OOP solution, it makes a really basic mistake
about the usage of inheritance. What follows is a copy-n-paste, with
added emphasis:
"The traditional approach is to introduce an (abstract) base class,
from which ALL "WORKER" CLASSES INHERIT. This class defines a method
lock() and a method unlock() which must be called before and after the
actual work is done (semaphores, basically)."
In fact, the lock() and unlock() method are not part of the API of the
Worker classes, so one would just use an instance of such a Lock
class, not inherit from it. And even writing "semaphore" is slightly
incorrect.

This article is much better:
http://www.kentlai.me/articles-journal/locks-via-spring-aop.html
still, it shares the same fundamental limitations. All method in a
class share the same lock, and locking only happens on method
boundaries. In fact, the article seems to acknowledge that AOP is just
a cheap way to introduce really simplistic locking.

Having said that, I'm really interested to see how JBoss uses AOP for
locking. But those problems seem really fundamental to me, so probably
just the simple cases are handled through AOP.

Regards
-- 
Paolo Giarrusso

From santagada at gmail.com  Sat Jan  3 12:05:25 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 3 Jan 2009 09:05:25 -0200
Subject: [pypy-dev] GIL removal in PyPy (was: Re: Threaded
	interpretation)
In-Reply-To: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
References: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
Message-ID: <26B655DE-C3D2-4BA3-851E-29BA0B9CB728@gmail.com>


On Jan 3, 2009, at 4:18 AM, Paolo Giarrusso wrote:

> There was, luckily, somebody on the mailing list who said "maybe we
> should drop refcounting", but people didn't listen for some reason.


You repeated this meme many times in your emails, so I thought that  
maybe you really didn't see the full picture. This is what I  
understand from the reasoning behind it.

Dropping refcounting and move to free threading would completely break  
all C modules so they would have to be rewritten and would make the  
CPython API much more complex and integration with C libraries hard.  
That's why no one took it seriously. Think like this, breaking all c  
modules would make CPython as usable as haskell :), or just look at  
the number of libraries not available right now for Python 3.0.

It is not some retarded choice made by GvR, but a pragmatic one.  
Python as a language used by millions of people can't completely  
change semantics from version to version.

--
Leonardo Santagada
santagada at gmail.com




From cfbolz at gmx.de  Sat Jan  3 13:23:03 2009
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 03 Jan 2009 13:23:03 +0100
Subject: [pypy-dev] GIL removal in PyPy (was: Re:
	Threaded	interpretation)
In-Reply-To: <26B655DE-C3D2-4BA3-851E-29BA0B9CB728@gmail.com>
References: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
	<26B655DE-C3D2-4BA3-851E-29BA0B9CB728@gmail.com>
Message-ID: <495F58A7.6080303@gmx.de>

Hi Leonardo,

Leonardo Santagada wrote:
> On Jan 3, 2009, at 4:18 AM, Paolo Giarrusso wrote:
> 
>> There was, luckily, somebody on the mailing list who said "maybe we
>> should drop refcounting", but people didn't listen for some reason.
> 
> 
> You repeated this meme many times in your emails, so I thought that  
> maybe you really didn't see the full picture. This is what I  
> understand from the reasoning behind it.
> 
> Dropping refcounting and move to free threading would completely break  
> all C modules so they would have to be rewritten and would make the  
> CPython API much more complex and integration with C libraries hard.  
> That's why no one took it seriously. Think like this, breaking all c  
> modules would make CPython as usable as haskell :), or just look at  
> the number of libraries not available right now for Python 3.0.
> 
> It is not some retarded choice made by GvR, but a pragmatic one.  
> Python as a language used by millions of people can't completely  
> change semantics from version to version.

While I understand the reasoning above, I don't agree with the last
paragraph. Switching to a different GC doesn't lead to that huge a
change in semantics. There are subtle difference about finalizers (see
the blog posts about that:

http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-1.html
http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-2.html

It definitely leads to a different C API, of course. Thus your above
points remain valid.

Cheers,

Carl Friedrich

From arigo at tunes.org  Sat Jan  3 15:12:44 2009
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 3 Jan 2009 15:12:44 +0100
Subject: [pypy-dev] Threaded interpretation (was: Re: compiler
	optimizations: collecting ideas)
In-Reply-To: <a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>
References: <a4d0c86c0812202206l6d05b74fq8b9dbe6b175a2f66@mail.gmail.com>
	<494EC7EE.7000205@gmail.com>
	<a4d0c86c0812222116o458f9fcchd0543c871c6c1af0@mail.gmail.com>
	<20081223150113.GA1522@code0.codespeak.net>
	<a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>
	<20090102132338.GA17873@code0.codespeak.net>
	<a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>
Message-ID: <20090103141244.GA6999@code0.codespeak.net>

Hi,

Your writing style is too annoying to answer properly :-(
I will not try to answer all of the points you mention;
instead, let's just point out (as others have done) that
the picture is more complex that you suppose, because it
includes a lot of aspects that you completely ignore.  I
know definitely that with changes to the semantics of the
language, Python could be made seriously faster.  PyPy is
not about this; it's a collection of people who either
don't care too much about performance, or believe that
even with these semantics Python *can* be made seriously
fast -- it's just harder, but not impossible.


A bientot,

Armin.

From arigo at tunes.org  Sat Jan  3 16:33:07 2009
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 3 Jan 2009 16:33:07 +0100
Subject: [pypy-dev] GIL removal in PyPy (was: Re: Threaded
	interpretation)
In-Reply-To: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
References: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
Message-ID: <20090103153306.GA13032@code0.codespeak.net>

Hi Paolo,

On Sat, Jan 03, 2009 at 07:18:22AM +0100, Paolo Giarrusso wrote:
> > Given enough interest we can implement full GIL-free
> > multithreading the slow way: by going over most features of the
> > interpreter and thinking.
> 
> That'll take more time now than it would have taken to do it in the first place.

Yes, so we probably won't do it (except if we can find another
orthogonal technique to try, i.e. a separable way, which is only at the
state of vague discussion so far).

> > Note that it would also be possible to do
> > that in CPython (but it would be even more work).
> 
> That has been proven to be false :-).

Wrong.  For example, Jython is such an example, showing that we "can" do
it even in C (becauce anything possible in Java is possible in C, given
enough efforts).  I know it has been tried several times over the course
of the CPython development, but never succeeded.  That's why I'm saying
it would be even more work in CPython than in PyPy.


A bientot,

Armin.

From p.giarrusso at gmail.com  Sat Jan  3 19:36:29 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sat, 3 Jan 2009 19:36:29 +0100
Subject: [pypy-dev] GIL removal in PyPy (was: Re: Threaded
	interpretation)
In-Reply-To: <20090103153306.GA13032@code0.codespeak.net>
References: <a4d0c86c0901022218y5ef828f6ja00b04fd0b4da0d7@mail.gmail.com>
	<20090103153306.GA13032@code0.codespeak.net>
Message-ID: <a4d0c86c0901031036x44292f0bp4d5d9892dfa6e577@mail.gmail.com>

Hi Armin,

On Sat, Jan 3, 2009 at 16:33, Armin Rigo <arigo at tunes.org> wrote:
>> > Note that it would also be possible to do
>> > that in CPython (but it would be even more work).

>> That has been proven to be false :-).

> Wrong.  For example, Jython is such an example, showing that we "can" do
> it even in C (becauce anything possible in Java is possible in C, given
> enough efforts).  I know it has been tried several times over the course
> of the CPython development, but never succeeded.  That's why I'm saying
> it would be even more work in CPython than in PyPy.

Obviously what can be done in Java can be done in C, but atomic
refcount manipulation is too expensive, so that, again, would require
dropping refcounting; in fact Jython has no refcounting.

Regards
-- 
Paolo Giarrusso

From p.giarrusso at gmail.com  Sun Jan  4 01:23:12 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sun, 4 Jan 2009 01:23:12 +0100
Subject: [pypy-dev] Threaded interpretation (was: Re: compiler
	optimizations: collecting ideas)
In-Reply-To: <20090103141244.GA6999@code0.codespeak.net>
References: <a4d0c86c0812202206l6d05b74fq8b9dbe6b175a2f66@mail.gmail.com>
	<494EC7EE.7000205@gmail.com>
	<a4d0c86c0812222116o458f9fcchd0543c871c6c1af0@mail.gmail.com>
	<20081223150113.GA1522@code0.codespeak.net>
	<a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>
	<20090102132338.GA17873@code0.codespeak.net>
	<a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>
	<20090103141244.GA6999@code0.codespeak.net>
Message-ID: <a4d0c86c0901031623h22875581i5013dea57ef7bce7@mail.gmail.com>

On Sat, Jan 3, 2009 at 15:12, Armin Rigo <arigo at tunes.org> wrote:
> Hi,

> Your writing style is too annoying to answer properly :-(

Sorry for that, I don't usually do it. I'm trying to do advocacy of my
position, but sometimes I realize there's a subtle boundary with
flaming; and most of that flaming is actually addressed to CPython
developers.

> I will not try to answer all of the points you mention;
> instead, let's just point out (as others have done) that
> the picture is more complex that you suppose, because it
> includes a lot of aspects that you completely ignore.

Tradeoffs are always involved, but in lots of cases, in the Python
community, misaddressed concerns about simplicity of the Python VM,
and ignorance of literature on Virtual Machines, lead to mistakes. And
I think we mostly agree on the facts underlying this statement, even
if we express that with different harshness.

I don't claim to be an expert of VM development, not at all.
But since there is a big disagreement between what I learned during 3
months of lessons (which was a first course on VM development) and
CPython, either one is wrong; my professor suggests that the Python
side is wrong and that implementers of Smalltalk, Self, and other
languages have some point. And all the evidence I could find
(including your project) agrees with that, on topics like:

- threading in the interpreter
- garbage collection vs reference counting
- tagged pointers
- hidden classes (i.e. V8 maps, which mostly date back to the early '90s).

If anybody is interested, this is the homepage of the course, with the
needed references (most of the links to papers are not directly
accessible, but the papers can be downloaded from other sources):

http://www.daimi.au.dk/~amoeller/VM/

== __DEL__ SEMANTICS ===

> I
> know definitely that with changes to the semantics of the
> language, Python could be made seriously faster.

> PyPy is
> not about this; it's a collection of people who either
> don't care too much about performance, or believe that
> even with these semantics Python *can* be made seriously
> fast -- it's just harder, but not impossible.

Just to make sure: I don't advocate changing semantics which have
value for the programmer; I advocate changing semantics when they have
problems anyway, like for this case, or for the int/long distinction,
which created problems both for tagged integers and for implementers.

Those two posts pointed out by Carl are interesting:

http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-1.html
http://morepypy.blogspot.com/2008/02/python-finalizers-semantics-part-2.html

I'm interested about the costs of guaranteeing destructor calls in
topological order, I might reconsider my position on that particular
point; however, "topological-order-unless-a-cycle-is-found" is IMHO a
worse semantics than "no-topological-order", and that's what actually
matters, because it complicates implementation without being easier to
use correctly, because using fields requires proving that the object
cannot be part of a cycle, and that requires in turn an analysis of
the internals of other objects; this shouldn't be required to the
programmer since data hiding mostly forbids relying on this.

It would maybe be more useful to implement "reverse topological order"
and to clear all pointers to children of an object, to make sure that
nobody tries to rely on undefined behaviour, at least for Java, but
maybe also for Python.

=== SUMMARY ===
So, the full Python semantics (i.e. __del__ is like a C++ destructor)
are surely easier to use than Java/.NET finalizers, and are safe,
because one cannot get NULL pointers; but they have for the huge
problem of memory leaks; I'm undecided about
"topological-order-unless-a-cycle-is-found"; Java finalizers are
something that nobody should use; Python weakrefs still allow
resurrection, which causes slowdown as documented by Boehm; Java
weakrefs involve a fair amount of boilerplate code and they also
require the programmer to decide where the reference queue should be
polled, but at least they are quite difficult (if not impossible) to
misuse.

Regards
-- 
Paolo Giarrusso

From cfbolz at gmx.de  Sun Jan  4 11:19:13 2009
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sun, 04 Jan 2009 11:19:13 +0100
Subject: [pypy-dev] Threaded interpretation (was: Re:
 compiler	optimizations: collecting ideas)
In-Reply-To: <a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>
References: <a4d0c86c0812202206l6d05b74fq8b9dbe6b175a2f66@mail.gmail.com>	<494EC7EE.7000205@gmail.com>	<a4d0c86c0812222116o458f9fcchd0543c871c6c1af0@mail.gmail.com>	<20081223150113.GA1522@code0.codespeak.net>	<a4d0c86c0812241542r28291965s92ab5c603451d0ab@mail.gmail.com>	<20090102132338.GA17873@code0.codespeak.net>
	<a4d0c86c0901022059v7821bf13n481ff2839a1384d2@mail.gmail.com>
Message-ID: <49608D21.5080301@gmx.de>

Hi Paolo,

Happy new year!

I am getting back to one of the points that Anto has made which you
dismissed. I still think that it is the central one to explain why we
don't think that threaded interpretation is the panacea that you think
it is.

Paolo Giarrusso wrote:
> On Fri, Jan 2, 2009 at 14:23, Armin Rigo <arigo at tunes.org> wrote:
>> On Thu, Dec 25, 2008 at 12:42:18AM +0100, Paolo Giarrusso wrote:
>>> If I'll want to try something without refcounting, I'll guess I'd
>>>  turn to PyPy, but don't hold your breath for that. The fact that
>>>  indirect threading didn't work, that you're 1.5-2x slower than 
>>> CPython, and that you store locals in frame objects, they all 
>>> show that the abstraction overhead of the interpret is too high.
> 
>> True, but a 1.5x slowdown is not a big deal on many application; 
>> the blocker is mostly elsewhere.
> 
> Let's say 1.5x * 2x = 3x, since CPython is not as fast as it could 
> be, because of refcounting for instance. The 2x is taken from PyVM 
> performance reports (see http://www.python.org/dev/implementations/).
>  And for PyPy a slower interpreter means a slower JIT output, isn't 
> it? See below.
> 
> Also, according to CS literature, interpreter performance makes more
>  difference than JIT. According to the paper about efficient 
> interpreters I'm mentioning endless times, an inefficient interpreter
>  is 1000x slower than C, while an efficient one is only 10x slower. 
> Python is not that slow, but what you wrote about Psyco seems to 
> imply that there is still lots of room for improvement.

I finally looked at the paper. I think that one of the basic premises of
the paper does not apply to Python. This is the premise that a VM
instruction itself is a simple operation. For example the paper states:

"Most VM instructions are quite simple (e.g., add the top two numbers on
the stack), and can be implemented in a few native instructions plus
dispatch code. Complex VM instructions occur relatively rarely, and the
setup for them typically takes many simple VM instructions; [...]"

This is just nonsense in Python, and indeed in many other dynamic
languages as well. Take a simple fibonacci function:

>>> def fibo(n):
...     if n <= 2:
...         return 1
...     return fibo(n - 1) + fibo(n - 2)
...
>>> import dis dis.dis(fibo)
  2           0 LOAD_FAST                0 (n)
              3 LOAD_CONST               1 (2)
              6 COMPARE_OP               1 (<=)
              9 JUMP_IF_FALSE            8 (to 20)
             12 POP_TOP

  3          13 LOAD_CONST               2 (1)
             16 RETURN_VALUE
             17 JUMP_FORWARD             1 (to 21)
        >>   20 POP_TOP

  4     >>   21 LOAD_GLOBAL              0 (fibo)
             24 LOAD_FAST                0 (n)
             27 LOAD_CONST               2 (1)
             30 BINARY_SUBTRACT
             31 CALL_FUNCTION            1
             34 LOAD_GLOBAL              0 (fibo)
             37 LOAD_FAST                0 (n)
             40 LOAD_CONST               1 (2)
             43 BINARY_SUBTRACT
             44 CALL_FUNCTION            1
             47 BINARY_ADD
             48 RETURN_VALUE


Of the opcodes present here, some are indeed simple (like LOAD_CONST and
JUMP_FORWARD, etc). However, most of them perform potentially complex
operations and a disproportionate amount of time is spent in them. E.g.
COMPARE_OP, BINARY_SUBTRACT, BINARY_ADD, CALL_FUNCTION
are implemented using special methods, so are essentially method calls,
which need a full method lookup (which can take a lot of time, given
complex class hierarchies). Then the method can be implemented in Python
which leads to even more overhead (yes, one could try to speed up common
cases like integers for binary operations, but those turn out to not be
all that common in normal Python code).

Even a seemingly harmless opcode like LOAD_GLOBAL is rather complex to
implement. First you need to check the global dictionary and then
potentially the builtins dictionary. I guess to be really conclusive one
would have to do measurements about which bytecodes take the most time
at runtime. However, there is some evidence from PyPy that what I said
is right: Implementing a method cache gave PyPy much larger speedups
than any of the experiments we ever did for speeding up dispatching.

For these reasons I argue that the paper you keep citing vastly
exaggerates the effect of bytecode dispatch for Python, and I would
guess also for other dynamic languages. I am entering pure speculation
here, but I could guess that the reason the Perl interpreter appears so
inefficient in the paper is because it has similar characteristics.
Being a bit mean, the paper seems to be a bit of a self-fulfilling
prophecy: "Efficient interpreters are those that have a large number of
indirect branches.  Therefore, efficient interpreters will be helped by
our techniques. If our techniques don't help, well, then the interpreter
just wasn't efficient enough in the first place."

Note that I am not saying that techniques like a threaded interpreter
loop cannot lead to a faster Python interpreter. I am pretty sure that
they can help. My main point is that they might not help as much as you
seem to hope, and that one needs additional things to get really great
performance results (and with great I mean basically anything that is
more than an order of magnitude better).

[snip]

Cheers,

Carl Friedrich


From guillem at torroja.dmt.upm.es  Fri Jan  9 10:54:51 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Fri, 9 Jan 2009 10:54:51 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
Message-ID: <200901091054.51758.guillem@torroja.dmt.upm.es>

Hello everyone

My name is Guillem Borrell and I am a researcher in the Computational Fluid 
Dynamics Laboratory in the School of Aeronautics, Universidad Polit?cnica de 
Madrid.

This year I'm in charge of organising the Supercomputing Day. Its aim is to 
present in form of talks the recent and future developments in high performance 
computing in science and engineering. It holds five talks, open to anyone, on 
one day and we expect that it will take place on Agust the 1st of 2009.

Most of supercomputing software use static languages such as Fortran or C but 
postprocessing is usually done in dynamic  languages, mostly matlab. While it 
uses dynamic languages on a daily basis the scientific community has almost 
never taken a look at what the virtual machines can offer in terms of 
versatility, ease of use and performance. 

Pypy is now at the bleeding edge of dynamic language implementation and I think 
that some of you can give some insight on what dynamic languages (and pypy, of 
course) can offer in the field of supercomputing.  I know that pypy is not only 
focused in performance but our intention is to get an overview, not only to talk 
about a single project.

My question is ?Is some of the pypy developers interested in give such a talk? 
If the answer is yes  ther's still plenty of time to think about the details.

I personall think that pypy is the most mind challenging software project and 
this can be a little effort to present some of its achievements to a broader 
audience.

We have two confirmed talks at the moment.  The first one will be on future 
microprocessor architecture development and the Cell architecture in particular 
and the second one will be about distributed file systems and Lustre.

Sincerely yours.
-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From holger at merlinux.eu  Fri Jan  9 19:34:21 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 9 Jan 2009 19:34:21 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901091054.51758.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
Message-ID: <20090109183421.GA2219@trillke.net>

Hi Guillemi!

thanks for mailing - this definitely sounds interesting to me.  
I discussed earlier today with Maciej Fijalkowski who is also
interested to come.  So let's stay in contact and discuss details. 

cheers,
holger


On Fri, Jan 09, 2009 at 10:54 +0100, Guillem Borrell i Nogueras wrote:
> Hello everyone
> 
> My name is Guillem Borrell and I am a researcher in the Computational Fluid 
> Dynamics Laboratory in the School of Aeronautics, Universidad Polit?cnica de 
> Madrid.
> 
> This year I'm in charge of organising the Supercomputing Day. Its aim is to 
> present in form of talks the recent and future developments in high performance 
> computing in science and engineering. It holds five talks, open to anyone, on 
> one day and we expect that it will take place on Agust the 1st of 2009.
> 
> Most of supercomputing software use static languages such as Fortran or C but 
> postprocessing is usually done in dynamic  languages, mostly matlab. While it 
> uses dynamic languages on a daily basis the scientific community has almost 
> never taken a look at what the virtual machines can offer in terms of 
> versatility, ease of use and performance. 
> 
> Pypy is now at the bleeding edge of dynamic language implementation and I think 
> that some of you can give some insight on what dynamic languages (and pypy, of 
> course) can offer in the field of supercomputing.  I know that pypy is not only 
> focused in performance but our intention is to get an overview, not only to talk 
> about a single project.
> 
> My question is ?Is some of the pypy developers interested in give such a talk? 
> If the answer is yes  ther's still plenty of time to think about the details.
> 
> I personall think that pypy is the most mind challenging software project and 
> this can be a little effort to present some of its achievements to a broader 
> audience.
> 
> We have two confirmed talks at the moment.  The first one will be on future 
> microprocessor architecture development and the Cell architecture in particular 
> and the second one will be about distributed file systems and Lustre.
> 
> Sincerely yours.
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mec?nica de Fluidos Computacional
> Escuela T?cnica Superior de Ingenieros Aeron?uticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From inhahe at gmail.com  Sun Jan 11 12:57:21 2009
From: inhahe at gmail.com (inhahe)
Date: Sun, 11 Jan 2009 06:57:21 -0500
Subject: [pypy-dev] fast
Message-ID: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>

With trace trees, hidden classes, aggressive type speculation, etc.,
javascript engines run code about as fast as unoptimized C.
And I know that Python isn't any more intractable than Javascript
because pyjamas automatically translates from one to the other.

So however you're doing it now.. you're doing it wrong. ;)

If CPython won't do it, maybe you will..

HTH,
inhahe

From fijall at gmail.com  Sun Jan 11 13:26:22 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 11 Jan 2009 13:26:22 +0100
Subject: [pypy-dev] fast
In-Reply-To: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
Message-ID: <693bc9ab0901110426s1476ff29yd78e84009219e1d2@mail.gmail.com>

We had this discussion couple of times already, but I'll try to recap.

On Sun, Jan 11, 2009 at 12:57 PM, inhahe <inhahe at gmail.com> wrote:
> With trace trees, hidden classes, aggressive type speculation, etc.,
> javascript engines run code about as fast as unoptimized C.
> And I know that Python isn't any more intractable than Javascript
> because pyjamas automatically translates from one to the other.

Well, wrong:

1. pyjamas is translating python syntax to js syntax (even not all of
them). Which mean mostly that semantics are python. That said, you
cannot run the same program under python and under js after pyjamas
and expect the same result. Simple example would be as follow:

d[index]

In python, if d is a dictionary, this will raise KeyError, if d is a
list this will raise IndexError, if d is a custom object, it'll try to
call it's __getitem__. In JS however (after pyjamas) this will simply
be translated to d[index], which will return undefined if index is not
there (and not raise any exception).

2. python is far more complex than javascript, someone tried to count
LOC of our python implementation vs our javascript implementation and
it was something in order of 6-10x more code for python. It's also far
more dynamic.

3. Psyco runs programs (let's say integer computations) as far as
unoptimized C. Our prototype JIT does as well and we're working hard
on improving things (in fact we work a lot on JIT these days).

>
> So however you're doing it now.. you're doing it wrong. ;)

Well, tell us how to do it better/faster? Your points are rather
shallow as it seems you didn't look into anything in detail. Look
deeper, maybe you can give us a useful advice.

>
> If CPython won't do it, maybe you will..

Thanks for trust :)

cheers,
fijal

From jewel at subvert-the-dominant-paradigm.net  Sun Jan 11 17:20:20 2009
From: jewel at subvert-the-dominant-paradigm.net (John Leuner)
Date: Sun, 11 Jan 2009 18:20:20 +0200
Subject: [pypy-dev] fast
In-Reply-To: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
Message-ID: <1231690820.13208.5.camel@cmalu.WAG54GS>

On Sun, 2009-01-11 at 06:57 -0500, inhahe wrote:
> With trace trees, hidden classes, aggressive type speculation, etc.,
> javascript engines run code about as fast as unoptimized C.

What benchmarks/tests have you seen that show this?

John



From p.giarrusso at gmail.com  Sun Jan 11 20:44:54 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sun, 11 Jan 2009 20:44:54 +0100
Subject: [pypy-dev] fast
In-Reply-To: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
Message-ID: <a4d0c86c0901111144s297a115p5ab76ab948c9fb0c@mail.gmail.com>

On Sun, Jan 11, 2009 at 12:57, inhahe <inhahe at gmail.com> wrote:
> With trace trees, hidden classes, aggressive type speculation, etc.,
> javascript engines run code about as fast as unoptimized C.

> And I know that Python isn't any more intractable than Javascript
> because pyjamas automatically translates from one to the other.

This is not a valid reasoning, as discussed in the other mail.

Having said that, in the end we all agree that even Python can get
that speed, the only thing we argue about is how complex is doing
that.

> So however you're doing it now.. you're doing it wrong. ;)

> If CPython won't do it, maybe you will..

I don't see the point of your mail, is it just trolling? IMHO, one can
only write "you're doing it wrong" if he's able to do better, and has
proven that in practice.

Btw, I'm not (yet) part of the PyPy project, and I've been discussing
the same issues as you. That doesn't make me agree with such mails.
-- 
Paolo Giarrusso

From p.giarrusso at gmail.com  Sun Jan 11 21:09:05 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sun, 11 Jan 2009 21:09:05 +0100
Subject: [pypy-dev] fast
In-Reply-To: <693bc9ab0901110426s1476ff29yd78e84009219e1d2@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
	<693bc9ab0901110426s1476ff29yd78e84009219e1d2@mail.gmail.com>
Message-ID: <a4d0c86c0901111209x3bbf3528uc0ec9e0e3f6c5a04@mail.gmail.com>

On Sun, Jan 11, 2009 at 13:26, Maciej Fijalkowski <fijall at gmail.com> wrote:
> We had this discussion couple of times already, but I'll try to recap.

> On Sun, Jan 11, 2009 at 12:57 PM, inhahe <inhahe at gmail.com> wrote:
>> With trace trees, hidden classes, aggressive type speculation, etc.,
>> javascript engines run code about as fast as unoptimized C.
>> And I know that Python isn't any more intractable than Javascript
>> because pyjamas automatically translates from one to the other.

> Well, wrong:

> 1. pyjamas is translating python syntax to js syntax (even not all of
> them). Which mean mostly that semantics are python. That said, you
> cannot run the same program under python and under js after pyjamas
> and expect the same result. Simple example would be as follow:

> d[index]

> In python, if d is a dictionary, this will raise KeyError, if d is a
> list this will raise IndexError, if d is a custom object, it'll try to
> call it's __getitem__. In JS however (after pyjamas) this will simply
> be translated to d[index], which will return undefined if index is not
> there (and not raise any exception).

You have a point on semantic differences. However, such a problem does
not make any of the two implementations faster than the other, and the
error handling can be as slow as needed to make the common case fast.

> 2. python is far more complex than javascript, someone tried to count
> LOC of our python implementation vs our javascript implementation and
> it was something in order of 6-10x more code for python.

Well, I expect your Python implementation to be far more optimized
than your Javascript one, isn't it? And the 80-20% (or 90-10%) rule
still applies. If the language allows fancier things, one would
probably focus on standard practices. Take __slots__. Once one
implements V8 maps, __slots__ becomes useless, and it's implementation
would simply reuse the one for normal objects.

Also, if extra complexity is needed for weird stuff, most of the times
it shouldn't affect the fast path. The existence of sys._getframe() is
one of the exceptions, and f_lasti a particularly bad one (it has
10-20% impact on execution runtime).

> It's also far
> more dynamic.

Can you explain why, or point to the archives where it was discussed?
I made a quick search but I couldn't find the right keywords to use.

> 3. Psyco runs programs (let's say integer computations) as far as
> unoptimized C. Our prototype JIT does as well and we're working hard
> on improving things (in fact we work a lot on JIT these days).

-- 
Paolo Giarrusso

From fijall at gmail.com  Sun Jan 11 21:16:15 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 11 Jan 2009 21:16:15 +0100
Subject: [pypy-dev] fast
In-Reply-To: <a4d0c86c0901111209x3bbf3528uc0ec9e0e3f6c5a04@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
	<693bc9ab0901110426s1476ff29yd78e84009219e1d2@mail.gmail.com>
	<a4d0c86c0901111209x3bbf3528uc0ec9e0e3f6c5a04@mail.gmail.com>
Message-ID: <693bc9ab0901111216s61787a7dybaeac54b9c430c27@mail.gmail.com>

On Sun, Jan 11, 2009 at 9:09 PM, Paolo Giarrusso <p.giarrusso at gmail.com> wrote:
> On Sun, Jan 11, 2009 at 13:26, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> We had this discussion couple of times already, but I'll try to recap.
>
>> On Sun, Jan 11, 2009 at 12:57 PM, inhahe <inhahe at gmail.com> wrote:
>>> With trace trees, hidden classes, aggressive type speculation, etc.,
>>> javascript engines run code about as fast as unoptimized C.
>>> And I know that Python isn't any more intractable than Javascript
>>> because pyjamas automatically translates from one to the other.
>
>> Well, wrong:
>
>> 1. pyjamas is translating python syntax to js syntax (even not all of
>> them). Which mean mostly that semantics are python. That said, you
>> cannot run the same program under python and under js after pyjamas
>> and expect the same result. Simple example would be as follow:
>
>> d[index]
>
>> In python, if d is a dictionary, this will raise KeyError, if d is a
>> list this will raise IndexError, if d is a custom object, it'll try to
>> call it's __getitem__. In JS however (after pyjamas) this will simply
>> be translated to d[index], which will return undefined if index is not
>> there (and not raise any exception).
>
> You have a point on semantic differences. However, such a problem does
> not make any of the two implementations faster than the other, and the
> error handling can be as slow as needed to make the common case fast.

I was just pointing out that argument is completely void since pyjamas
is not transforming semantics.

>
>> 2. python is far more complex than javascript, someone tried to count
>> LOC of our python implementation vs our javascript implementation and
>> it was something in order of 6-10x more code for python.
>
> Well, I expect your Python implementation to be far more optimized
> than your Javascript one, isn't it? And the 80-20% (or 90-10%) rule
> still applies. If the language allows fancier things, one would
> probably focus on standard practices. Take __slots__. Once one
> implements V8 maps, __slots__ becomes useless, and it's implementation
> would simply reuse the one for normal objects.

That's true.

>
> Also, if extra complexity is needed for weird stuff, most of the times
> it shouldn't affect the fast path. The existence of sys._getframe() is
> one of the exceptions, and f_lasti a particularly bad one (it has
> 10-20% impact on execution runtime).
>
>> It's also far
>> more dynamic.
>
> Can you explain why, or point to the archives where it was discussed?
> I made a quick search but I couldn't find the right keywords to use.

Well for various reasons. Take for example descriptor semantics or
metaclasses. I'm not up to discuss exactly which language is more
dynamic, because this is a valid point for static compilation. The
point is that python is more complex and require more effort to make
it fast.

cheers,
fijal

From jewel at subvert-the-dominant-paradigm.net  Sun Jan 11 22:02:14 2009
From: jewel at subvert-the-dominant-paradigm.net (John Leuner)
Date: Sun, 11 Jan 2009 23:02:14 +0200
Subject: [pypy-dev] fast
In-Reply-To: <a4d0c86c0901111206o1e3a4dcya4af6cd65a542aec@mail.gmail.com>
References: <da776a8c0901110357v36017268r76dcef55cc1d74b4@mail.gmail.com>
	<1231690820.13208.5.camel@cmalu.WAG54GS>
	<a4d0c86c0901111206o1e3a4dcya4af6cd65a542aec@mail.gmail.com>
Message-ID: <1231707734.13208.16.camel@cmalu.WAG54GS>

On Sun, 2009-01-11 at 21:06 +0100, Paolo Giarrusso wrote:
> On Sun, Jan 11, 2009 at 17:20, John Leuner
> <jewel at subvert-the-dominant-paradigm.net> wrote:
> > On Sun, 2009-01-11 at 06:57 -0500, inhahe wrote:
> >> With trace trees, hidden classes, aggressive type speculation, etc.,
> >> javascript engines run code about as fast as unoptimized C.
> >
> > What benchmarks/tests have you seen that show this?
> 
> That's first a claim from this post, from TraceMonkey author
> http://andreasgal.com/2008/08/22/tracing-the-web/
> 
> This is the benchmark saying that:
> http://shaver.off.net/diary/2008/08/22/the-birth-of-a-faster-monkey/
> 
> And TraceMonkey is quite young.

Quoting from that site:

"The goal of the TraceMonkey project ? which is still in its early
stages ? is to take JavaScript performance to another level, where
instead of competing against other interpreters, we start to compete
against native code. Even with this very, very early version we?re
already seeing some promising results: a simple ?for loop? is getting
close to unoptimized gcc:"

This claim about unoptimized C is only about a "for loop". We all know
that javascript and python execution involves much more than simple for
loops.

John





From guillem at torroja.dmt.upm.es  Mon Jan 12 12:10:50 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Mon, 12 Jan 2009 12:10:50 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <20090109183421.GA2219@trillke.net>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
Message-ID: <200901121210.50200.guillem@torroja.dmt.upm.es>

Hi again

Let's discuss the details.

I'll try to explain why I've thought about pypy when planning the conference 
sessions.

My work as Computational Fluid Dynamics researcher is intimately related to 
supercomputing for obvious reasons. Most of applications we work on are fine 
tuned codes with pieces that are more than twenty years old.  They are rock 
solid, fortan implemented, and run for hours in clusters of thousand of 
computing nodes.

Since the last couple of years computer architectures are becoming more and more 
complex. I'm playing with the cell processor lately and that little bastard is 
causing me real pain. While programming is easier every day, supercomputing is 
harder and harder.  Think about an arhitecture like the roadrunner, AMD Opteron 
PPU with PowerPC SPU... Two assemblers in one chip!

Talking with Stanley Ahalt (Ohio Supercomputing Center) about a year ago he 
called that the "software gap". In computing, as times goes, low performance is 
easier but high performance is harder. And that gap gets wider. Platform SDK are 
helpful but they are not a huge leap.

I've always thought that virtual machines could help supercomputing like they 
have helped grid and cloud computing.  This is the point where I need someone to 
proof if I am wright or wrong.  Pypy is the most versatile, albeit complex, 
dynamic language implementation. I've been following the project during the last 
year and a half or so and I am impressed. I've thought that you could have a 
vision on how interpreted languages and virtual machines can help managing 
complexity.

In addition, most of postprocessing tools are written in matlab, an interpreted 
language.  Running not-so-high performance tasks in a workstation efficiently is 
sometimes as important as running a simulation in a 12000 node supercomputer. It 
yould be nice if someone would remind the audience that matlab is not the only 
suitable (or best) tool for that job.

I hope that I managed to explain the topics I'm interested in.

Those are the speakers that accepted the invitation at the moment:

* J.M?. Cela, Computer Architecture Department, Universitat Polit?cnica de 
Catalunya and Barcelona Supercomputing Center. Future supercomputers, CFD 
applications and the Cell processor.

* Sun Microsystems. Distributed Filesystems and Lustre.

* S. Hoyas. School of Aeronautics. Universitat Polit?cnica de Val?ncia. Running 
a supercompuingr simulation in more than 2000 processors.

I'm very interested in your comments.

cheers.

guillem

El Friday 09 January 2009 19:34:21 holger krekel escribi?:
> Hi Guillemi!
>
> thanks for mailing - this definitely sounds interesting to me.
> I discussed earlier today with Maciej Fijalkowski who is also
> interested to come.  So let's stay in contact and discuss details.
>
> cheers,
> holger
>
> On Fri, Jan 09, 2009 at 10:54 +0100, Guillem Borrell i Nogueras wrote:
> > Hello everyone
> >
> > My name is Guillem Borrell and I am a researcher in the Computational
> > Fluid Dynamics Laboratory in the School of Aeronautics, Universidad
> > Polit?cnica de Madrid.
> >
> > This year I'm in charge of organising the Supercomputing Day. Its aim is
> > to present in form of talks the recent and future developments in high
> > performance computing in science and engineering. It holds five talks,
> > open to anyone, on one day and we expect that it will take place on Agust
> > the 1st of 2009.
> >
> > Most of supercomputing software use static languages such as Fortran or C
> > but postprocessing is usually done in dynamic  languages, mostly matlab.
> > While it uses dynamic languages on a daily basis the scientific community
> > has almost never taken a look at what the virtual machines can offer in
> > terms of versatility, ease of use and performance.
> >
> > Pypy is now at the bleeding edge of dynamic language implementation and I
> > think that some of you can give some insight on what dynamic languages
> > (and pypy, of course) can offer in the field of supercomputing.  I know
> > that pypy is not only focused in performance but our intention is to get
> > an overview, not only to talk about a single project.
> >
> > My question is ?Is some of the pypy developers interested in give such a
> > talk? If the answer is yes  ther's still plenty of time to think about
> > the details.
> >
> > I personall think that pypy is the most mind challenging software project
> > and this can be a little effort to present some of its achievements to a
> > broader audience.
> >
> > We have two confirmed talks at the moment.  The first one will be on
> > future microprocessor architecture development and the Cell architecture
> > in particular and the second one will be about distributed file systems
> > and Lustre.
> >
> > Sincerely yours.
> > --
> > guillem
> >
> > Guillem Borrell i Nogueras
> > Laboratorio de Mec?nica de Fluidos Computacional
> > Escuela T?cnica Superior de Ingenieros Aeron?uticos
> > guillem at torroja.dmt.upm.es
> > Web: http://torroja.dmt.upm.es/guillem/blog
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From p.giarrusso at gmail.com  Mon Jan 12 12:50:35 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Mon, 12 Jan 2009 12:50:35 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901121210.50200.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
Message-ID: <a4d0c86c0901120350p296e5a2cn85c4d8634c42f6c9@mail.gmail.com>

Hi, first I'd like to qualify myself as a student of Virtual Machine
implementations, not working (yet) on PyPy itself, and aware of some
HPC issues at a basic level. Still, I'd like to help pinpointing the
discussion.

On Mon, Jan 12, 2009 at 12:10, Guillem Borrell i Nogueras
<guillem at torroja.dmt.upm.es> wrote:
> Hi again

> Let's discuss the details.

> I'll try to explain why I've thought about pypy when planning the conference
> sessions.

> My work as Computational Fluid Dynamics researcher is intimately related to
> supercomputing for obvious reasons. Most of applications we work on are fine
> tuned codes with pieces that are more than twenty years old.  They are rock
> solid, fortan implemented, and run for hours in clusters of thousand of
> computing nodes.

> Since the last couple of years computer architectures are becoming more and more
> complex. I'm playing with the cell processor lately and that little bastard is
> causing me real pain. While programming is easier every day, supercomputing is
> harder and harder.  Think about an arhitecture like the roadrunner, AMD Opteron
> PPU with PowerPC SPU... Two assemblers in one chip!

> Talking with Stanley Ahalt (Ohio Supercomputing Center) about a year ago he
> called that the "software gap". In computing, as times goes, low performance is
> easier but high performance is harder. And that gap gets wider. Platform SDK are
> helpful but they are not a huge leap.

> I've always thought that virtual machines could help supercomputing like they
> have helped grid and cloud computing.  This is the point where I need someone to
> proof if I am wright or wrong.  Pypy is the most versatile, albeit complex,
> dynamic language implementation. I've been following the project during the last
> year and a half or so and I am impressed. I've thought that you could have a
> vision on how interpreted languages and virtual machines can help managing
> complexity.

> In addition, most of postprocessing tools are written in matlab, an interpreted
> language.  Running not-so-high performance tasks in a workstation efficiently is
> sometimes as important as running a simulation in a 12000 node supercomputer. It
> yould be nice if someone would remind the audience that matlab is not the only
> suitable (or best) tool for that job.

This is IMHO an issue with the expressivity of Matlab & Python. There
are Python libraries for that, but Python has not a Domain-Specific
syntax I guess - you'd need to spell out method names sometimes,
instead of using / and ./. Slicing is something already existing,
though.
However, JIT-compiled Python would have the huge advantage to make
also loops more efficient, instead of forcing the user to write all
loops in terms of matrix parallel operations. That was the biggest
slowdown in Matlab development I experienced. And I remember fixing
that in somebody else's program, getting from 12 hours to 1 minute of
runtime. 720x speedup. Interpreters can be much faster than that, even
CPython is already faster in that area.

The advantage of vectorized loops is that they easily use optimized
BLAS routines, maybe SSE/Cell processor based ones.

> I'm very interested in your comments.

Now, the first question is: do you have Python to be faster than
Matlab or faster than Fortran? VMs can be faster than C++ (for
instance, static C++ can't inline virtual methods).

That makes a huge difference. I'm unaware of why one can't use its
Fortran source on Cell. OK, I can wildly guess that having two
specialized microprocessors, one might like having an automatic
parallelizer to split certain operations to one and certain to
another? Can you make some better example?

The obvious ones are automatic vectorization and automatic
parallelization, but I'm not aware of any VM-specific research on that
topic. And automatic parallelization is a quite difficult topic
anyway, as far as I know.

In other words: is there any special advantage given by adaptive
optimization (even profile-based one) that static optimizations (like
the ones done by ICC) cannot match?
None is obvious for vectorization, automatic tuning of sizes comes to
mind for auto parallelization.

In general it is well known that the more a language is high-level,
the more informations the compiler has to optimize it, but also the
more the language has fancy features not trivial to optimize.
Actually, Fortran is better than C exactly because is more high-level.
Most C code assumes for instance GCC option -fno-strict-aliasing,
which is contrary to the language semantics and forbids many
interesting optimizations.

See this website for information (note: I found the link somewhere
else, but be careful that according to Google&Firefox, the server is
under control of hackers and is spreading viruses; I run Linux and I'm
safe, YMMV):
http://www.cellperformance.com/mike_acton/2006/06/understanding_strict_aliasing.html

Having said that, the point is to understand which optimizations do
you need to perform manually and could be performed automatically by a
VM? Note that your Fortran compiler probably has a far better static
code optimizer. If you write plain Fortran code in Python, it's going
to be much slower (like if no optimization were present) until
dataflow analysis, register allocation, instruction scheduling etc.
will be implemented in PyPy, after all the rest is finished. It's just
a matter of implementation costs, but it is huge.
Where VM shine is when they can do optimizations unavailable to static
compilers, like adaptive optimizations. Inlining of virtual method is
an example, but automatic prefetching from memory into cache (by
adding SSE prefetch instructions) is another on which research has
been done, for instance, just to make an example.

Regards
-- 
Paolo Giarrusso

From fijall at gmail.com  Mon Jan 12 17:05:15 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 12 Jan 2009 17:05:15 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <a4d0c86c0901120350p296e5a2cn85c4d8634c42f6c9@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<a4d0c86c0901120350p296e5a2cn85c4d8634c42f6c9@mail.gmail.com>
Message-ID: <693bc9ab0901120805l74e95faan2d0117fe826a2021@mail.gmail.com>

On Mon, Jan 12, 2009 at 12:50 PM, Paolo Giarrusso <p.giarrusso at gmail.com> wrote:
> Hi, first I'd like to qualify myself as a student of Virtual Machine
> implementations, not working (yet) on PyPy itself, and aware of some
> HPC issues at a basic level. Still, I'd like to help pinpointing the
> discussion.
> ...

Sorry, but I would like to point out that this thread is about
possible conference appearance, not general discussion about HPC.

Things that you mentioned are interesting, but off topic and are
possibly well known to Guillem I suppose.

Paolo, please feel free to use pypy-dev as a discussion place,
especially regarding implementing dynamic VMs, but we would be
grateful if you stick to some guidelines:

* Stick to the topic of current discussion.

* Discuss one issue at a time, and not a complete flow of buzzwords.

* Try to provide more sources/details rather than general statements.

Thanks and cheers,
fijal

From van.lindberg at gmail.com  Mon Jan 12 18:12:39 2009
From: van.lindberg at gmail.com (VanL)
Date: Mon, 12 Jan 2009 11:12:39 -0600
Subject: [pypy-dev] Non-circular path through pypy module imports
Message-ID: <gkftm6$qfk$1@ger.gmane.org>

Hello,

I have been going through the pypy sources recently and I wanted to start from
the leaf modules first (in terms of import dependencies)... but there don't
appear to be any leaves.

For example, starting with baseobjspace in the interpreter, it has 30 or so
non-stdlib imports, many of them delayed by placing them in functions. Even
something like rarithmetic eventually pulls in most of the lltype hierarchy. The
closest I found to a leaf module was extregistry.py.

Is there a reason for the circularities that I just don't get? If not, why are
there so many?

Thanks,

Van


From holger at merlinux.eu  Tue Jan 13 13:31:07 2009
From: holger at merlinux.eu (holger krekel)
Date: Tue, 13 Jan 2009 13:31:07 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901121210.50200.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
Message-ID: <20090113123107.GL2219@trillke.net>

Hi Guillem! 

On Mon, Jan 12, 2009 at 12:10 +0100, Guillem Borrell i Nogueras wrote:
> Hi again
> 
> Let's discuss the details.
> 
> I'll try to explain why I've thought about pypy when planning the conference 
> sessions.
> 
> My work as Computational Fluid Dynamics researcher is intimately related to 
> supercomputing for obvious reasons. Most of applications we work on are fine 
> tuned codes with pieces that are more than twenty years old.  They are rock 
> solid, fortan implemented, and run for hours in clusters of thousand of 
> computing nodes.
> 
> Since the last couple of years computer architectures are becoming more and more 
> complex. I'm playing with the cell processor lately and that little bastard is 
> causing me real pain. While programming is easier every day, supercomputing is 
> harder and harder.  Think about an arhitecture like the roadrunner, AMD Opteron 
> PPU with PowerPC SPU... Two assemblers in one chip!

Could you give examples on the concrete challenges you face?

> Talking with Stanley Ahalt (Ohio Supercomputing Center) about a year ago he 
> called that the "software gap". In computing, as times goes, low performance is 
> easier but high performance is harder. And that gap gets wider. Platform SDK are 
> helpful but they are not a huge leap.
> 
> I've always thought that virtual machines could help
> supercomputing like they have helped grid and cloud
> computing.  This is the point where I need someone to proof
> if I am wright or wrong.  Pypy is the most versatile, albeit
> complex, dynamic language implementation. I've been
> following the project during the last year and a half or so
> and I am impressed. I've thought that you could have a
> vision on how interpreted languages and virtual machines can
> help managing complexity.

The PyPy project is mostly concerned with reducing the complexity 
of writing efficient dynamic language interpreters. 
PyPy's JIT and its flexible backend implementation 
should help with bridging the software gap IIUC. 
It probably makes sense to stress and elaborate
on this point and to discuss examples in the talk. 

Managing the complexity of software deployment and integration
is not PyPy's primary concern but it provides some interesting
building blocks.  For example, i am interested in using using
PyPy's sandboxing and virtualization capabilities to build an
open "cloud at home" network: people would run a bare-bones
executable, configure & control CPU, RAM, Network, Disk
resources and access rights. On this network e.g. open source
developers or scientists could launch programs and
computations or start os-level virtual machines where
non-python code runs.  

> In addition, most of postprocessing tools are written in matlab, an interpreted 
> language.  Running not-so-high performance tasks in a workstation efficiently is 
> sometimes as important as running a simulation in a 12000 node supercomputer. It 
> yould be nice if someone would remind the audience that matlab is not the only 
> suitable (or best) tool for that job.

Right, I guess that something like Numpy combined with PyPy's JIT 
would could make a killer combination.  That's work, though. 

> I hope that I managed to explain the topics I'm interested in.

sure, let us know what you think makes most sense for
the audience - who will be the audience btw? 

thanks & best,

holger


 
> Those are the speakers that accepted the invitation at the moment:
> 
> * J.M?. Cela, Computer Architecture Department, Universitat Polit?cnica de 
> Catalunya and Barcelona Supercomputing Center. Future supercomputers, CFD 
> applications and the Cell processor.
> 
> * Sun Microsystems. Distributed Filesystems and Lustre.
> 
> * S. Hoyas. School of Aeronautics. Universitat Polit?cnica de Val?ncia. Running 
> a supercompuingr simulation in more than 2000 processors.
> 
> I'm very interested in your comments.

> cheers.
> 
> guillem
> 
> El Friday 09 January 2009 19:34:21 holger krekel escribi?:
> > Hi Guillemi!
> >
> > thanks for mailing - this definitely sounds interesting to me.
> > I discussed earlier today with Maciej Fijalkowski who is also
> > interested to come.  So let's stay in contact and discuss details.
> >
> > cheers,
> > holger
> >
> > On Fri, Jan 09, 2009 at 10:54 +0100, Guillem Borrell i Nogueras wrote:
> > > Hello everyone
> > >
> > > My name is Guillem Borrell and I am a researcher in the Computational
> > > Fluid Dynamics Laboratory in the School of Aeronautics, Universidad
> > > Polit?cnica de Madrid.
> > >
> > > This year I'm in charge of organising the Supercomputing Day. Its aim is
> > > to present in form of talks the recent and future developments in high
> > > performance computing in science and engineering. It holds five talks,
> > > open to anyone, on one day and we expect that it will take place on Agust
> > > the 1st of 2009.
> > >
> > > Most of supercomputing software use static languages such as Fortran or C
> > > but postprocessing is usually done in dynamic  languages, mostly matlab.
> > > While it uses dynamic languages on a daily basis the scientific community
> > > has almost never taken a look at what the virtual machines can offer in
> > > terms of versatility, ease of use and performance.
> > >
> > > Pypy is now at the bleeding edge of dynamic language implementation and I
> > > think that some of you can give some insight on what dynamic languages
> > > (and pypy, of course) can offer in the field of supercomputing.  I know
> > > that pypy is not only focused in performance but our intention is to get
> > > an overview, not only to talk about a single project.
> > >
> > > My question is ?Is some of the pypy developers interested in give such a
> > > talk? If the answer is yes  ther's still plenty of time to think about
> > > the details.
> > >
> > > I personall think that pypy is the most mind challenging software project
> > > and this can be a little effort to present some of its achievements to a
> > > broader audience.
> > >
> > > We have two confirmed talks at the moment.  The first one will be on
> > > future microprocessor architecture development and the Cell architecture
> > > in particular and the second one will be about distributed file systems
> > > and Lustre.
> > >
> > > Sincerely yours.
> > > --
> > > guillem
> > >
> > > Guillem Borrell i Nogueras
> > > Laboratorio de Mec?nica de Fluidos Computacional
> > > Escuela T?cnica Superior de Ingenieros Aeron?uticos
> > > guillem at torroja.dmt.upm.es
> > > Web: http://torroja.dmt.upm.es/guillem/blog
> > > _______________________________________________
> > > pypy-dev at codespeak.net
> > > http://codespeak.net/mailman/listinfo/pypy-dev
> 
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mec?nica de Fluidos Computacional
> Escuela T?cnica Superior de Ingenieros Aeron?uticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From guillem at torroja.dmt.upm.es  Tue Jan 13 16:17:53 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Tue, 13 Jan 2009 16:17:53 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <20090113123107.GL2219@trillke.net>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<20090113123107.GL2219@trillke.net>
Message-ID: <200901131617.54395.guillem@torroja.dmt.upm.es>

Hi!

El Tuesday 13 January 2009 13:31:07 holger krekel escribi?:

>
> Could you give examples on the concrete challenges you face?
>

Multicore architectures are killing supercomputing because two kinds of 
parallelism are needed at the same time. A couple of years ago the computer 
architecture departments started suggesting multiple OpenMP threads in every MPI 
processes. This required a huge amount of work and never worked as expected 
(maybe compilers are to blame?).

Now the trend seems to be using MPI for processes and a superscalar compiler 
that uses all the cores and SPU for just a few kind of operations. This is not 
more useful than using vectorized versions of Blas, levels 1 and 2.

Some years ago I started experimenting with python and MPI. I coded a very 
limited MPI bindings collection for Python and launched python interpreters as 
MPI processes. Instantly thought that a JIT-accelerated, multicore-aware 
interpreter and a fast MPI-like message passing library would lower the 
complexity of using parallel computers. That would not give the highest 
performance but one has to be realistic. There's no need to distribute a VM, 
message passing is not that hard.

Maybe Java would be more suitable but I am contrary to Java from an almost 
religious point of view. I just hate it.

That's what I mean with narrowing the software gap. I've think that pypy has all 
the blocks to build this kind of interpeter, but maybe I am wrong. This is 
because I cant' really understand what pypy is ;) (maybe that's the reason why I 
find it so interesting).

>
> The PyPy project is mostly concerned with reducing the complexity
> of writing efficient dynamic language interpreters.
> PyPy's JIT and its flexible backend implementation
> should help with bridging the software gap IIUC.
> It probably makes sense to stress and elaborate
> on this point and to discuss examples in the talk.
>
> Managing the complexity of software deployment and integration
> is not PyPy's primary concern but it provides some interesting
> building blocks.  For example, i am interested in using using
> PyPy's sandboxing and virtualization capabilities to build an
> open "cloud at home" network: people would run a bare-bones
> executable, configure & control CPU, RAM, Network, Disk
> resources and access rights. On this network e.g. open source
> developers or scientists could launch programs and
> computations or start os-level virtual machines where
> non-python code runs.
>
> > In addition, most of postprocessing tools are written in matlab, an
> > interpreted language.  Running not-so-high performance tasks in a
> > workstation efficiently is sometimes as important as running a simulation
> > in a 12000 node supercomputer. It yould be nice if someone would remind
> > the audience that matlab is not the only suitable (or best) tool for that
> > job.
>
> Right, I guess that something like Numpy combined with PyPy's JIT
> would could make a killer combination.  That's work, though.

I think that you'll have to start explaining what a JIT is. See the note on the 
audience below. I've been using Python+Numpy+Scipy+... as a complete replacement 
for matlab for the last two years. I suggested all of my colleagues to do the 
same but they all believe that python is a toy language (!). I think that you 
could give a more authorized opinion on this.

> > I hope that I managed to explain the topics I'm interested in.
>
> sure, let us know what you think makes most sense for
> the audience - who will be the audience btw?

The talks will be in the school of aeronautics. Most of people in the audience 
will be engineers, mathematicians and physicists users of supercomputing without 
any academic background on Computer Science. 

Thany you so much!
-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From p.giarrusso at gmail.com  Tue Jan 13 19:04:24 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Tue, 13 Jan 2009 19:04:24 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <693bc9ab0901120805l74e95faan2d0117fe826a2021@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<a4d0c86c0901120350p296e5a2cn85c4d8634c42f6c9@mail.gmail.com>
	<693bc9ab0901120805l74e95faan2d0117fe826a2021@mail.gmail.com>
Message-ID: <a4d0c86c0901131004t72f5c94eof5610330a29ee4ea@mail.gmail.com>

On Mon, Jan 12, 2009 at 17:05, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Mon, Jan 12, 2009 at 12:50 PM, Paolo Giarrusso <p.giarrusso at gmail.com> wrote:
>> Hi, first I'd like to qualify myself as a student of Virtual Machine
>> implementations, not working (yet) on PyPy itself, and aware of some
>> HPC issues at a basic level. Still, I'd like to help pinpointing the
>> discussion.

> Sorry, but I would like to point out that this thread is about
> possible conference appearance, not general discussion about HPC.

> Things that you mentioned are interesting, but off topic and are
> possibly well known to Guillem I suppose.

I introduced them thinking they were on topic, as general ideas to
maybe discuss in the talk. Or rather, to suggest discussing facts
about them in the talk.

== MORE REFERENCES ==
Anwyay, some googling found a couple of interesting papers on
(dynamic) auto-parallelization, which are for Java.

http://portal.acm.org/citation.cfm?id=859668
http://cat.inist.fr/?aModele=afficheN&cpsidt=991008

>From the abstracts, I expect the second to be more challenging to
transport to Python. Existing static automatic parallelizing compilers
are already not satisfactory, as pointed out by Guillem, but the 1st
paper seem to suggest that a JIT can better tune the introduced
parallelism.

Anyway, this seems still research and not production-level technology.
Is the latter available?
> Paolo, please feel free to use pypy-dev as a discussion place,
> especially regarding implementing dynamic VMs, but we would be
> grateful if you stick to some guidelines:

> * Stick to the topic of current discussion.

> * Discuss one issue at a time, and not a complete flow of buzzwords.

> * Try to provide more sources/details rather than general statements

Maciej, I first want to apologize if my mail was not indeed useful,
and if I seemed to use pypy-dev as a forum.
But I was trying to discuss the topic of the conference appearance,
given that the topics (of the talk itself) are probably going to be
"Ideas for a JIT for numerical computing in Python" and "Is/Will be
Python better than alternatives, as of today/in the future?". The
"future" part is where all the "buzzwords" came from, even if more
experience could have detailed some of them. And I did want to provide
an overview, so that one can then focus on specific points.

Given the interest and resource on HPC computing, implementing in a
JIT optimizations for static compilers which are suited to HPC could
be useful and could find enough resources, if the optimizations are
worth their cost, and given somebody willing to solve the formidable
challenges I expect of this idea.

Regards
-- 
Paolo Giarrusso

From renesd at gmail.com  Tue Jan 13 23:41:47 2009
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Wed, 14 Jan 2009 09:41:47 +1100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <20090113123107.GL2219@trillke.net>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<20090113123107.GL2219@trillke.net>
Message-ID: <64ddb72c0901131441w98b1e12mfb3d872ebc2c20fd@mail.gmail.com>

On Tue, Jan 13, 2009 at 11:31 PM, holger krekel <holger at merlinux.eu> wrote:
>
> Managing the complexity of software deployment and integration
> is not PyPy's primary concern but it provides some interesting
> building blocks.  For example, i am interested in using using
> PyPy's sandboxing and virtualization capabilities to build an
> open "cloud at home" network: people would run a bare-bones
> executable, configure & control CPU, RAM, Network, Disk
> resources and access rights. On this network e.g. open source
> developers or scientists could launch programs and
> computations or start os-level virtual machines where
> non-python code runs.
>

You could call it -- Vapour-ware ?


    ... sorry!   I'll go back to lurking now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090114/03eb2957/attachment.htm 

From fijall at gmail.com  Wed Jan 14 09:49:14 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 Jan 2009 09:49:14 +0100
Subject: [pypy-dev] wroclaw 2009 sprint announcement
Message-ID: <693bc9ab0901140049t34174e7l9a059e034589615b@mail.gmail.com>

========================================================
Wroc?aw PyPy sprint 7-14th February, 2009
========================================================

The next PyPy sprint will be held in Wroc?aw, Poland. This is fully public
sprint and all newcomers are welcomed. Preceeding the sprint there
will be a talk at University of Technology in Wroc?aw held at 22nd of January.

Worth noting is that this is a perfect time and place to go cross country
skiing either before or after sprint, so if anyone wants to participate in
such activity, we can organize something.

--------
Location
--------

Wroc?aw University of Technology, Poland
Wybrze?e Wyspia?skiego 23-25, Wroc?aw
building: C-13 (picture: http://tinyurl.com/6vvzwq)
room: 3.01 (3rd floor)
opening-hours: 8-21 (but the building is opened 24h/7)
campus map: http://www.pwr.wroc.pl/upload_module/images/english/campus_map.png
city map: http://tinyurl.com/7q3tfx

---------------
Native speakers
---------------

Official language in Poland is polish. There shouldn't be any problem with
english at all (especially during hotel booking etc) but if you need any help,
here is a list of native speakers who participate in the sprint. Feel free
to contact with any of them:

 * Maciej Fija?kowski ( fijall at gmail.com )
 * Bartosz Skowron ( getxsick at gmail.com )
 * Jakub Gustak ( jgustak at gmail.com )

------------
Registration
------------

If you'd like to come, please subscribe to the `pypy-sprint mailing list`_
and drop a note about your interests and post any questions.  More
organisational information will be sent to that list.

Please register by adding yourself on the following list (via svn):

  http://codespeak.net/svn/pypy/extradoc/sprintinfo/wroclaw2009/people.txt

or announce yourself on the pypy-sprint mailing list if you do not
yet have check-in rights:

  http://codespeak.net/mailman/listinfo/pypy-sprint

-------
Flights
-------

You can either fly directly to Wroc?aw, with many international connections
(like one to Dortmund and one to Duesseldorf). Details are available on
airport's webpage: http://www.airport.wroclaw.pl/en/. You can also fly to:
Warsaw (5h by train), Berlin (6h by train), Poznan (2h by train),
Katowice (3h by train) or Krak?w (4h by train).

--------------------------------------
Preparation (if you feel it is needed)
--------------------------------------

* read the `getting-started`_ pages on http://codespeak.net/pypy

* for inspiration, overview and technical status you are welcome to
  read `the technical reports available and other relevant documentation`_

* please direct any technical and/or development oriented questions to
  pypy-dev at codespeak.net and any sprint organizing/logistical
  questions to pypy-sprint at codespeak.net

We are looking forward to meet you at the Wroc?aw PyPy sprint!

The PyPy team

.. See also ..

.. _getting-started:
http://codespeak.net/pypy/dist/pypy/doc/getting-started.html
.. _`pypy-sprint mailing list`:
http://codespeak.net/mailman/listinfo/pypy-sprint
.. _`the technical reports available and other relevant
documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html

From fijall at gmail.com  Wed Jan 14 10:10:36 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 Jan 2009 10:10:36 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901131617.54395.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<20090113123107.GL2219@trillke.net>
	<200901131617.54395.guillem@torroja.dmt.upm.es>
Message-ID: <693bc9ab0901140110l73656338na16626271863ab5d@mail.gmail.com>

Hi Guillem

> I think that you'll have to start explaining what a JIT is. See the note on the
> audience below. I've been using Python+Numpy+Scipy+... as a complete replacement
> for matlab for the last two years. I suggested all of my colleagues to do the
> same but they all believe that python is a toy language (!). I think that you
> could give a more authorized opinion on this.
>

The JIT is a just-in-time compiler that compiles some intermediate
representation (for example python bytecode), to native code (for
example x86 assembler). The only existing JIT for python, psyco
(http://psyco.sourceforge.net) also massively specializes operations,
which means it gives much greater speedups than just getting read of
bytecode dispatch. It also gets read of type dispatching. For simple
loops with integer operations it can give speedups up to non-optimized
C (gcc -O0). PyPy is in some sense a followup project to psyco, which
should give you much larger benefits at some point in the future.

The exact gap that you might be interested in is the difference of
performance between numpy operations (which are rather fast, since
they're calling to years old fortran code) and a situation where you
need to for example iterate over the whole array and do some
computations in python (which is deadly slow).

I cannot really judge how python compares to matlab in terms of
availability of mathematic routines (scipy + numpy), but it's surely
nicer language and if it's fast enough, it could be a real win.

Now regarding pypy, the status is that we don't even have a numpy
bindings. And while we're surely working a lot on JIT these days, it's
still not ready for production use. So speaking about what we could
possibly do with jit & numpy is a bit speculative (but just a bit,
it's rather obvious how it could work, it's just not there yet).

Well, regarding toyness of a language, it's really hard to discuss
with such vague argument. If you provide us a reasoning why python is
a toy language we can try to provide you with some argumentation :)
>From my POV I can say that I wouldn't write a serious project size of
pypy in a toy language.

Cheers,
Maciej Fijalkowski

From holger at merlinux.eu  Wed Jan 14 10:12:04 2009
From: holger at merlinux.eu (holger krekel)
Date: Wed, 14 Jan 2009 10:12:04 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <64ddb72c0901131441w98b1e12mfb3d872ebc2c20fd@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<20090109183421.GA2219@trillke.net>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<20090113123107.GL2219@trillke.net>
	<64ddb72c0901131441w98b1e12mfb3d872ebc2c20fd@mail.gmail.com>
Message-ID: <20090114091204.GM2219@trillke.net>

Hi Rene,

On Wed, Jan 14, 2009 at 09:41 +1100, Ren? Dudfield wrote:
> On Tue, Jan 13, 2009 at 11:31 PM, holger krekel <holger at merlinux.eu> wrote:
> >
> > Managing the complexity of software deployment and integration
> > is not PyPy's primary concern but it provides some interesting
> > building blocks.  For example, i am interested in using using
> > PyPy's sandboxing and virtualization capabilities to build an
> > open "cloud at home" network: people would run a bare-bones
> > executable, configure & control CPU, RAM, Network, Disk
> > resources and access rights. On this network e.g. open source
> > developers or scientists could launch programs and
> > computations or start os-level virtual machines where
> > non-python code runs.
> 
> You could call it -- Vapour-ware ?

I don't think that stating my interests amounts to pre-announcing
a release of non-existent software.  

Regarding this interest I am actually looking for like-minded
developers because i find it more fun to tackle this in a group. 

cheers,
holger

From holger at merlinux.eu  Wed Jan 14 10:44:29 2009
From: holger at merlinux.eu (holger krekel)
Date: Wed, 14 Jan 2009 10:44:29 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901131617.54395.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901121210.50200.guillem@torroja.dmt.upm.es>
	<20090113123107.GL2219@trillke.net>
	<200901131617.54395.guillem@torroja.dmt.upm.es>
Message-ID: <20090114094429.GN2219@trillke.net>

Hi Guillem, 

On Tue, Jan 13, 2009 at 16:17 +0100, Guillem Borrell i Nogueras wrote:
> Hi!
> 
> El Tuesday 13 January 2009 13:31:07 holger krekel escribi?:
> 
> >
> > Could you give examples on the concrete challenges you face?
> >
> 
> Multicore architectures are killing supercomputing because two kinds of 
> parallelism are needed at the same time. A couple of years ago the computer 
> architecture departments started suggesting multiple OpenMP threads in every MPI 
> processes. This required a huge amount of work and never worked as expected 
> (maybe compilers are to blame?).
> 
> Now the trend seems to be using MPI for processes and a superscalar compiler 
> that uses all the cores and SPU for just a few kind of operations. This is not 
> more useful than using vectorized versions of Blas, levels 1 and 2.
> 
> Some years ago I started experimenting with python and MPI. I coded a very 
> limited MPI bindings collection for Python and launched python interpreters as 
> MPI processes. Instantly thought that a JIT-accelerated, multicore-aware 
> interpreter and a fast MPI-like message passing library would lower the 
> complexity of using parallel computers. That would not give the highest 
> performance but one has to be realistic. There's no need to distribute a VM, 
> message passing is not that hard.

For that scenario PyPy is currently missing the JIT, multicore-awareness and bindings 
to numeric libraries.  
 
> > > In addition, most of postprocessing tools are written in matlab, an
> > > interpreted language.  Running not-so-high performance tasks in a
> > > workstation efficiently is sometimes as important as running a simulation
> > > in a 12000 node supercomputer. It yould be nice if someone would remind
> > > the audience that matlab is not the only suitable (or best) tool for that
> > > job.
> >
> > Right, I guess that something like Numpy combined with PyPy's JIT
> > would could make a killer combination.  That's work, though.
> 
> I think that you'll have to start explaining what a JIT is. See the note on the 
> audience below. I've been using Python+Numpy+Scipy+... as a complete replacement 
> for matlab for the last two years. I suggested all of my colleagues to do the 
> same but they all believe that python is a toy language (!). I think that you 
> could give a more authorized opinion on this.

If that is your main idea - and considering the audience - 
i'd rather suggest to get someone authoritative from the
Numpy/Scipy community - they did two dedicated conferences
last year, the November issue of the Python Magazine was about
Python in scientific environments.   So it shouldn't be hard
to get someone who authentically confirms that Python is
a seriously used language for scientific computing.  

Of course, we'd be happy to discuss and incorporate a "PyPy"
perspective in such a talk.  

cheers,
holger

> > > I hope that I managed to explain the topics I'm interested in.
> >
> > sure, let us know what you think makes most sense for
> > the audience - who will be the audience btw?
> 
> The talks will be in the school of aeronautics. Most of people in the audience 
> will be engineers, mathematicians and physicists users of supercomputing without 
> any academic background on Computer Science. 
> 
> Thany you so much!
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mec?nica de Fluidos Computacional
> Escuela T?cnica Superior de Ingenieros Aeron?uticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From guillem at torroja.dmt.upm.es  Wed Jan 14 11:46:06 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Wed, 14 Jan 2009 11:46:06 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <693bc9ab0901140110l73656338na16626271863ab5d@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901131617.54395.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140110l73656338na16626271863ab5d@mail.gmail.com>
Message-ID: <200901141146.07064.guillem@torroja.dmt.upm.es>

Hi!

I'll try to sort things out a bit.

I think that there are two separate but related topics.

* What can dynamic languages can offer to massive parallel computing in. Ease of 
use gain vs. performance loss. How to handle the multicore problem. How pypy and 
python could help. Total vaporware (sic.) but the trends and ideas are important 
too. 

* Interpreted languages in workstations. Interpreters and VM are becoming 
smarter (JIT or why using loops is not a crime anymore). Ease of use gain vs. 
performance loss (again). What can happen in the future and the pypy point of 
view.

(The audience will not have any knowledge about VM implementation.  They just 
know that from Matlab v6.5 simple loops are faster by some kind of unexplainable 
reason. They also believe that fortran and MPI is the only path to parallel 
computing.)

I asked you to give such a talk because, as pypy is a general VM implementation 
you have a wider vision of such things. I know that one could organize hundreds 
of congresses on this but the point here is to lighten the audience's interest. 
I know that *today* pypy lacks lots of technologies to be the definitive answer 
but I never asked for a solution to any problem.

I am more interested in what you would do than in what you have done (that is 
already impressive). The goal of the supercomputing day is to talk about 
interesting technologies, no matter if they are not ready yet.

Numpy support and bindings to numerical libraries is a problem I'd like to 
tackle myself in the future but I'm afraid I don't quite understand how to 
extend pypy.  Maybe it's just a matter of time or that I'm dumb. However I'm 
sure you could help me to understand lots of things if you come to Madrid in 
April.

cheers

PS: I'm not trying to tell you what to say in the talk. I'm just trying to 
explain what could be more interesting to the audience. You are free to 
completely ignore anything I suggest.

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From fijall at gmail.com  Wed Jan 14 11:59:18 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 Jan 2009 11:59:18 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901141146.07064.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901131617.54395.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140110l73656338na16626271863ab5d@mail.gmail.com>
	<200901141146.07064.guillem@torroja.dmt.upm.es>
Message-ID: <693bc9ab0901140259k6a9a6ffds275d36321f6fc84@mail.gmail.com>

> Numpy support and bindings to numerical libraries is a problem I'd like to
> tackle myself in the future but I'm afraid I don't quite understand how to
> extend pypy.  Maybe it's just a matter of time or that I'm dumb. However I'm
> sure you could help me to understand lots of things if you come to Madrid in
> April.
>

Confusion. Is it April or August? (you said August in your first mail)

Adding numpy to pypy shouldn't be that hard. It's work though and it
requires a bit of knowledge that is not documented anywhere. We're up
to help though.

Cheers,
fijal

From guillem at torroja.dmt.upm.es  Wed Jan 14 12:25:56 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Wed, 14 Jan 2009 12:25:56 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <693bc9ab0901140259k6a9a6ffds275d36321f6fc84@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901141146.07064.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140259k6a9a6ffds275d36321f6fc84@mail.gmail.com>
Message-ID: <200901141225.56173.guillem@torroja.dmt.upm.es>

Ooops April the 1st

sorry

El Wednesday 14 January 2009 11:59:18 Maciej Fijalkowski escribi?:
> > Numpy support and bindings to numerical libraries is a problem I'd like
> > to tackle myself in the future but I'm afraid I don't quite understand
> > how to extend pypy.  Maybe it's just a matter of time or that I'm dumb.
> > However I'm sure you could help me to understand lots of things if you
> > come to Madrid in April.
>
> Confusion. Is it April or August? (you said August in your first mail)
>
> Adding numpy to pypy shouldn't be that hard. It's work though and it
> requires a bit of knowledge that is not documented anywhere. We're up
> to help though.
>
> Cheers,
> fijal

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From fijall at gmail.com  Wed Jan 14 13:59:31 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 Jan 2009 13:59:31 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901141225.56173.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901141146.07064.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140259k6a9a6ffds275d36321f6fc84@mail.gmail.com>
	<200901141225.56173.guillem@torroja.dmt.upm.es>
Message-ID: <693bc9ab0901140459i66accf43ja116f564b682333d@mail.gmail.com>

On Wed, Jan 14, 2009 at 12:25 PM, Guillem Borrell i Nogueras
<guillem at torroja.dmt.upm.es> wrote:
> Ooops April the 1st
>
> sorry
>

Then, it's completely out of scope, at least for some of us. Pycon US
(http://us.pycon.org) ends 2nd of April in Chicago.

Cheers,
fijal

From bea at changemaker.nu  Wed Jan 14 14:10:43 2009
From: bea at changemaker.nu (Bea During)
Date: Wed, 14 Jan 2009 14:10:43 +0100
Subject: [pypy-dev] wroclaw 2009 sprint announcement
In-Reply-To: <693bc9ab0901140049t34174e7l9a059e034589615b@mail.gmail.com>
References: <693bc9ab0901140049t34174e7l9a059e034589615b@mail.gmail.com>
Message-ID: <496DE453.2080605@changemaker.nu>

Hi all

Just a tip when booking your flights - be prepared for the
flight search/booking instances to call Wroclaw:

- Strachowice
- Breslavia (german name for the city is Breslau)

Thanks Maciek for the help!

Cheers

Bea

Maciej Fijalkowski wrote:
> ========================================================
> Wroc?aw PyPy sprint 7-14th February, 2009
> ========================================================
>
> The next PyPy sprint will be held in Wroc?aw, Poland. This is fully public
> sprint and all newcomers are welcomed. Preceeding the sprint there
> will be a talk at University of Technology in Wroc?aw held at 22nd of January.
>
> Worth noting is that this is a perfect time and place to go cross country
> skiing either before or after sprint, so if anyone wants to participate in
> such activity, we can organize something.
>
> --------
> Location
> --------
>
> Wroc?aw University of Technology, Poland
> Wybrze?e Wyspia?skiego 23-25, Wroc?aw
> building: C-13 (picture: http://tinyurl.com/6vvzwq)
> room: 3.01 (3rd floor)
> opening-hours: 8-21 (but the building is opened 24h/7)
> campus map: http://www.pwr.wroc.pl/upload_module/images/english/campus_map.png
> city map: http://tinyurl.com/7q3tfx
>
> ---------------
> Native speakers
> ---------------
>
> Official language in Poland is polish. There shouldn't be any problem with
> english at all (especially during hotel booking etc) but if you need any help,
> here is a list of native speakers who participate in the sprint. Feel free
> to contact with any of them:
>
>  * Maciej Fija?kowski ( fijall at gmail.com )
>  * Bartosz Skowron ( getxsick at gmail.com )
>  * Jakub Gustak ( jgustak at gmail.com )
>
> ------------
> Registration
> ------------
>
> If you'd like to come, please subscribe to the `pypy-sprint mailing list`_
> and drop a note about your interests and post any questions.  More
> organisational information will be sent to that list.
>
> Please register by adding yourself on the following list (via svn):
>
>   http://codespeak.net/svn/pypy/extradoc/sprintinfo/wroclaw2009/people.txt
>
> or announce yourself on the pypy-sprint mailing list if you do not
> yet have check-in rights:
>
>   http://codespeak.net/mailman/listinfo/pypy-sprint
>
> -------
> Flights
> -------
>
> You can either fly directly to Wroc?aw, with many international connections
> (like one to Dortmund and one to Duesseldorf). Details are available on
> airport's webpage: http://www.airport.wroclaw.pl/en/. You can also fly to:
> Warsaw (5h by train), Berlin (6h by train), Poznan (2h by train),
> Katowice (3h by train) or Krak?w (4h by train).
>
> --------------------------------------
> Preparation (if you feel it is needed)
> --------------------------------------
>
> * read the `getting-started`_ pages on http://codespeak.net/pypy
>
> * for inspiration, overview and technical status you are welcome to
>   read `the technical reports available and other relevant documentation`_
>
> * please direct any technical and/or development oriented questions to
>   pypy-dev at codespeak.net and any sprint organizing/logistical
>   questions to pypy-sprint at codespeak.net
>
> We are looking forward to meet you at the Wroc?aw PyPy sprint!
>
> The PyPy team
>
> .. See also ..
>
> .. _getting-started:
> http://codespeak.net/pypy/dist/pypy/doc/getting-started.html
> .. _`pypy-sprint mailing list`:
> http://codespeak.net/mailman/listinfo/pypy-sprint
> .. _`the technical reports available and other relevant
> documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev


From guillem at torroja.dmt.upm.es  Wed Jan 14 15:22:54 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Wed, 14 Jan 2009 15:22:54 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <693bc9ab0901140459i66accf43ja116f564b682333d@mail.gmail.com>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901141225.56173.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140459i66accf43ja116f564b682333d@mail.gmail.com>
Message-ID: <200901141522.54790.guillem@torroja.dmt.upm.es>


Ok, I'll try to move it to a later day.

guillem

El Wednesday 14 January 2009 13:59:31 Maciej Fijalkowski escribi?:
> On Wed, Jan 14, 2009 at 12:25 PM, Guillem Borrell i Nogueras
>
> <guillem at torroja.dmt.upm.es> wrote:
> > Ooops April the 1st
> >
> > sorry
>
> Then, it's completely out of scope, at least for some of us. Pycon US
> (http://us.pycon.org) ends 2nd of April in Chicago.
>
> Cheers,
> fijal

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From guillem at torroja.dmt.upm.es  Wed Jan 14 16:23:22 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Wed, 14 Jan 2009 16:23:22 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901141522.54790.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140459i66accf43ja116f564b682333d@mail.gmail.com>
	<200901141522.54790.guillem@torroja.dmt.upm.es>
Message-ID: <200901141623.22706.guillem@torroja.dmt.upm.es>

I have a new date

What about April 14th or 15th?

guillem

El Wednesday 14 January 2009 15:22:54 Guillem Borrell i Nogueras escribi?:
> Ok, I'll try to move it to a later day.
>
> guillem
>
> El Wednesday 14 January 2009 13:59:31 Maciej Fijalkowski escribi?:
> > On Wed, Jan 14, 2009 at 12:25 PM, Guillem Borrell i Nogueras
> >
> > <guillem at torroja.dmt.upm.es> wrote:
> > > Ooops April the 1st
> > >
> > > sorry
> >
> > Then, it's completely out of scope, at least for some of us. Pycon US
> > (http://us.pycon.org) ends 2nd of April in Chicago.
> >
> > Cheers,
> > fijal

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From hakan at debian.org  Thu Jan 15 13:26:14 2009
From: hakan at debian.org (Hakan Ardo)
Date: Thu, 15 Jan 2009 13:26:14 +0100
Subject: [pypy-dev] Index checking for setitem
Message-ID: <e7e642fc0901150426v2277bf2aqa3ff4e14c61972e2@mail.gmail.com>

Hi,
getitem operations gets converted into getitem_idx if they are
surrounded by try: except IndexError. I'v duplicated that
implementation to make setitem work in the same way. A patch is
available here:

  http://hakan.ardoe.net/pypy/setitem.patch

Also, I've updated my special_methods implementation to map
getitem_idx to the special method __getitem_idx__ and likewise for
setitem.

Would it be possible to build support for constructs such as:

  def get(lst,i):
    return lst[i]

  try:
    get([1,2,3],10)
  excpt IndexError:
    print "OK"

I suppose the convention from getitem to getitem_idx would have to
happen during annotation as different callsites are discovered and
each conversion would have to force a reflow of the converted
operation?

Thanx!

-- 
H?kan Ard?

From holger at merlinux.eu  Thu Jan 15 14:54:54 2009
From: holger at merlinux.eu (holger krekel)
Date: Thu, 15 Jan 2009 14:54:54 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <200901141623.22706.guillem@torroja.dmt.upm.es>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<693bc9ab0901140459i66accf43ja116f564b682333d@mail.gmail.com>
	<200901141522.54790.guillem@torroja.dmt.upm.es>
	<200901141623.22706.guillem@torroja.dmt.upm.es>
Message-ID: <20090115135454.GP2219@trillke.net>

Hi Guillem,

On Wed, Jan 14, 2009 at 16:23 +0100, Guillem Borrell i Nogueras wrote:
> I have a new date
>
> What about April 14th or 15th?

Maciej and me discussed a bit.  April in general does not fit
well, unfortunately.  Apart from Pycon we'll be away and busy 
with pypy release works and moving the JIT forward.

In general we feel it's a bit early to talk about PyPy's
potential regarding super-computing, particularly to people
not primarily interested in compilers or Python.  

If you happen to go for such a day second half 2009
or in 2010 we'd be very happy to attend.

cheers & many thanks for your interest!
holger

> El Wednesday 14 January 2009 15:22:54 Guillem Borrell i Nogueras escribi?:
> > Ok, I'll try to move it to a later day.
> >
> > guillem
> >
> > El Wednesday 14 January 2009 13:59:31 Maciej Fijalkowski escribi?:
> > > On Wed, Jan 14, 2009 at 12:25 PM, Guillem Borrell i Nogueras
> > >
> > > <guillem at torroja.dmt.upm.es> wrote:
> > > > Ooops April the 1st
> > > >
> > > > sorry
> > >
> > > Then, it's completely out of scope, at least for some of us. Pycon US
> > > (http://us.pycon.org) ends 2nd of April in Chicago.
> > >
> > > Cheers,
> > > fijal
> 
> -- 
> guillem
> 
> Guillem Borrell i Nogueras
> Laboratorio de Mec?nica de Fluidos Computacional
> Escuela T?cnica Superior de Ingenieros Aeron?uticos
> guillem at torroja.dmt.upm.es
> Web: http://torroja.dmt.upm.es/guillem/blog
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From guillem at torroja.dmt.upm.es  Thu Jan 15 17:28:59 2009
From: guillem at torroja.dmt.upm.es (Guillem Borrell i Nogueras)
Date: Thu, 15 Jan 2009 17:28:59 +0100
Subject: [pypy-dev] Talk in the Supercomputing Day, Madrid
In-Reply-To: <20090115135454.GP2219@trillke.net>
References: <200901091054.51758.guillem@torroja.dmt.upm.es>
	<200901141623.22706.guillem@torroja.dmt.upm.es>
	<20090115135454.GP2219@trillke.net>
Message-ID: <200901151728.59724.guillem@torroja.dmt.upm.es>

Hi Holger!

I can understand your decision. We plan to organize this supercomputing day 
yearly, this means that probably you will recieve the same invitation for the 
2010 event.  I'm afraid that it will be in April too, hopefully not the same day 
as Pycon US.

I plan to attend one pypy-sprint this year so we can discuss this things in 
person.

Thank you very much for your time!

Yours

guillem

El Thursday 15 January 2009 14:54:54 holger krekel escribi?:
> Hi Guillem,
>
> On Wed, Jan 14, 2009 at 16:23 +0100, Guillem Borrell i Nogueras wrote:
> > I have a new date
> >
> > What about April 14th or 15th?
>
> Maciej and me discussed a bit.  April in general does not fit
> well, unfortunately.  Apart from Pycon we'll be away and busy
> with pypy release works and moving the JIT forward.
>
> In general we feel it's a bit early to talk about PyPy's
> potential regarding super-computing, particularly to people
> not primarily interested in compilers or Python.
>
> If you happen to go for such a day second half 2009
> or in 2010 we'd be very happy to attend.
>
> cheers & many thanks for your interest!
> holger
>
> > El Wednesday 14 January 2009 15:22:54 Guillem Borrell i Nogueras escribi?:
> > > Ok, I'll try to move it to a later day.
> > >
> > > guillem
> > >
> > > El Wednesday 14 January 2009 13:59:31 Maciej Fijalkowski escribi?:
> > > > On Wed, Jan 14, 2009 at 12:25 PM, Guillem Borrell i Nogueras
> > > >
> > > > <guillem at torroja.dmt.upm.es> wrote:
> > > > > Ooops April the 1st
> > > > >
> > > > > sorry
> > > >
> > > > Then, it's completely out of scope, at least for some of us. Pycon US
> > > > (http://us.pycon.org) ends 2nd of April in Chicago.
> > > >
> > > > Cheers,
> > > > fijal
> >
> > --
> > guillem
> >
> > Guillem Borrell i Nogueras
> > Laboratorio de Mec?nica de Fluidos Computacional
> > Escuela T?cnica Superior de Ingenieros Aeron?uticos
> > guillem at torroja.dmt.upm.es
> > Web: http://torroja.dmt.upm.es/guillem/blog

-- 
guillem

Guillem Borrell i Nogueras
Laboratorio de Mec?nica de Fluidos Computacional
Escuela T?cnica Superior de Ingenieros Aeron?uticos
guillem at torroja.dmt.upm.es
Web: http://torroja.dmt.upm.es/guillem/blog

From getxsick at gmail.com  Thu Jan 15 19:27:45 2009
From: getxsick at gmail.com (Bartosz SKOWRON)
Date: Thu, 15 Jan 2009 19:27:45 +0100
Subject: [pypy-dev] wroclaw 2009 sprint announcement
In-Reply-To: <693bc9ab0901140049t34174e7l9a059e034589615b@mail.gmail.com>
References: <693bc9ab0901140049t34174e7l9a059e034589615b@mail.gmail.com>
Message-ID: <77887e110901151027o60e8eb6fl249228ed94b78d06@mail.gmail.com>

On Wed, Jan 14, 2009 at 9:49 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> ========================================================
> Wroc?aw PyPy sprint 7-14th February, 2009
> ========================================================

I added short info about accomodation.

Index: announcement.txt
===================================================================
--- announcement.txt	(wersja 60905)
+++ announcement.txt	(kopia robocza)
@@ -62,6 +62,20 @@
 Warsaw (5h by train), Berlin (6h by train), Poznan (2h by train),
 Katowice (3h by train) or Krak?w (4h by train).

+------------
+Accomodation
+------------
+
+Unfortunately, there aren't any hotels in really close area to sprint venue.
+There is a lot of hotels in city centre area and it's like 10-20 mins far away.
+List of possible accomodations:
+
+* http://www.wroclaw.pl/m6874/p7318.aspx (hotels)
+* http://www.wroclaw.pl/m6875/p7137.aspx (hostels)
+
+You can simple check the location at http://maps.google.com
+If you have any additional questions or advice, drop a line to Bartosz Skowron.
+
 --------------------------------------
 Preparation (if you feel it is needed)
 --------------------------------------

From pedronis at openend.se  Sun Jan 18 14:16:37 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Sun, 18 Jan 2009 14:16:37 +0100
Subject: [pypy-dev] forcing builds Re: another issue with buildbot
In-Reply-To: <693bc9ab0901180241n112cf28ck47883cb84c83e9dc@mail.gmail.com>
References: <693bc9ab0901180241n112cf28ck47883cb84c83e9dc@mail.gmail.com>
Message-ID: <49732BB5.4070405@openend.se>

Maciej Fijalkowski wrote:
> I found out that I'm unable to force run by hand. It checks out the
> revision from last build. Do you know how to bump it?
>   
you need to use the rebuild from the builder page, the rebuild from the 
build page will reuse the got_revision


From fijall at gmail.com  Sun Jan 18 15:17:10 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 18 Jan 2009 15:17:10 +0100
Subject: [pypy-dev] forcing builds Re: another issue with buildbot
In-Reply-To: <49732BB5.4070405@openend.se>
References: <693bc9ab0901180241n112cf28ck47883cb84c83e9dc@mail.gmail.com>
	<49732BB5.4070405@openend.se>
Message-ID: <693bc9ab0901180617x142fe532mce6b5eaa093013c4@mail.gmail.com>

On Sun, Jan 18, 2009 at 2:16 PM, Samuele Pedroni <pedronis at openend.se> wrote:
> Maciej Fijalkowski wrote:
>>
>> I found out that I'm unable to force run by hand. It checks out the
>> revision from last build. Do you know how to bump it?
>>
>
> you need to use the rebuild from the builder page, the rebuild from the
> build page will reuse the got_revision
>
>

I did. The thing is it does not use svn trunk, but instead uses
got_revision. How to bump got_revision?

From pedronis at openend.se  Sun Jan 18 16:15:08 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Sun, 18 Jan 2009 16:15:08 +0100
Subject: [pypy-dev] forcing builds Re: another issue with buildbot
In-Reply-To: <49732BB5.4070405@openend.se>
References: <693bc9ab0901180241n112cf28ck47883cb84c83e9dc@mail.gmail.com>
	<49732BB5.4070405@openend.se>
Message-ID: <4973477C.20905@openend.se>

Samuele Pedroni wrote:
> Maciej Fijalkowski wrote:
>   
>> I found out that I'm unable to force run by hand. It checks out the
>> revision from last build. Do you know how to bump it?
>>   
>>     
> you need to use the rebuild from the builder page, the rebuild from the 
> build page will reuse the got_revision
>   
to be clear (for anybody interested/following this):

http://codespeak.net:8099/builders/pypy-c-app-level-linux-x86-32

is a Builder page and the right one.

http://codespeak.net:8099/builders/pypy-c-app-level-linux-x86-32/builds/11

is a Build page and not so useful for forcing runs unless one is 
debugging a failure that is about the buildbot environment and not the 
code itself.

From nvetoshkin at naumen.ru  Tue Jan 20 19:55:49 2009
From: nvetoshkin at naumen.ru (=?UTF-8?B?0JLQtdGC0L7RiNC60LjQvSDQndC40LrQuNGC0LA=?=)
Date: Tue, 20 Jan 2009 23:55:49 +0500
Subject: [pypy-dev] Translate failes =(
Message-ID: <49761E35.3020307@naumen.ru>

Hi, all!

I'm trying to translate PyPy on Windows Vista x64 with Python2.4 32 bit 
and Visual Studio 2008 Express Edition. 32 bit - cause Express Edition 
isn't shipped with x64 compiler.
Translation failes: http://pastebin.com/m70de1376.
Fixing Visual Studio version recognition ( warning about KeyError 
VS71COMNTOOLS) doesn't help, translation failes with the same error.
Any suggestions?

P.S. In answers, please, put my mail in copy, cause I'm subscribed to 
digest =)

From nvetoshkin at naumen.ru  Tue Jan 20 22:13:51 2009
From: nvetoshkin at naumen.ru (=?UTF-8?B?0JLQtdGC0L7RiNC60LjQvSDQndC40LrQuNGC0LA=?=)
Date: Wed, 21 Jan 2009 02:13:51 +0500
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49761E35.3020307@naumen.ru>
References: <49761E35.3020307@naumen.ru>
Message-ID: <49763E8F.4010304@naumen.ru>

Thanks to fijal, I've installed ctypes and now I have MessageBox saying 
that system can't find "MSVCR90.DLL". After putting this library into 
current directory, I have this error:
---------------------------
Microsoft Visual C++ Runtime Library
---------------------------
Runtime Error!

Program: C:\Python24\python.exe

R6034

An application has made an attempt to load the C runtime library 
incorrectly.
Please contact the application's support team for more information.


---------------------------
OK
---------------------------

Google found this: http://msdn.microsoft.com/en-us/library/ms235560.aspx
As I understand - we need to include manifest into resulting binary or 
perhaps link externmod_2.dll statically.
The resulting stacktrace: http://pastebin.com/m6965b0f7
Please help =)

20.01.2009 23:55, ???????? ?????? ?????:
> Hi, all!
>
> I'm trying to translate PyPy on Windows Vista x64 with Python2.4 32 bit
> and Visual Studio 2008 Express Edition. 32 bit - cause Express Edition
> isn't shipped with x64 compiler.
> Translation failes: http://pastebin.com/m70de1376.
> Fixing Visual Studio version recognition ( warning about KeyError
> VS71COMNTOOLS) doesn't help, translation failes with the same error.
> Any suggestions?
>
> P.S. In answers, please, put my mail in copy, cause I'm subscribed to
> digest =)

From fijall at gmail.com  Tue Jan 20 22:23:38 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 20 Jan 2009 22:23:38 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49763E8F.4010304@naumen.ru>
References: <49761E35.3020307@naumen.ru> <49763E8F.4010304@naumen.ru>
Message-ID: <693bc9ab0901201323i53626928j38af1020536eca16@mail.gmail.com>

> Thanks to fijal, I've installed ctypes and now I have MessageBox saying
> that system can't find "MSVCR90.DLL". After putting this library into
> current directory, I have this error:
> ---------------------------
> Microsoft Visual C++ Runtime Library
> ---------------------------
> Runtime Error!
>
> Program: C:\Python24\python.exe
>
> R6034
>
> An application has made an attempt to load the C runtime library
> incorrectly.
> Please contact the application's support team for more information.
>

Heh :)

I suppose that's us, but I have no clue, sorry ;-)

Cheers,
fijal

From lutz_p at gmx.net  Tue Jan 20 22:56:05 2009
From: lutz_p at gmx.net (Lutz Paelike)
Date: Tue, 20 Jan 2009 22:56:05 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49763E8F.4010304@naumen.ru>
References: <49761E35.3020307@naumen.ru> <49763E8F.4010304@naumen.ru>
Message-ID: <B6537249-DE6D-4686-B022-49F56A7312D7@gmx.net>

I have a guess how that could happen.

The reason is probably that two different versions of  MSVCR dll are  
loaded into the process that
result in a name clash of the symbols provided by the library.

Now *why* this happens is not entirely clear but i have also a wild  
guess here.

The standard Python2.4 distribution is using MSVCR71.dll an older  
version (7.1)
that is used when compiled with e.g. Visual Studio 2005

If you use python all installed  modules are  linked against  the same  
version of the MSVCR dll
and thus it works without a problem (provided you have MSVCR71.dll on  
your system).

Now the problem arises if you compile an extension yourself with a  
different compiler version
which links your compiled extension against a different  version of  
the library (in this case MSVCR90.dll = Version 9.0).

Solution:
1. You should use the same compiler version (VS 2005) as the one used  
of your python installation
or
2. Compile Python and all modules you use yourself from source and use  
that instead of an precompiled distribution

Another variable in this setup might be ctypes.
If ctypes loads the MSVCR library without providing the exact  
requested version it might
determine the best (newest) version itself and that might not be right  
one.

Hope this helps.
Cheers,

Lutz


Am 20.01.2009 um 22:13 schrieb ???????? ??????:

> Thanks to fijal, I've installed ctypes and now I have MessageBox  
> saying
> that system can't find "MSVCR90.DLL". After putting this library into
> current directory, I have this error:
> ---------------------------
> Microsoft Visual C++ Runtime Library
> ---------------------------
> Runtime Error!
>
> Program: C:\Python24\python.exe
>
> R6034
>
> An application has made an attempt to load the C runtime library
> incorrectly.
> Please contact the application's support team for more information.
>
>
> ---------------------------
> OK
> ---------------------------
>
> Google found this: http://msdn.microsoft.com/en-us/library/ms235560.aspx
> As I understand - we need to include manifest into resulting binary or
> perhaps link externmod_2.dll statically.
> The resulting stacktrace: http://pastebin.com/m6965b0f7
> Please help =)
>
> 20.01.2009 23:55, ???????? ?????? ?????:
>> Hi, all!
>>
>> I'm trying to translate PyPy on Windows Vista x64 with Python2.4 32  
>> bit
>> and Visual Studio 2008 Express Edition. 32 bit - cause Express  
>> Edition
>> isn't shipped with x64 compiler.
>> Translation failes: http://pastebin.com/m70de1376.
>> Fixing Visual Studio version recognition ( warning about KeyError
>> VS71COMNTOOLS) doesn't help, translation failes with the same error.
>> Any suggestions?
>>
>> P.S. In answers, please, put my mail in copy, cause I'm subscribed to
>> digest =)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev


From lutz_p at gmx.net  Tue Jan 20 22:57:30 2009
From: lutz_p at gmx.net (Lutz Paelike)
Date: Tue, 20 Jan 2009 22:57:30 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49763E8F.4010304@naumen.ru>
References: <49761E35.3020307@naumen.ru> <49763E8F.4010304@naumen.ru>
Message-ID: <DC3EFDC5-F48E-41CC-B49A-A9B68CECC29D@gmx.net>

I have a guess how that could happen.

The reason is probably that two different versions of  MSVCR dll are  
loaded into the process that
result in a name clash of the symbols provided by the library.

Now *why* this happens is not entirely clear but i have also a wild  
guess here.

The standard Python2.4 distribution is using MSVCR71.dll an older  
version (7.1)
that is used when compiled with e.g. Visual Studio 2005

If you use python all installed  modules are  linked against  the same  
version of the MSVCR dll
and thus it works without a problem (provided you have MSVCR71.dll on  
your system).

Now the problem arises if you compile an extension yourself with a  
different compiler version
which links your compiled extension against a different  version of  
the library (in this case MSVCR90.dll = Version 9.0).

Solution:
1. You should use the same compiler version (VS 2005) as the one used  
of your python installation
or
2. Compile Python and all modules you use yourself from source and use  
that instead of an precompiled distribution

Another variable in this setup might be ctypes.
If ctypes loads the MSVCR library without providing the exact  
requested version it might
determine the best (newest) version itself and that might not be right  
one.

Hope this helps.
Cheers,

Lutz


Am 20.01.2009 um 22:13 schrieb ???????? ??????:

> Thanks to fijal, I've installed ctypes and now I have MessageBox  
> saying
> that system can't find "MSVCR90.DLL". After putting this library into
> current directory, I have this error:
> ---------------------------
> Microsoft Visual C++ Runtime Library
> ---------------------------
> Runtime Error!
>
> Program: C:\Python24\python.exe
>
> R6034
>
> An application has made an attempt to load the C runtime library
> incorrectly.
> Please contact the application's support team for more information.
>
>
> ---------------------------
> OK
> ---------------------------
>
> Google found this: http://msdn.microsoft.com/en-us/library/ms235560.aspx
> As I understand - we need to include manifest into resulting binary or
> perhaps link externmod_2.dll statically.
> The resulting stacktrace: http://pastebin.com/m6965b0f7
> Please help =)
>
> 20.01.2009 23:55, ???????? ?????? ?????:
>> Hi, all!
>>
>> I'm trying to translate PyPy on Windows Vista x64 with Python2.4 32  
>> bit
>> and Visual Studio 2008 Express Edition. 32 bit - cause Express  
>> Edition
>> isn't shipped with x64 compiler.
>> Translation failes: http://pastebin.com/m70de1376.
>> Fixing Visual Studio version recognition ( warning about KeyError
>> VS71COMNTOOLS) doesn't help, translation failes with the same error.
>> Any suggestions?
>>
>> P.S. In answers, please, put my mail in copy, cause I'm subscribed to
>> digest =)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev


From nvetoshkin at naumen.ru  Tue Jan 20 23:24:34 2009
From: nvetoshkin at naumen.ru (=?UTF-8?B?0JLQtdGC0L7RiNC60LjQvSDQndC40LrQuNGC0LA=?=)
Date: Wed, 21 Jan 2009 03:24:34 +0500
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <DC3EFDC5-F48E-41CC-B49A-A9B68CECC29D@gmx.net>
References: <49761E35.3020307@naumen.ru> <49763E8F.4010304@naumen.ru>
	<DC3EFDC5-F48E-41CC-B49A-A9B68CECC29D@gmx.net>
Message-ID: <49764F22.7080606@naumen.ru>

I used /MT flag - that is static linking (in 
pypy\translator\platform\windows.py). Then installed zlib, added zlib's 
include dir to my INCLUDE env variable. And... now it's building =)

21.01.2009 2:57, Lutz Paelike ?????:
> I have a guess how that could happen.
>
> The reason is probably that two different versions of MSVCR dll are
> loaded into the process that
> result in a name clash of the symbols provided by the library.
>
> Now *why* this happens is not entirely clear but i have also a wild
> guess here.
>
> The standard Python2.4 distribution is using MSVCR71.dll an older
> version (7.1)
> that is used when compiled with e.g. Visual Studio 2005
>
> If you use python all installed modules are linked against the same
> version of the MSVCR dll
> and thus it works without a problem (provided you have MSVCR71.dll on
> your system).
>
> Now the problem arises if you compile an extension yourself with a
> different compiler version
> which links your compiled extension against a different version of the
> library (in this case MSVCR90.dll = Version 9.0).
>
> Solution:
> 1. You should use the same compiler version (VS 2005) as the one used of
> your python installation
> or
> 2. Compile Python and all modules you use yourself from source and use
> that instead of an precompiled distribution
>
> Another variable in this setup might be ctypes.
> If ctypes loads the MSVCR library without providing the exact requested
> version it might
> determine the best (newest) version itself and that might not be right one.
>
> Hope this helps.
> Cheers,
>
> Lutz
>
>
> Am 20.01.2009 um 22:13 schrieb ???????? ??????:
>
>> Thanks to fijal, I've installed ctypes and now I have MessageBox saying
>> that system can't find "MSVCR90.DLL". After putting this library into
>> current directory, I have this error:
>> ---------------------------
>> Microsoft Visual C++ Runtime Library
>> ---------------------------
>> Runtime Error!
>>
>> Program: C:\Python24\python.exe
>>
>> R6034
>>
>> An application has made an attempt to load the C runtime library
>> incorrectly.
>> Please contact the application's support team for more information.
>>
>>
>> ---------------------------
>> OK
>> ---------------------------
>>
>> Google found this: http://msdn.microsoft.com/en-us/library/ms235560.aspx
>> As I understand - we need to include manifest into resulting binary or
>> perhaps link externmod_2.dll statically.
>> The resulting stacktrace: http://pastebin.com/m6965b0f7
>> Please help =)
>>
>> 20.01.2009 23:55, ???????? ?????? ?????:
>>> Hi, all!
>>>
>>> I'm trying to translate PyPy on Windows Vista x64 with Python2.4 32 bit
>>> and Visual Studio 2008 Express Edition. 32 bit - cause Express Edition
>>> isn't shipped with x64 compiler.
>>> Translation failes: http://pastebin.com/m70de1376.
>>> Fixing Visual Studio version recognition ( warning about KeyError
>>> VS71COMNTOOLS) doesn't help, translation failes with the same error.
>>> Any suggestions?
>>>
>>> P.S. In answers, please, put my mail in copy, cause I'm subscribed to
>>> digest =)
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>

From amauryfa at gmail.com  Tue Jan 20 23:53:31 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 20 Jan 2009 23:53:31 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49764F22.7080606@naumen.ru>
References: <49761E35.3020307@naumen.ru> <49763E8F.4010304@naumen.ru>
	<DC3EFDC5-F48E-41CC-B49A-A9B68CECC29D@gmx.net>
	<49764F22.7080606@naumen.ru>
Message-ID: <e27efe130901201453q3003adb2x9360518855bac1aa@mail.gmail.com>

Hello,

On Tue, Jan 20, 2009 at 23:24, ???????? ?????? <nvetoshkin at naumen.ru> wrote:
> I used /MT flag - that is static linking (in
> pypy\translator\platform\windows.py). Then installed zlib, added zlib's
> include dir to my INCLUDE env variable. And... now it's building =)

The /MT flag is a nice trick.
But the ctypes module of the generated pypy-c will not work correctly:
get_libc() is supposed to return the MSVCRT.dll linked with the
executable, this function won't work if the c runtime is linked
statically.

The best would be to use /MT (=static C runtime) to build extension
modules (during the "configure" phase) and /MD (=shared C runtime)
when building the pypy-c executable.

-- 
Amaury Forgeot d'Arc

From nvetoshkin at naumen.ru  Wed Jan 21 00:12:37 2009
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Wed, 21 Jan 2009 04:12:37 +0500
Subject: [pypy-dev] Translate failes =(
Message-ID: <49765A65.6040102@naumen.ru>

Could you please point me source files where to look/digg:
"configure" phase building and
pypy-c executable building.
As I see now, /MT flag I used applies to all compiled binaries and you 
suggest using it only during first building process?
Any ideas about building manifest file to proper msvcrt.dll linking?

 >The /MT flag is a nice trick.
 >But the ctypes module of the generated pypy-c will not work correctly:
 >get_libc() is supposed to return the MSVCRT.dll linked with the
 >executable, this function won't work if the c runtime is linked
 >statically.

 >The best would be to use /MT (=static C runtime) to build extension
 >modules (during the "configure" phase) and /MD (=shared C runtime)
 >when building the pypy-c executable.
 >
 >-- Amaury Forgeot d'Arc

From fijall at gmail.com  Wed Jan 21 00:17:25 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 Jan 2009 00:17:25 +0100
Subject: [pypy-dev] windows buildbot
Message-ID: <693bc9ab0901201517ma487febub82414bdfe4174e0@mail.gmail.com>

I just triggered translation here:

http://codespeak.net:8099/builders/pypy-c-lib-python-win-32/builds/23/steps/translate/logs/stdio

and I noticed that there are 48 implement files. Is it because files
are split to smaller chunks for some reason,
or something went wrong along the lines?

Cheers,
fijal

From amauryfa at gmail.com  Wed Jan 21 00:40:45 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Jan 2009 00:40:45 +0100
Subject: [pypy-dev] windows buildbot
In-Reply-To: <693bc9ab0901201517ma487febub82414bdfe4174e0@mail.gmail.com>
References: <693bc9ab0901201517ma487febub82414bdfe4174e0@mail.gmail.com>
Message-ID: <e27efe130901201540i4c23df8bpba1b1fb3b6afe561@mail.gmail.com>

On Wed, Jan 21, 2009 at 00:17, Maciej Fijalkowski <fijall at gmail.com> wrote:
> I just triggered translation here:
>
> http://codespeak.net:8099/builders/pypy-c-lib-python-win-32/builds/23/steps/translate/logs/stdio
>
> and I noticed that there are 48 implement files. Is it because files
> are split to smaller chunks for some reason,
> or something went wrong along the lines?

It's normal if you consider this code in translator/c/genc.py:

SPLIT_CRITERIA = 65535 # support VC++ 7.2
[...]
        if py.std.sys.platform != "win32":
            split_criteria_big = SPLIT_CRITERIA * 4
        else:
            split_criteria_big = SPLIT_CRITERIA


Old versions of Visual Studio had problems with large files.
Even today, debug information is wrong when lineno>65536.

-- 
Amaury Forgeot d'Arc

From renesd at gmail.com  Wed Jan 21 00:43:42 2009
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Wed, 21 Jan 2009 10:43:42 +1100
Subject: [pypy-dev] windows buildbot
In-Reply-To: <e27efe130901201540i4c23df8bpba1b1fb3b6afe561@mail.gmail.com>
References: <693bc9ab0901201517ma487febub82414bdfe4174e0@mail.gmail.com>
	<e27efe130901201540i4c23df8bpba1b1fb3b6afe561@mail.gmail.com>
Message-ID: <64ddb72c0901201543k133d1d7dm11c02e41052cda48@mail.gmail.com>

hi,

I guess you already know that newer versions of gcc are slower with
larger files.  It can be faster even for gcc to split up files into
smaller pieces.

cheers,


On Wed, Jan 21, 2009 at 10:40 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> On Wed, Jan 21, 2009 at 00:17, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> I just triggered translation here:
>>
>> http://codespeak.net:8099/builders/pypy-c-lib-python-win-32/builds/23/steps/translate/logs/stdio
>>
>> and I noticed that there are 48 implement files. Is it because files
>> are split to smaller chunks for some reason,
>> or something went wrong along the lines?
>
> It's normal if you consider this code in translator/c/genc.py:
>
> SPLIT_CRITERIA = 65535 # support VC++ 7.2
> [...]
>        if py.std.sys.platform != "win32":
>            split_criteria_big = SPLIT_CRITERIA * 4
>        else:
>            split_criteria_big = SPLIT_CRITERIA
>
>
> Old versions of Visual Studio had problems with large files.
> Even today, debug information is wrong when lineno>65536.
>
> --
> Amaury Forgeot d'Arc

From amauryfa at gmail.com  Wed Jan 21 01:26:54 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Jan 2009 01:26:54 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49765A65.6040102@naumen.ru>
References: <49765A65.6040102@naumen.ru>
Message-ID: <e27efe130901201626l7bbfd9dch95557ebd00e2166c@mail.gmail.com>

On Wed, Jan 21, 2009 at 00:12, Vetoshkin Nikita <nvetoshkin at naumen.ru> wrote:
> Could you please point me source files where to look/digg:
> "configure" phase building and
> pypy-c executable building.
> As I see now, /MT flag I used applies to all compiled binaries and you
> suggest using it only during first building process?

Yes. My suggestion was that "standalone" builds should use /MD and
shared libraries ("not standalone")
use /MT.
But I realize that this would not solve all problems.

> Any ideas about building manifest file to proper msvcrt.dll linking?

I am currently trying to to it, borrowing ideas from cpython's code:
http://svn.python.org/view/python/trunk/Lib/distutils/msvc9compiler.py?rev=68081&view=markup
look for "manifest"

-- 
Amaury Forgeot d'Arc

From amauryfa at gmail.com  Wed Jan 21 02:14:34 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Jan 2009 02:14:34 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49765A65.6040102@naumen.ru>
References: <49765A65.6040102@naumen.ru>
Message-ID: <e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>

On Wed, Jan 21, 2009 at 00:12, Vetoshkin Nikita <nvetoshkin at naumen.ru> wrote:
> Any ideas about building manifest file to proper msvcrt.dll linking?

I just changed the compilation tools in windows.py to build and embed
this manifest.
Can you try again?

-- 
Amaury Forgeot d'Arc

From nvetoshkin at naumen.ru  Wed Jan 21 02:42:02 2009
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Wed, 21 Jan 2009 06:42:02 +0500
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
References: <49765A65.6040102@naumen.ru>
	<e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
Message-ID: <49767D6A.8080607@naumen.ru>

I've got this now: http://pastebin.com/d4f3dfefd

21.01.2009 6:14, Amaury Forgeot d'Arc ?????:
> On Wed, Jan 21, 2009 at 00:12, Vetoshkin Nikita<nvetoshkin at naumen.ru>  wrote:
>> Any ideas about building manifest file to proper msvcrt.dll linking?
>
> I just changed the compilation tools in windows.py to build and embed
> this manifest.
> Can you try again?
>

From amauryfa at gmail.com  Wed Jan 21 02:50:03 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Jan 2009 02:50:03 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49767D6A.8080607@naumen.ru>
References: <49765A65.6040102@naumen.ru>
	<e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
	<49767D6A.8080607@naumen.ru>
Message-ID: <e27efe130901201750oaf47a25s2cff92abb73afad0@mail.gmail.com>

On Wed, Jan 21, 2009 at 02:42, Vetoshkin Nikita <nvetoshkin at naumen.ru> wrote:
> I've got this now: http://pastebin.com/d4f3dfefd

This output is missing the few lines that can be important.
I would like to see the (long) lines starting with
    "[platform:execute] link.exe..."
and
    "[platform:execute] mt.exe..."
and which contain the manifest file name

-- 
Amaury Forgeot d'Arc

From amauryfa at gmail.com  Wed Jan 21 02:50:48 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 21 Jan 2009 02:50:48 +0100
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <49767D6A.8080607@naumen.ru>
References: <49765A65.6040102@naumen.ru>
	<e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
	<49767D6A.8080607@naumen.ru>
Message-ID: <e27efe130901201750s1b17c12cqa8489d958d377a7c@mail.gmail.com>

On Wed, Jan 21, 2009 at 02:42, Vetoshkin Nikita <nvetoshkin at naumen.ru> wrote:
> I've got this now: http://pastebin.com/d4f3dfefd

Oh, and did you reset the /MD option? /MT is wrong now.

-- 
Amaury Forgeot d'Arc

From nvetoshkin at naumen.ru  Wed Jan 21 02:52:07 2009
From: nvetoshkin at naumen.ru (Vetoshkin Nikita)
Date: Wed, 21 Jan 2009 06:52:07 +0500
Subject: [pypy-dev] Translate failes =(
In-Reply-To: <e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
References: <49765A65.6040102@naumen.ru>
	<e27efe130901201714w4ec2f4bcyc7b7f499ec55a03d@mail.gmail.com>
Message-ID: <49767FC7.9010902@naumen.ru>

Sorry, I was wrong. It works fine... awaiting translation to complete =)

21.01.2009 6:14, Amaury Forgeot d'Arc ?????:
> On Wed, Jan 21, 2009 at 00:12, Vetoshkin Nikita<nvetoshkin at naumen.ru>  wrote:
>> Any ideas about building manifest file to proper msvcrt.dll linking?
>
> I just changed the compilation tools in windows.py to build and embed
> this manifest.
> Can you try again?
>

From anto.cuni at gmail.com  Wed Jan 21 14:07:58 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 21 Jan 2009 14:07:58 +0100
Subject: [pypy-dev] Non-circular path through pypy module imports
In-Reply-To: <gkftm6$qfk$1@ger.gmane.org>
References: <gkftm6$qfk$1@ger.gmane.org>
Message-ID: <49771E2E.4030301@gmail.com>

VanL wrote:
> Hello,

Hi!
Sorry for the late response, everyone thought that someone else would have 
been answered, but nobody did at the end :-)

> I have been going through the pypy sources recently and I wanted to start from
> the leaf modules first (in terms of import dependencies)... but there don't
> appear to be any leaves.
> 
> For example, starting with baseobjspace in the interpreter, it has 30 or so
> non-stdlib imports, many of them delayed by placing them in functions. Even
> something like rarithmetic eventually pulls in most of the lltype hierarchy. The
> closest I found to a leaf module was extregistry.py.
> 
> Is there a reason for the circularities that I just don't get? If not, why are
> there so many?

Well, for at least some of circularities is partly our fault, we know that the 
import relationships between our modules are a mess currently. Sorry for that.

Anyway, I don't think that starting from the leaves is the best option to 
grasp pypy: probably it's better to pick a topic you are interested in, and 
start from there.

If you are more interested in the translation toolchain (i.e., the 
rpython-to-{c,cli,jvm} compiler), you can start from either the beginning or 
the end of the chain; look for example at the annotation/ directory or at the 
various backends in translator/{c,cli,jvm}.

On the other hand, if you are more interested in the interpreter (i.e., the 
python implementation written in rpython) you want to have a look to 
interpreter/ and objspace/std. The former directory contains the core of the 
interpreter, while the latter contains the implementation of all the standard 
builtin types such as lists, dictionaries, classes, etc.

A good starting point could be the main loop of the interpreter: look at the 
dispatch_bytecode function inside interpreter/pyopcode.py.

I assume that you have already read the pypy documentation online; if not, you 
are strongly encouraged to do that before looking at the source; in 
particular, this document describes the high level architecture of pypy:
http://codespeak.net/pypy/dist/pypy/doc/architecture.html

More generally, all the pypy documentation is available here:
http://codespeak.net/pypy/dist/pypy/doc/

ciao,
Anto

From anto.cuni at gmail.com  Thu Jan 22 11:16:45 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 22 Jan 2009 11:16:45 +0100
Subject: [pypy-dev] [pypy-svn] r61220 - pypy/trunk/lib-python
In-Reply-To: <20090122093417.85D5A168460@codespeak.net>
References: <20090122093417.85D5A168460@codespeak.net>
Message-ID: <4978478D.1030909@gmail.com>

fijal at codespeak.net wrote:

> we need to have RegrTest for each file starting with test_ anyway...
> 
> 
> Modified: pypy/trunk/lib-python/conftest.py
> ==============================================================================
> --- pypy/trunk/lib-python/conftest.py	(original)
> +++ pypy/trunk/lib-python/conftest.py	Thu Jan 22 10:34:15 2009
> @@ -294,6 +294,7 @@
>      RegrTest('test_mmap.py'),
>      RegrTest('test_module.py', core=True),
>      RegrTest('test_multibytecodec.py', skip="unsupported codecs"),
> +    RegrTest('test_multibytecodec.py_support', skip="not a test"),

uhm... did you want to write test_multibytecodec_support.py, maybe?

From fijall at gmail.com  Thu Jan 22 11:47:27 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 22 Jan 2009 11:47:27 +0100
Subject: [pypy-dev] [pypy-svn] r61220 - pypy/trunk/lib-python
In-Reply-To: <4978478D.1030909@gmail.com>
References: <20090122093417.85D5A168460@codespeak.net>
	<4978478D.1030909@gmail.com>
Message-ID: <693bc9ab0901220247y1303d009vaaf23e36057f2dfc@mail.gmail.com>

argh, yes

On 1/22/09, Antonio Cuni <anto.cuni at gmail.com> wrote:
> fijal at codespeak.net wrote:
>
>> we need to have RegrTest for each file starting with test_ anyway...
>>
>>
>> Modified: pypy/trunk/lib-python/conftest.py
>> ==============================================================================
>> --- pypy/trunk/lib-python/conftest.py	(original)
>> +++ pypy/trunk/lib-python/conftest.py	Thu Jan 22 10:34:15 2009
>> @@ -294,6 +294,7 @@
>>      RegrTest('test_mmap.py'),
>>      RegrTest('test_module.py', core=True),
>>      RegrTest('test_multibytecodec.py', skip="unsupported codecs"),
>> +    RegrTest('test_multibytecodec.py_support', skip="not a test"),
>
> uhm... did you want to write test_multibytecodec_support.py, maybe?
>

From fijall at gmail.com  Sat Jan 24 20:42:55 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 24 Jan 2009 20:42:55 +0100
Subject: [pypy-dev] parser compiler and _ast
Message-ID: <693bc9ab0901241142j50a5dd96s591b219620ba8b60@mail.gmail.com>

I did a bit of research on a matter of different python interfaces to
parsing. This is a bit of a mess, but it is as follows:

a) there is a parser module which gives you very low-level interface
    (so-called concrete syntax tree), which is essentially
    a list of tuples with numbers inside. you can compile those tuples
    to bytecode afterwards. It's slowly being deprecated.

b) there is a compiler module, which is a wrapper around parser. it
    does not let you compile ast, but it gives you ast. It's deprecated since
    2.6.

c) there is an ast module which lets you have ast *and* compile it. very neat.

Now the state of the art is that we have parser, compiler is pure
app-level and we
don't provide ast. We already discussed a bit whether to have ast module for 1.1
and the answer is "very likely no", raise hands if you want to have it.

Now I want to know how much do we care about all of this and how much do we care
about any of those? My suggestions are as follows:

1. We keep parser as it is and don't try to fix bugs. Bugs (as
reported by CPython's
test failures) are that we don't do extra walk around the tree to
detect syntax errors.
those errors are only detected when we generated the code

- or -

1a. We just always generate the code and throw it away. We loose
performance, but we don't care
(and we pass tests)

2. We slowly deprecate parser & compiler, with some removal in future

3. If we develop a new parser, we drop compatibility with exact
concrete syntax trees and
we only keep ast around (as ast module) if people want to interact
with it (this is one less
worry for writing parser).

What do you think?

Cheers,
fijal

From fijall at gmail.com  Mon Jan 26 11:05:20 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 26 Jan 2009 11:05:20 +0100
Subject: [pypy-dev] resource module for pypy
Message-ID: <693bc9ab0901260205o5a144cfcx368a992a372b96a5@mail.gmail.com>

Hi Victor, hi pypy-dev

I think you're the one who implemented the resource module using
ctypes. Right now it timouts/fails when
running cpython's test suite (1.) - last run. Our policy is to either
fix it, or remove it for 1.1 release.
Do you eventually would like to take a look?

Cheers,
fijal

1. http://codespeak.net:8099/summary/longrepr?testname=unmodified&builder=pypy-c-lib-python-linux-x86-32&build=129&mod=lib-python.test_resource

From anto.cuni at gmail.com  Mon Jan 26 14:24:14 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Mon, 26 Jan 2009 14:24:14 +0100
Subject: [pypy-dev] parser compiler and _ast
In-Reply-To: <693bc9ab0901241142j50a5dd96s591b219620ba8b60@mail.gmail.com>
References: <693bc9ab0901241142j50a5dd96s591b219620ba8b60@mail.gmail.com>
Message-ID: <497DB97E.6070103@gmail.com>

Maciej Fijalkowski wrote:
> I did a bit of research on a matter of different python interfaces to
> parsing. This is a bit of a mess, but it is as follows:
> 
[cut]
> 1a. We just always generate the code and throw it away. We loose
> performance, but we don't care
> (and we pass tests)

a quick google code search reveals that there are a couple of projects using 
parser, like pychecker, epydoc and quixote. If it's not too time consuming, I 
would go for this option.

> 2. We slowly deprecate parser & compiler, with some removal in future

+1

> 3. If we develop a new parser, we drop compatibility with exact
> concrete syntax trees and
> we only keep ast around (as ast module) if people want to interact
> with it (this is one less
> worry for writing parser).

That's an option, but not for 1.1, I think.

ciao,
Anto

From jacob at openend.se  Tue Jan 27 22:24:20 2009
From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Tue, 27 Jan 2009 22:24:20 +0100
Subject: [pypy-dev] Europython sprint
Message-ID: <200901272224.20283.jacob@openend.se>

Hi everyone,

the Europython organisers would like us to settle on dates for a sprint around 
Europython as soon as possible. They are very eager to do well in the 
sprinting department, as sprints are something that hasn't taken off in the 
UK so far. It is something they very much want to change.

PyPy is naturally considered to be the leader in the field of sprints and the 
organisers would like to accomdate our needs as best they can. This means 
that they have to book rooms in the next week or so.

The outine of the programme is as follows:

    *  Sunday 28th June and Monday 29th June : Tutorial Days
    * Tuesday 30th June to Thursday 2nd July : Conference
    * Friday 3rd July, as long as needed : Sprints

We could have a sprint(an internal one?)  in parallel with the tutorials if we 
want.

How long would we like to sprint after the conference? Friday-Sunday?

Birmingham is quite a nice environment and the rate of the British pound is 
not so outrageous, that we probably can afford a decent length of stay.

Jacob

From anto.cuni at gmail.com  Wed Jan 28 13:27:25 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 28 Jan 2009 13:27:25 +0100
Subject: [pypy-dev] Europython sprint
In-Reply-To: <200901272224.20283.jacob@openend.se>
References: <200901272224.20283.jacob@openend.se>
Message-ID: <49804F2D.3060005@gmail.com>

Hi Jacob, hi all,

Jacob Hall?n wrote:

> The outine of the programme is as follows:
> 
>     *  Sunday 28th June and Monday 29th June : Tutorial Days
>     * Tuesday 30th June to Thursday 2nd July : Conference
>     * Friday 3rd July, as long as needed : Sprints
> 
> We could have a sprint(an internal one?)  in parallel with the tutorials if we 
> want.
> 
> How long would we like to sprint after the conference? Friday-Sunday?

Just a quick input from my side: when deciding the sprint dates, we should 
keep in mind that ECOOP 2009 starts on Monday 6th July. I will surely be 
there, and possibly other pypyers as well (both Carl Friedrich and Armin 
showed interest e.g.).

ciao,
Anto

From fijall at gmail.com  Wed Jan 28 13:35:25 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 28 Jan 2009 13:35:25 +0100
Subject: [pypy-dev] official announcement of snakebite
Message-ID: <693bc9ab0901280435h7b0ca1d8mc007cd4f5216d197@mail.gmail.com>

Some very good news for us:

http://groups.google.com/group/snakebite-list/browse_thread/thread/6402c23808063cbe

seems like going towards end of windows problems :)

Cheers and thanks to Trent,
fijal

From tismer at stackless.com  Mon Feb  2 11:28:08 2009
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 02 Feb 2009 02:28:08 -0800
Subject: [pypy-dev] Psyco version 2 preview is available
Message-ID: <4986CAB8.30604@stackless.com>

This is not an official announcement of Psyco V2, yet.

But everybody who is interested should check out

http://codespeak.net:/svn/psyco/v2

and do some testing.

The major improvement besides lots of builtins is the new
generator support, which is disabled by default, due to an
internal error in Psyco.

To use and test generators, create preferences.py, following
the instructions in setup.py.

This software is considered asan alpha release.
The final version should be expected before end of February.

regards -- chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From kfh at mqsoftware.com  Wed Feb 11 19:28:58 2009
From: kfh at mqsoftware.com (Kelly F. Hickel)
Date: Wed, 11 Feb 2009 12:28:58 -0600
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and dbm
Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>

Hi all, I didn't see a pypy-user list, so I'm posting here.

I came across PyPy while trying to find a way to speed up cvs2svn while
converting a large CVS repo.  I've pulled the latest via svn, got it
built and have translated the interpreter, but cvs2svn using pypy-c
fails with the output:
debug: WARNING: library path not found, using compiled-in sys.path
ERROR: cvs2svn uses the anydbm package, which depends on lower level dbm
libraries.  Your system has dumbdbm, with which cvs2svn is known to have
problems.  To use cvs2svn, you must install a Python dbm library other
than
dumbdbm or dbm.  See
http://python.org/doc/current/lib/module-anydbm.html
for more information.

Now, when we've run into this issue with python, someone just rebuilt
python enabling built in libgdbm support, but I haven't found any way to
do that or include some other dbm within PyPy.

Since I'm not that familiar with python to start with, I'm hoping that
I'm just missing something quite basic, and that some kind soul here can
point me in the right direction...

Should I be trying to translate cvs2svn instead of trying using pypy-c ?
(this seems to involve building a target file or something).

Am I just out of luck?


--

Kelly F. Hickel
Senior Product Architect
MQSoftware, Inc.
952-345-8677 Office
952-345-8721 Fax
kfh at mqsoftware.com
www.mqsoftware.com
Certified IBM SOA Specialty
Your Full Service Provider for IBM WebSphere
Learn more at www.mqsoftware.com 



From santagada at gmail.com  Wed Feb 11 19:36:35 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Wed, 11 Feb 2009 16:36:35 -0200
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and
	dbm
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>
References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>
Message-ID: <26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com>


On Feb 11, 2009, at 4:28 PM, Kelly F. Hickel wrote:

> Hi all, I didn't see a pypy-user list, so I'm posting here.
>
> I came across PyPy while trying to find a way to speed up cvs2svn  
> while
> converting a large CVS repo.  I've pulled the latest via svn, got it
> built and have translated the interpreter, but cvs2svn using pypy-c
> fails with the output:
> debug: WARNING: library path not found, using compiled-in sys.path
> ERROR: cvs2svn uses the anydbm package, which depends on lower level  
> dbm
> libraries.  Your system has dumbdbm, with which cvs2svn is known to  
> have
> problems.  To use cvs2svn, you must install a Python dbm library other
> than
> dumbdbm or dbm.  See
> http://python.org/doc/current/lib/module-anydbm.html
> for more information.
>
> Now, when we've run into this issue with python, someone just rebuilt
> python enabling built in libgdbm support, but I haven't found any  
> way to
> do that or include some other dbm within PyPy.
>
> Since I'm not that familiar with python to start with, I'm hoping that
> I'm just missing something quite basic, and that some kind soul here  
> can
> point me in the right direction...
>
> Should I be trying to translate cvs2svn instead of trying using pypy- 
> c ?
> (this seems to involve building a target file or something).
>
> Am I just out of luck?

Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know would  
not give you any improvements in performance from CPython for this  
(maybe even psyco will not help).

ps: Isn't cvs2svn a one time thing?

--
Leonardo Santagada
santagada at gmail.com




From kfh at mqsoftware.com  Wed Feb 11 19:41:47 2009
From: kfh at mqsoftware.com (Kelly F. Hickel)
Date: Wed, 11 Feb 2009 12:41:47 -0600
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm and
	dbm
In-Reply-To: <26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com>
References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>
	<26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com>
Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com>

> > I'm just missing something quite basic, and that some kind soul here
> > can
> > point me in the right direction...
> >
> > Should I be trying to translate cvs2svn instead of trying using
pypy-
> > c ?
> > (this seems to involve building a target file or something).
> >
> > Am I just out of luck?
> 
> Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know would
> not give you any improvements in performance from CPython for this
> (maybe even psyco will not help).
> 
> ps: Isn't cvs2svn a one time thing?
> 
> --
> Leonardo Santagada
> santagada at gmail.com

Thanks for the quick reply, I'll give it a try. I had looked at pysco
back in June, but got the impression that I'd be better off with
PyPy....

Yes, cvs2svn is a one time thing in principle, but I've been running it
a number of times, trying different things, as a means of validating the
switch to git (cvs2svn also does cvs2git).  Since it takes 6.5 days on a
2.4ghz AMD with 32gb of ram to translate our repo, even a small tweak to
the process is quite expensive....

-Kelly

From kfh at mqsoftware.com  Wed Feb 11 19:49:59 2009
From: kfh at mqsoftware.com (Kelly F. Hickel)
Date: Wed, 11 Feb 2009 12:49:59 -0600
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com>
References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com><26AC1219-68EB-469E-A814-10B0FAB39FEF@gmail.com>
	<63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com>
Message-ID: <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com>

> >
> > Try to use psyco to speed up cvs2svn. I'm afraid PyPy right know
> would
> > not give you any improvements in performance from CPython for this
> > (maybe even psyco will not help).
> >
> > ps: Isn't cvs2svn a one time thing?
> >
> > --
> > Leonardo Santagada
> > santagada at gmail.com
> 
> Thanks for the quick reply, I'll give it a try. I had looked at pysco
> back in June, but got the impression that I'd be better off with
> PyPy....
> 
> Yes, cvs2svn is a one time thing in principle, but I've been running
it
> a number of times, trying different things, as a means of validating
> the
> switch to git (cvs2svn also does cvs2git).  Since it takes 6.5 days on
> a
> 2.4ghz AMD with 32gb of ram to translate our repo, even a small tweak
> to
> the process is quite expensive....
> 
> -Kelly


Ahh, yes, now I remember.  Psyco ONLY works with 32 bit python, whereas
I MUST use 64 bit because the memory footprint for cvs2svn with our repo
is just too large for 32 bit processes.

So, dead in the water again?

-Kelly

From alexandre.fayolle at logilab.fr  Thu Feb 12 09:14:20 2009
From: alexandre.fayolle at logilab.fr (Alexandre Fayolle)
Date: Thu, 12 Feb 2009 09:14:20 +0100
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm
In-Reply-To: <63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com>
References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>
	<63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com>
	<63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com>
Message-ID: <200902120914.26028.alexandre.fayolle@logilab.fr>

Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez ?crit?:
> Ahh, yes, now I remember.  Psyco ONLY works with 32 bit python, whereas
> I MUST use 64 bit because the memory footprint for cvs2svn with our repo
> is just too large for 32 bit processes.
>
> So, dead in the water again?

Hmm, just a quick question: are you sure your problem is CPU bound (as opposed 
to IO bound)? If it is not, you are looking at the wrong place to solve your 
performance issue, and you should rather look for a faster disk, faster 
network, etc. Also from what you say hereabove, maybe adding some RAM could 
help. 


-- 
Alexandre Fayolle                              LOGILAB, Paris (France)
Formations Python, Zope, Plone, Debian:  http://www.logilab.fr/formations
D?veloppement logiciel sur mesure:       http://www.logilab.fr/services
Informatique scientifique:               http://www.logilab.fr/science
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 481 bytes
Desc: This is a digitally signed message part.
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20090212/cf4240c9/attachment-0001.pgp 

From santagada at gmail.com  Thu Feb 12 12:20:25 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 12 Feb 2009 09:20:25 -0200
Subject: [pypy-dev] PyPy/python newb - need dbm other than dumbdbm anddbm
In-Reply-To: <200902120914.26028.alexandre.fayolle@logilab.fr>
References: <63BEA5E623E09F4D92233FB12A9F794302BC6F70@emailmn.mqsoftware.com>
	<63BEA5E623E09F4D92233FB12A9F794302BC6F74@emailmn.mqsoftware.com>
	<63BEA5E623E09F4D92233FB12A9F794302BC6F7B@emailmn.mqsoftware.com>
	<200902120914.26028.alexandre.fayolle@logilab.fr>
Message-ID: <CFEE1C53-1D76-4ABF-858F-F4BC48D1F167@gmail.com>


On Feb 12, 2009, at 6:14 AM, Alexandre Fayolle wrote:

> Le Wednesday 11 February 2009 19:49:59 Kelly F. Hickel, vous avez  
> ?crit :
>> Ahh, yes, now I remember.  Psyco ONLY works with 32 bit python,  
>> whereas
>> I MUST use 64 bit because the memory footprint for cvs2svn with our  
>> repo
>> is just too large for 32 bit processes.
>>
>> So, dead in the water again?
>
> Hmm, just a quick question: are you sure your problem is CPU bound  
> (as opposed
> to IO bound)? If it is not, you are looking at the wrong place to  
> solve your
> performance issue, and you should rather look for a faster disk,  
> faster
> network, etc. Also from what you say hereabove, maybe adding some  
> RAM could
> help.


I'm answering this as we already talked about it in some private  
emails. He already figured out the problem is cpu bond, one of the  
cpus goes to 100% and stays almost the whole time this way for 6.5  
days. His machine has 32 gb of RAM and the python conversion program  
uses good part of it that's why he can't run the thing on 32bit mode.  
His repo is only 6gb in size so I think maybe the problem is either a  
bad performing algo no cvs2svn or a bad cvs library (or whatever is  
used to access cvs).

If it is the cvs library there is nothing no one can do (even psyco or  
pypy would not help). If the problem is a wrong pure python algo it  
should probably be changed (if a better one exists of course).

Now why would a program reading a 6gb repo, even using a database file  
for intermediate data still uses tons of ram to do this conversion I  
don't know.

--
Leonardo Santagada
santagada at gmail.com




From stephan at transvection.de  Mon Feb 16 19:17:33 2009
From: stephan at transvection.de (Stephan Diehl)
Date: Mon, 16 Feb 2009 19:17:33 +0100
Subject: [pypy-dev] [Stackless] Fwd:  Stackless on top of greenlets
In-Reply-To: <952d92df0902160822q2baf5712jda9de5947a7215e3@mail.gmail.com>
References: <7bc21cb60902160210r1de6e235o63006f5d6f2abfbc@mail.gmail.com>	<952d92df0902160442p633f8a5fp26b9916cf5aec130@mail.gmail.com>	<7bc21cb60902160733s26ec2e4o30153a1797befaa9@mail.gmail.com>	<952d92df0902160821oef07395uab0938cf73104033@mail.gmail.com>
	<952d92df0902160822q2baf5712jda9de5947a7215e3@mail.gmail.com>
Message-ID: <4999ADBD.5050301@transvection.de>

Hi,

it would be quite nice, if the different implementations could somehow
be united (I've written the pypy one, but never used it, so there are
indeed lots of errors lurking).
pypy's stackless is actually implemented using pypy's coroutine
implementation, which, in case CPython is used, is emulated by greenlets.

pypy's stackless
http://codespeak.net/svn/pypy/trunk/pypy/lib/stackless.py

has a corresponding test file
http://codespeak.net/svn/pypy/trunk/pypy/lib/app_test/test_stackless.py

which is supposed to be compatible with regular stackless as well.

Maybe we could come up with a set of tests that cover all of stackless
interface and therefore can be used to test our greenlet based
implementations against.

Cheers, Stephan

Richard Tew wrote:
> ---------- Forwarded message ----------
> From: Richard Tew <richard.m.tew at gmail.com>
> Date: Mon, Feb 16, 2009 at 11:21 AM
> Subject: Re: [Stackless] Stackless on top of greenlets
> To: Henk Punt <henkpunt at gmail.com>
> 
> 
> On Mon, Feb 16, 2009 at 10:33 AM, Henk Punt <henkpunt at gmail.com> wrote:
>> Can you give some pointers to those implementations?,
>>
>> The only one I could find is the one included in PyPy and that one is
>> not-working, e.g. it lacks certains methods in the API and also it
>> does not behave
>> exactly as stackless does, especially w.r.t. the order of task-scheduling.
>> Also it contains quite a lot of bugs, for instance raising an
>> exception in a task that is
>> blocked on a channel does not restore the correct balance and similar issues...
>>
> 
> None of the implementations I have seen have been perfect, including my own.
> 
> http://www.stackless.com/pipermail/stackless/2005-May/002416.html
> 
> Cheers,
> Richard.
> 
> _______________________________________________
> Stackless mailing list
> Stackless at stackless.com
> http://www.stackless.com/mailman/listinfo/stackless

From amauryfa at gmail.com  Tue Feb 17 19:20:12 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 17 Feb 2009 19:20:12 +0100
Subject: [pypy-dev] pypy and the comtypes module
Message-ID: <e27efe130902171020q5c9ed44fpabc0501314aba153@mail.gmail.com>

Hello Thomas, hello to all pypy-ers,

I'm sure you will be interested to know that the ctypes implementation of pypy
has matured enough to be able to run the comtypes module,
at least some of the examples I found on the web site:
http://starship.python.net/crew/theller/comtypes/
See the copy of a pypy session below.

Not everything works, though.
For example, the COMError exception does not give any message,
some errors are caused by the alternate garbage collector of pypy
(opposed to the determinism of reference counting),
and I'm sure that I missed many important details...
At least it is now possible to run the comtypes test suite: some tests
even pass ;-)

Hey, I did not know that ctypes and comtypes where so intricate:
ctypes obviously has features designed for comtypes only (OUT parameters...)
and ctypes/__init__.py even tries to import comtypes.server.inprocserver!

But this was all fun, and I learned a lot about ffi and COM from
reading the code.
Thanks for these wonderful modules!



Python 2.5.2 (61966, Feb 17 2009, 13:39:17)
[PyPy 1.0.0] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``who am I and if yes, how many?''
>>>> from comtypes.client import CreateObject
>>>> xl = CreateObject("Excel.Application")
# Generating comtypes.gen._00020813_0000_0000_C000_000000000046_0_1_3
# Generating comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0
# Generating comtypes.gen.stdole
# Generating comtypes.gen._2DF8D04C_5BFA_101B_BDE5_00AA0044DE52_0_2_1
# Generating comtypes.gen.Office
# Generating comtypes.gen._0002E157_0000_0000_C000_000000000046_0_5_3
# Generating comtypes.gen.VBIDE
# Generating comtypes.gen.Excel
>>>> xl.Workbooks.Add()
<POINTER(_Workbook) ptr=0x16af6c at 35>
>>>> xl.Range("E1").Formula = "=SUM(A1:D1)"
>>>> print xl.Range("E1").Value
0.0
>>>> xl.Range("A1", "D1").Value = ("Data", 10, 20, 30)
>>>> print xl.Range("E1").Value
60.0
>>>> xl.Visible = True
>>>>

Cheers,

-- 
Amaury Forgeot d'Arc

From dalius.dobravolskas at gmail.com  Wed Feb 18 14:01:45 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Wed, 18 Feb 2009 15:01:45 +0200
Subject: [pypy-dev] pypy sandbox
Message-ID: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>

Hello, All,

I'm trying to translate sandboxed pypy but I get following error:

[sandbox:WARNING] Not Implemented: sandboxing for external function 'inet_ntop'
*
[rtyper] -=- specialized 3 more blocks -=-
[Timer] Timings:
[Timer] annotate                       --- 1380.9 s
[Timer] rtype_lltype                   --- 1414.1 s
[Timer] backendopt_lltype              ---  647.0 s
[Timer] stackcheckinsertion_lltype     ---  218.3 s
[Timer] database_c                     --- 1395.4 s
[Timer] ===========================================
[Timer] Total:                         --- 5055.7 s
[translation:ERROR] Error:
[translation:ERROR]  Traceback (most recent call last):
[translation:ERROR]    File "./translate.py", line 278, in main
[translation:ERROR]     drv.proceed(goals)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 802,
in proceed
[translation:ERROR]     return self._execute(goals, task_skip =
self._maybe_skip())
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/tool/taskengine.py",
line 116, in _execute
[translation:ERROR]     res = self._do(goal, taskcallable, *args, **kwds)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 270,
in _do
[translation:ERROR]     res = func()
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/driver.py", line 480,
in task_database_c
[translation:ERROR]     database = cbuilder.build_database()
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/genc.py", line 73,
in build_database
[translation:ERROR]     db.complete()
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line
310, in complete
[translation:ERROR]     add_dependencies(node.enum_dependencies())
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line
298, in add_dependencies
[translation:ERROR]     self.get(value)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line
218, in get
[translation:ERROR]     node = self.getcontainernode(container)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/database.py", line
156, in getcontainernode
[translation:ERROR]     node = nodefactory(self, T, container, **buildkwds)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 699,
in __init__
[translation:ERROR]     self.make_funcgens()
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 705,
in make_funcgens
[translation:ERROR]     self.funcgens =
select_function_code_generators(self.obj, self.db, self.name)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 825,
in select_function_code_generators
[translation:ERROR]     return sandbox_stub(fnobj, db)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/c/node.py", line 787,
in sandbox_stub
[translation:ERROR]     force_stub=True)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/translator/sandbox/rsandbox.py",
line 143, in get_external_function_sandbox_graph
[translation:ERROR]     args_s = [annmodel.lltype_to_annotation(ARG)
for ARG in FUNCTYPE.ARGS]
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 639,
in lltype_to_annotation
[translation:ERROR]     return SomePtr(T)
[translation:ERROR]    File
"/home/dalius/projects/pypy-dist/pypy/annotation/model.py", line 537,
in __init__
[translation:ERROR]     assert isinstance(ll_ptrtype, lltype.Ptr)
[translation:ERROR]  AssertionError
[translation] start debugger...
> /home/dalius/projects/pypy-dist/pypy/annotation/model.py(537)__init__()
-> assert isinstance(ll_ptrtype, lltype.Ptr)
(Pdb+) q

I was running command: /pypy-dist/pypy/translator/goal$ ./translate.py
--sandbox --opt=1 --gc=hybrid targetpypystandalone.py

I have just quited this time but it failed with same error as I run
./translate.py --sandbox targetpypystandalone.py (without options).

ll_ptrtype is sock_addr if I remember correctly (when I will run it
next time I will check).

My system:
$ uname -a
Linux dalius 2.6.27-11-generic #1 SMP Thu Jan 29 19:24:39 UTC 2009
i686 GNU/Linux
$ python -V
Python 2.5.2

-- 
Dalius
http://blog.sandbox.lt

From anto.cuni at gmail.com  Wed Feb 18 14:08:30 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 18 Feb 2009 14:08:30 +0100
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
Message-ID: <499C084E.6040806@gmail.com>

Dalius Dobravolskas wrote:
> Hello, All,

Hi Dalius,

[cut]
>> /home/dalius/projects/pypy-dist/pypy/annotation/model.py(537)__init__()

Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk to 
dist soon) please try with pypy/trunk.

ciao,
Anto

From dalius.dobravolskas at gmail.com  Wed Feb 18 21:39:48 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Wed, 18 Feb 2009 22:39:48 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <499C084E.6040806@gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
Message-ID: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>

Hello,

On Wed, Feb 18, 2009 at 3:08 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Unfortunately pypy/dist is largely outdated nowadays (we plan to copy trunk
> to dist soon) please try with pypy/trunk.
trunk has worked just perfect.

I will try to implement WSGI sample similar to Django (as mentioned
here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html).

-- 
Dalius
http://blog.sandbox.lt

From anto.cuni at gmail.com  Wed Feb 18 23:45:21 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 18 Feb 2009 23:45:21 +0100
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
Message-ID: <499C8F81.6070103@gmail.com>

Dalius Dobravolskas wrote:

> trunk has worked just perfect.

nice :-)

> I will try to implement WSGI sample similar to Django (as mentioned
> here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html).

that's cool. Please keep us informed, and feel free to come to #pypy on 
irc.freenode.net if you have any question.

ciao,
Anto

From theller at ctypes.org  Thu Feb 19 21:41:48 2009
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 19 Feb 2009 21:41:48 +0100
Subject: [pypy-dev] pypy and the comtypes module
In-Reply-To: <e27efe130902171020q5c9ed44fpabc0501314aba153@mail.gmail.com>
References: <e27efe130902171020q5c9ed44fpabc0501314aba153@mail.gmail.com>
Message-ID: <499DC40C.3060801@ctypes.org>

[I added comtypes-users to the list of recipients, in case any existing
comtypes users is interested in this]

Amaury Forgeot d'Arc schrieb:
> Hello Thomas, hello to all pypy-ers,
> 
> I'm sure you will be interested to know that the ctypes implementation of pypy
> has matured enough to be able to run the comtypes module,
> at least some of the examples I found on the web site:
> http://starship.python.net/crew/theller/comtypes/
> See the copy of a pypy session below.

Really cool to hear this.  I must say that I sometimes look over the commit messages
in the pypy.cvs mailing list (via gmane) and I'm fascinated by the work with ctypes
that is done there!

What I fear is that the day is not far where you know much more than myself about ctypes ;-)

> Not everything works, though.
> For example, the COMError exception does not give any message,
> some errors are caused by the alternate garbage collector of pypy
> (opposed to the determinism of reference counting),
> and I'm sure that I missed many important details...
> At least it is now possible to run the comtypes test suite: some tests
> even pass ;-)
> 
> Hey, I did not know that ctypes and comtypes where so intricate:
> ctypes obviously has features designed for comtypes only (OUT parameters...)
> and ctypes/__init__.py even tries to import comtypes.server.inprocserver!
> 
> But this was all fun, and I learned a lot about ffi and COM from
> reading the code.
> Thanks for these wonderful modules!
> 
> 
> 
> Python 2.5.2 (61966, Feb 17 2009, 13:39:17)
> [PyPy 1.0.0] on win32
> Type "help", "copyright", "credits" or "license" for more information.
> And now for something completely different: ``who am I and if yes, how many?''
>>>>> from comtypes.client import CreateObject
>>>>> xl = CreateObject("Excel.Application")
> # Generating comtypes.gen._00020813_0000_0000_C000_000000000046_0_1_3
> # Generating comtypes.gen._00020430_0000_0000_C000_000000000046_0_2_0
> # Generating comtypes.gen.stdole
> # Generating comtypes.gen._2DF8D04C_5BFA_101B_BDE5_00AA0044DE52_0_2_1
> # Generating comtypes.gen.Office
> # Generating comtypes.gen._0002E157_0000_0000_C000_000000000046_0_5_3
> # Generating comtypes.gen.VBIDE
> # Generating comtypes.gen.Excel
>>>>> xl.Workbooks.Add()
> <POINTER(_Workbook) ptr=0x16af6c at 35>
>>>>> xl.Range("E1").Formula = "=SUM(A1:D1)"
>>>>> print xl.Range("E1").Value
> 0.0
>>>>> xl.Range("A1", "D1").Value = ("Data", 10, 20, 30)
>>>>> print xl.Range("E1").Value
> 60.0
>>>>> xl.Visible = True
>>>>>
> 
> Cheers,
> 

Fantastic!  I wish I had time to use pypy or even participate, but
there is no chance, unfortunately.

-- 
Thanks,
Thomas

From dalius.dobravolskas at gmail.com  Fri Feb 20 07:30:32 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Fri, 20 Feb 2009 08:30:32 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <499C8F81.6070103@gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
Message-ID: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>

Hello,

>> I will try to implement WSGI sample similar to Django (as mentioned
>> here
>> http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html).
>
> that's cool. Please keep us informed, and feel free to come to #pypy on
> irc.freenode.net if you have any question.

Here is result: http://py.sandbox.lt/

Basically it is the same code as Django example but only written in
WSGI (I will share that later).

Here some of my experiences that I think I should mention:

1. I'm running my code behind Apache mod_wsgi. As you might know
mod_wsgi is very strict WSGI implementation. pypy_interact failed when
was calling isatty function (mod_wsgi.Log does not have this
function). I have fixed that by adding "'isatty in dir(log)". I will
give later exact locations and names if I have confused them.

2. It looks like that PyPySandboxedProc expects --heapsize as first
argument in argument list.

3. Is settimeout actually working? If I run following code (please
don't run it on my server):

x = 1
for y in xrange(1, 1024*1024):
     x *= y
print x

Process is not terminated after one second and uses significant part
of my CPU resources. I set timeout in following way:

sandproc = pypy_interact.PyPySandboxedProc(SANDBOX_BIN, ['--heapsize',
str(1024*1024), '-c', code])
sandproc.settimeout(1, interrupt_main=True)

I have tried 5 seconds as well. I will analyze the code myself a
little bit but if you have ideas already please share them ;-)

-- 
Dalius
http://blog.sandbox.lt

From dalius.dobravolskas at gmail.com  Fri Feb 20 09:05:45 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Fri, 20 Feb 2009 10:05:45 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
Message-ID: <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com>

Hello,

On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas
<dalius.dobravolskas at gmail.com> wrote:
> 3. Is settimeout actually working? If I run following code (please
> don't run it on my server):
OK. Everything is OK in PyPy code. timeout works (at least log
indicates that). There is something specific about mod_wsgi and Apache
in this case.

-- 
Dalius
http://blog.sandbox.lt

From dalius.dobravolskas at gmail.com  Fri Feb 20 10:13:11 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Fri, 20 Feb 2009 11:13:11 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
	<98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com>
Message-ID: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com>

On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas
<dalius.dobravolskas at gmail.com> wrote:
> Hello,
>
> On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas
> <dalius.dobravolskas at gmail.com> wrote:
>> 3. Is settimeout actually working? If I run following code (please
>> don't run it on my server):
> OK. Everything is OK in PyPy code. timeout works (at least log
> indicates that). There is something specific about mod_wsgi and Apache
> in this case.
That's what I have found out:
pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed.
That's how I solved problem now (modifed pypy/tool/killsubprocess.py).
I will dig deeper of course but I will do that later.

HTH for other people who will run sandboxed pypy.

-- 
Dalius
http://blog.sandbox.lt

From fijall at gmail.com  Fri Feb 20 13:42:22 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 20 Feb 2009 13:42:22 +0100
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
	<98dd087e0902200005r733d1ed7gd6ee638a937a74fe@mail.gmail.com>
	<98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com>
Message-ID: <693bc9ab0902200442t44bb52a6l58bb2945ddeccea1@mail.gmail.com>

That's all very good news. Keep us informed!

Cheers,
fijal

On Fri, Feb 20, 2009 at 10:13 AM, Dalius Dobravolskas
<dalius.dobravolskas at gmail.com> wrote:
> On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas
> <dalius.dobravolskas at gmail.com> wrote:
>> Hello,
>>
>> On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas
>> <dalius.dobravolskas at gmail.com> wrote:
>>> 3. Is settimeout actually working? If I run following code (please
>>> don't run it on my server):
>> OK. Everything is OK in PyPy code. timeout works (at least log
>> indicates that). There is something specific about mod_wsgi and Apache
>> in this case.
> That's what I have found out:
> pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed.
> That's how I solved problem now (modifed pypy/tool/killsubprocess.py).
> I will dig deeper of course but I will do that later.
>
> HTH for other people who will run sandboxed pypy.
>
> --
> Dalius
> http://blog.sandbox.lt
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From exarkun at divmod.com  Fri Feb 20 16:05:51 2009
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 20 Feb 2009 10:05:51 -0500
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com>
Message-ID: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com>

On Fri, 20 Feb 2009 11:13:11 +0200, Dalius Dobravolskas <dalius.dobravolskas at gmail.com> wrote:
>On Fri, Feb 20, 2009 at 10:05 AM, Dalius Dobravolskas
><dalius.dobravolskas at gmail.com> wrote:
>> Hello,
>>
>> On Fri, Feb 20, 2009 at 8:30 AM, Dalius Dobravolskas
>> <dalius.dobravolskas at gmail.com> wrote:
>>> 3. Is settimeout actually working? If I run following code (please
>>> don't run it on my server):
>> OK. Everything is OK in PyPy code. timeout works (at least log
>> indicates that). There is something specific about mod_wsgi and Apache
>> in this case.
>That's what I have found out:
>pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed.
>That's how I solved problem now (modifed pypy/tool/killsubprocess.py).
>I will dig deeper of course but I will do that later.

Check out the sigprocmask Apache sets for itself.  Unfortunately, this
is inherited by all processes Apache runs, usually leading to surprising
behavior, since it prevents the delivery of almost all signals.

Jean-Paul

From arigo at tunes.org  Fri Feb 20 16:59:06 2009
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 20 Feb 2009 16:59:06 +0100
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
Message-ID: <20090220155905.GA3445@code0.codespeak.net>

Hi Dalius,

On Fri, Feb 20, 2009 at 08:30:32AM +0200, Dalius Dobravolskas wrote:
> 2. It looks like that PyPySandboxedProc expects --heapsize as first
> argument in argument list.

Bit of a hack, but yes, indeed.  It also doesn't work well with a PyPy
translated with the default GC, which is the hybrid GC.  I recommend the
generational GC (--gc=generation).  (This should really warn you...)


A bientot,

Armin.

From holger at merlinux.eu  Sat Feb 21 12:39:36 2009
From: holger at merlinux.eu (holger krekel)
Date: Sat, 21 Feb 2009 12:39:36 +0100
Subject: [pypy-dev] pytest news: plugins, pycon, release-separation
Message-ID: <20090221113936.GQ4576@trillke.net>

Hi all, 

as only some of you are also following pytest developments, 
find below some news relevant to PyPy as well i think - 
i plan to merge the new pytest into py/trunk next week
which is an svn-external for PyPy.  

If you have any feedback, i think it's ok if you send
it here to pypy-dev, particularly if it is about
pypy-related concerns. 

cheers, 
holger


Hi folks, 

some good news.  as those glimpsing at pytest svn-commits know
i am heavily working on a new plugin architecture.  I am
really happy about it as those also following me on my
brand-new twitter account (hpk42) might guess.  

Around Pycon i am giving a tutorial about advanced testing
topics including cross-platform distributed testing and 
writing plugins and i plan to talk in detail about the new and 
much improved ways of doing things. Here is the status: 

* Terminal reporting is now a nice self-contained plugin
* there is a new "xfail" plugin, inspired by SimPy to mark 
  test functions as "expected to fail" and report specially 
* a "poccoo" plugin for sending test failures to http://paste.pocoo.org
* new plugins for managing setup/teardown of application state, 
  using a turns-out-to-work-great new mechanism 
  which i guess nosetests might imitate at some point :) 

Regarding the next release i consider doing the following: 

* open two google-code projects "pytest" and "pylib" 
  with their svn repos remaining at codespeak (where
  many projects share one respository - i prefer that)

* move current py/test/ to its own "pytest/" root project 
  and depend on the py lib for install. 

* aim to do two alpha releases, 
  pytest-1.0 alpha and py lib 1.0 alpha (containing the rest)

What do you think?  I am unfamiliar with commoditized 
code hosting and consider using google code.  I'd probably
like to also have an issue tracker for pytest.  Any
comments/recommendations?  Would someone be up
for co-admining such a project?  

Meanwhile i am finishing up the pytestplugins branch
for merge to trunk, stay tuned :)

cheers & have fun,
hope to meet some of you at Pycon, 
holger

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 
_______________________________________________
py-dev mailing list
py-dev at codespeak.net
http://codespeak.net/mailman/listinfo/py-dev


----- End forwarded message -----

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From holger at merlinux.eu  Sun Feb 22 17:40:32 2009
From: holger at merlinux.eu (holger krekel)
Date: Sun, 22 Feb 2009 17:40:32 +0100
Subject: [pypy-dev] new way to organise test state / new blog
Message-ID: <20090222164032.GU4576@trillke.net>

Hi py-dev, testing-in-python and pypy-dev, 

i just created a blog where i intend to write about news
regarding testing and python and metaprogramming. 

    http://tetamap.blogspot.com/

and did a first posting about a new way of doing test state
setup with py.test.   Um, does anybody know how to do code coloring
with blogspot? 

cheers,

holger

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From jgustak at gmail.com  Sun Feb 22 18:08:16 2009
From: jgustak at gmail.com (Jakub Gustak)
Date: Sun, 22 Feb 2009 18:08:16 +0100
Subject: [pypy-dev] new way to organise test state / new blog
In-Reply-To: <20090222164032.GU4576@trillke.net>
References: <20090222164032.GU4576@trillke.net>
Message-ID: <a26b53540902220908w20a3f102u48b3a6eb92691585@mail.gmail.com>

> i just created a blog where i intend to write about news
> regarding testing and python and metaprogramming.
>
>    http://tetamap.blogspot.com/

Next blog to subscribe. Good.

> and did a first posting about a new way of doing test state
> setup with py.test.   Um, does anybody know how to do code coloring
> with blogspot?

You might want to try this out:
http://techqi.blogspot.com/2009/02/how-to-display-code-nicely-in-blogger.html
http://alexgorbatchev.com/wiki/SyntaxHighlighter

Cheers,
Jakub

From holger at merlinux.eu  Sun Feb 22 19:09:51 2009
From: holger at merlinux.eu (holger krekel)
Date: Sun, 22 Feb 2009 19:09:51 +0100
Subject: [pypy-dev] new way to organise test state / new blog
In-Reply-To: <20090222164032.GU4576@trillke.net>
References: <20090222164032.GU4576@trillke.net>
Message-ID: <20090222180951.GV4576@trillke.net>

Hello again, 

thanks to all! I'll check out the suggested code coloring options. 

note that i decided to move to wordpress already, i.e. 
 
    http://tetamap.wordpress.com/ 

because i like the interface possibilities there better. 
sorry for the noise.  please send any more comments to me 
privately and not as a cross-post. 
best,
holger

On Sun, Feb 22, 2009 at 17:40 +0100, holger krekel wrote:
> Hi py-dev, testing-in-python and pypy-dev, 
> 
> i just created a blog where i intend to write about news
> regarding testing and python and metaprogramming. 
> 
>     http://tetamap.blogspot.com/
> 
> and did a first posting about a new way of doing test state
> setup with py.test.   Um, does anybody know how to do code coloring
> with blogspot? 
> 
> cheers,
> 
> holger
> 
> -- 
> collaborative expert contracting: http://merlinux.eu 
> PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
> pylib py.test/greenlets/svn APIs: http://pylib.org 
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
> 

-- 
collaborative expert contracting: http://merlinux.eu 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
pylib py.test/greenlets/svn APIs: http://pylib.org 

From donny.viszneki at gmail.com  Sun Feb 22 19:34:42 2009
From: donny.viszneki at gmail.com (Donny Viszneki)
Date: Sun, 22 Feb 2009 13:34:42 -0500
Subject: [pypy-dev] new way to organise test state / new blog
In-Reply-To: <20090222164032.GU4576@trillke.net>
References: <20090222164032.GU4576@trillke.net>
Message-ID: <44acbb800902221034o2979c390p5ce1dc3205534fb9@mail.gmail.com>

On Sun, Feb 22, 2009 at 11:40 AM, holger krekel <holger at merlinux.eu> wrote:
> Um, does anybody know how to do code coloring
> with blogspot?

If you use vim, it's :TOhtml<enter>

-- 
http://codebad.com/

From dalius.dobravolskas at gmail.com  Sun Feb 22 20:46:58 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Sun, 22 Feb 2009 21:46:58 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com>
References: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com>
Message-ID: <49A1ABB2.3090006@gmail.com>

Jean-Paul Calderone wrote:
>> That's what I have found out:
>> pypy-c when running behind Apache can't be SIGTERMed only SIGKILLed.
>> That's how I solved problem now (modifed pypy/tool/killsubprocess.py).
>> I will dig deeper of course but I will do that later.
>>     
>
> Check out the sigprocmask Apache sets for itself.  Unfortunately, this
> is inherited by all processes Apache runs, usually leading to surprising
> behavior, since it prevents the delivery of almost all signals.
>   
I fear that everything is more complicated here. There are some 
information in mod_wsgi documentation about signals. I will ask Graham 
(mod_wsgi author) about that. He might have some ideas what to do.

Regards,
Dalius

From dalius.dobravolskas at gmail.com  Sun Feb 22 20:57:22 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Sun, 22 Feb 2009 21:57:22 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>	
	<499C084E.6040806@gmail.com>	
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>	
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
Message-ID: <49A1AE22.2090002@gmail.com>

Hello,

Dalius Dobravolskas wrote:
> 1. I'm running my code behind Apache mod_wsgi. As you might know
> mod_wsgi is very strict WSGI implementation. pypy_interact failed when
> was calling isatty function (mod_wsgi.Log does not have this
> function). I have fixed that by adding "'isatty in dir(log)". I will
> give later exact locations and names if I have confused them
Here is traceback from this situation:

Traceback (most recent call last):, referer: http://py.sandbox.lt/
   File "/home/dalius/wsgi/pypysandboxserver.py", line 25, in __call__, 
referer: http://py.sandbox.lt/
     sandproc.interact(stdout=code_output, stderr=code_output), referer: 
http://py.sandbox.lt/
   File 
"/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", 
line 307, in interact, referer: http://py.sandbox.lt/
     returncode = self.handle_until_return(), referer: http://py.sandbox.lt/
   File 
"/home/dalius/projects/pypy-trunk/pypy/translator/sandbox/sandlib.py", 
line 228, in handle_until_return, referer: http://py.sandbox.lt/
     log.call('%s(%s)' % (fnname,, referer: http://py.sandbox.lt/
   File "/home/dalius/projects/pypy-trunk/py/log/producer.py", line 55, 
in __call__, referer: http://py.sandbox.lt/
     func(self.Message(self.keywords, args)), referer: http://py.sandbox.lt/
   File "/home/dalius/projects/pypy-trunk/pypy/tool/ansi_print.py", line 
72, in __call__, referer: http://py.sandbox.lt/
     file=self.file, newline=newline, flush=flush), referer: 
http://py.sandbox.lt/
   File "/home/dalius/projects/pypy-trunk/py/io/terminalwriter.py", line 
34, in ansi_print, referer: http://py.sandbox.lt/
     if esc and sys.platform != "win32" and file.isatty():, referer: 
http://py.sandbox.lt/
 AttributeError: 'mod_wsgi.Log' object has no attribute 'isatty', 
referer: http://py.sandbox.lt/


Here is diff how I have avoided this problem:

--- io/terminalwriter.py    (revision 62015)
+++ io/terminalwriter.py    (working copy)
@@ -31,7 +31,7 @@
     if file is None:
         file = sys.stderr
     text = text.rstrip()
-    if esc and sys.platform != "win32" and file.isatty():
+    if esc and sys.platform != "win32" and 'isatty' in dir(file) and 
file.isatty():
         if not isinstance(esc, tuple):
             esc = (esc,)
         text = (''.join(['\x1b[%sm' % cod for cod in esc])  + 

Could it be fixed in the code?

Regards,
Dalius

From cfbolz at gmx.de  Mon Feb 23 16:17:36 2009
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 23 Feb 2009 16:17:36 +0100
Subject: [pypy-dev] JavaScript parser in Python
Message-ID: <49A2BE10.1080609@gmx.de>

Hi all,

in case we ever get more serious about our JS interpreter, there seems
to be a pure-Python port of the JavaScript parser of the
Narcissus-project here:

http://code.google.com/p/pynarcissus/

Automatic semicolon insertion included.

Cheers,

Carl Friedrich

From santagada at gmail.com  Tue Feb 24 01:05:20 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 23 Feb 2009 21:05:20 -0300
Subject: [pypy-dev] JavaScript parser in Python
In-Reply-To: <49A2BE10.1080609@gmx.de>
References: <49A2BE10.1080609@gmx.de>
Message-ID: <C64545A8-8835-4740-BDB0-B2A8CA563BD8@gmail.com>


On Feb 23, 2009, at 12:17 PM, Carl Friedrich Bolz wrote:

> Hi all,
>
> in case we ever get more serious about our JS interpreter, there seems
> to be a pure-Python port of the JavaScript parser of the
> Narcissus-project here:
>
> http://code.google.com/p/pynarcissus/
>
> Automatic semicolon insertion included.


There was another port of the narcissus parser, the problem being that  
the license is incompatible with the mit license. Another idea is that  
the webkit parser is made using byson, if we had a yacc or something  
like that it would be trivial to port, and I think the license is more  
permissive (or I just forgot about it and it is completely  
incompatible with the mit license).


--
Leonardo Santagada
santagada at gmail.com




From marius at pov.lt  Tue Feb 24 17:18:33 2009
From: marius at pov.lt (Marius Gedminas)
Date: Tue, 24 Feb 2009 18:18:33 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <49A1AE22.2090002@gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
	<49A1AE22.2090002@gmail.com>
Message-ID: <20090224161833.GB21714@fridge.pov.lt>

On Sun, Feb 22, 2009 at 09:57:22PM +0200, Dalius Dobravolskas wrote:

> Here is diff how I have avoided this problem:
> 
> --- io/terminalwriter.py    (revision 62015)
> +++ io/terminalwriter.py    (working copy)
> @@ -31,7 +31,7 @@
>      if file is None:
>          file = sys.stderr
>      text = text.rstrip()
> -    if esc and sys.platform != "win32" and file.isatty():
> +    if esc and sys.platform != "win32" and 'isatty' in dir(file) and 
> file.isatty():

Please use hasattr(log, 'isatty') instead.

>          if not isinstance(esc, tuple):
>              esc = (esc,)
>          text = (''.join(['\x1b[%sm' % cod for cod in esc])  + 

Marius Gedminas
-- 
I am a computer. I am dumber than any human and smarter than any administrator.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20090224/39a3cd66/attachment.pgp 

From dalius.dobravolskas at gmail.com  Wed Feb 25 06:43:19 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Wed, 25 Feb 2009 07:43:19 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <20090224161833.GB21714@fridge.pov.lt>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
	<49A1AE22.2090002@gmail.com> <20090224161833.GB21714@fridge.pov.lt>
Message-ID: <98dd087e0902242143q2257a008t6f9d6ec92b808faa@mail.gmail.com>

>> + ? ?if esc and sys.platform != "win32" and 'isatty' in dir(file) and
>> file.isatty():
>
> Please use hasattr(log, 'isatty') instead.
A?i?, Mariau. I felt that there should be more efficient way to do that ;-)

-- 
Dalius
http://blog.sandbox.lt

From dalius.dobravolskas at gmail.com  Wed Feb 25 06:49:36 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Wed, 25 Feb 2009 07:49:36 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com>
References: <98dd087e0902200113m6756d78eyf860f8b91535f4b6@mail.gmail.com>
	<20090220150552.12853.1224425473.divmod.quotient.12156@henry.divmod.com>
Message-ID: <98dd087e0902242149h320d8544sf862578ac1f9ebb@mail.gmail.com>

> Check out the sigprocmask Apache sets for itself. ?Unfortunately, this
> is inherited by all processes Apache runs, usually leading to surprising
> behavior, since it prevents the delivery of almost all signals.
FYI, I have contacted mod_wsgi author. If you are using mod_wsgi 2.X
with deamon mode SIGTERM works. The problem still persists for
embedded mode. That means there is solution that works with SIGTERM
and no changes are required in pypy code.

If you are interested how it will work in the future, please follow
this thread in modwsgi group:
http://groups.google.com/group/modwsgi/browse_thread/thread/aae8d2d675ebcf9

Thanks everyone for your help.

-- 
Dalius
http://blog.sandbox.lt

From arigo at tunes.org  Wed Feb 25 19:12:21 2009
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 25 Feb 2009 19:12:21 +0100
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <20090220155905.GA3445@code0.codespeak.net>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
	<499C8F81.6070103@gmail.com>
	<98dd087e0902192230h14c1126em95059f3e50e7cd5a@mail.gmail.com>
	<20090220155905.GA3445@code0.codespeak.net>
Message-ID: <20090225181221.GA19288@code0.codespeak.net>

Hi,

On Fri, Feb 20, 2009 at 04:59:06PM +0100, Armin Rigo wrote:
> Bit of a hack, but yes, indeed.  It also doesn't work well with a PyPy
> translated with the default GC, which is the hybrid GC.  I recommend the
> generational GC (--gc=generation).  (This should really warn you...)

Ah, I just found a hack in pypy/config/translationoption.py.  The GC
defaults to 'generation' instead of 'hybrid' in case of a sandbox
translation.  So everything is fine :-)


A bientot,

Armin.

From santagada at gmail.com  Thu Feb 26 15:08:51 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 26 Feb 2009 11:08:51 -0300
Subject: [pypy-dev] Parsing in PyPy (and runicode)
Message-ID: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>

I remember some time ago people on #pypy were talking about redoing  
the parser for python because it was not good enough or something.  
Also the parser generator that cfbolz wrote doesn't support unicode  
and was not suited for automatic-semicolon-insertion.

I think it would be good for a javascript parser that supports unicode  
because the specs call for it and maybe it would be good to python too  
(don't know about prolog/smalltalk though). What would be better, to  
have a parser generator that supports unicode or just everyone write  
their own recursive descendant parser by hand?


--
Leonardo Santagada
santagada at gmail.com




From cfbolz at gmx.de  Thu Feb 26 18:40:08 2009
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 26 Feb 2009 18:40:08 +0100
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
Message-ID: <49A6D3F8.3020501@gmx.de>

Hi Leonardo,

Leonardo Santagada wrote:
> I remember some time ago people on #pypy were talking about redoing  
> the parser for python because it was not good enough or something.  
> Also the parser generator that cfbolz wrote doesn't support unicode  
> and was not suited for automatic-semicolon-insertion.
> 
> I think it would be good for a javascript parser that supports unicode  
> because the specs call for it and maybe it would be good to python too  
> (don't know about prolog/smalltalk though). What would be better, to  
> have a parser generator that supports unicode or just everyone write  
> their own recursive descendant parser by hand?

Are you sure that we shouldn't rather try to steal one of the existing
JS parsers? If license really is a problem, maybe we could ask the
authors whether they would be fine with relicensing under MIT.

Cheers,

Carl Friedrich

From santagada at gmail.com  Thu Feb 26 19:29:15 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 26 Feb 2009 15:29:15 -0300
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <49A6D3F8.3020501@gmx.de>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
Message-ID: <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>


On Feb 26, 2009, at 2:40 PM, Carl Friedrich Bolz wrote:

> Hi Leonardo,
>
> Leonardo Santagada wrote:
>> I remember some time ago people on #pypy were talking about redoing
>> the parser for python because it was not good enough or something.
>> Also the parser generator that cfbolz wrote doesn't support unicode
>> and was not suited for automatic-semicolon-insertion.
>>
>> I think it would be good for a javascript parser that supports  
>> unicode
>> because the specs call for it and maybe it would be good to python  
>> too
>> (don't know about prolog/smalltalk though). What would be better, to
>> have a parser generator that supports unicode or just everyone write
>> their own recursive descendant parser by hand?
>
> Are you sure that we shouldn't rather try to steal one of the existing
> JS parsers? If license really is a problem, maybe we could ask the
> authors whether they would be fine with relicensing under MIT.


All the ones that come from narcissus are actually just licensed by  
Mozilla, so we should talk to them. But I don't know if it is complete  
and without bugs (I think there was one with narcissus but I don't  
remember what was it) so maybe we should use one used by a functioning  
js interpreters like Rhino of JavascriptCore (Squirrelfish Extreme or  
whatever its name is now). Both uses an incompatible license, and I  
think it is on purpose... so I don't know how to deal with this.

The V8 parser is BSD so I think it is compatible, but it will be some  
work to convert it from c++ to rpython (http://code.google.com/p/v8/source/browse/trunk/src/parser.cc 
  )

My final answer would be, "I don't know, what do you guys think?".

--
Leonardo Santagada
santagada at gmail.com




From jbaker at zyasoft.com  Thu Feb 26 19:44:07 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Thu, 26 Feb 2009 11:44:07 -0700
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
Message-ID: <d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>

Perhaps consider Antlr? We've had good success with this for Jython, and
it's now also used by NetBeans support for Python in general. I took a look
at http://www.antlr.org/grammar/list, and there are a number of options for
JS. Most importantly, Antlr supports the parser chain in Python, so it's
possible this could be more readily converted to RPython.
Some potential issues:
Chris Lambrou has a parser for EcmaScript 3.0. But there's no license here,
so you'd definitely have to contact him on this. Like standalone grammars in
general, it's rather unlikely to have been extensively tested. With Jython
we started with a reference parser that Terrence Parr had made, and taking
CPython as reference here as to what the AST should be, over time we
targeted that by explicitly comparing ASTS. Some time later, including
incremental parse support and various syntax errors, we're pretty confident
that it's basically done.

- Jim

On Thu, Feb 26, 2009 at 11:29 AM, Leonardo Santagada <santagada at gmail.com>wrote:

>
> On Feb 26, 2009, at 2:40 PM, Carl Friedrich Bolz wrote:
>
> > Hi Leonardo,
> >
> > Leonardo Santagada wrote:
> >> I remember some time ago people on #pypy were talking about redoing
> >> the parser for python because it was not good enough or something.
> >> Also the parser generator that cfbolz wrote doesn't support unicode
> >> and was not suited for automatic-semicolon-insertion.
> >>
> >> I think it would be good for a javascript parser that supports
> >> unicode
> >> because the specs call for it and maybe it would be good to python
> >> too
> >> (don't know about prolog/smalltalk though). What would be better, to
> >> have a parser generator that supports unicode or just everyone write
> >> their own recursive descendant parser by hand?
> >
> > Are you sure that we shouldn't rather try to steal one of the existing
> > JS parsers? If license really is a problem, maybe we could ask the
> > authors whether they would be fine with relicensing under MIT.
>
>
> All the ones that come from narcissus are actually just licensed by
> Mozilla, so we should talk to them. But I don't know if it is complete
> and without bugs (I think there was one with narcissus but I don't
> remember what was it) so maybe we should use one used by a functioning
> js interpreters like Rhino of JavascriptCore (Squirrelfish Extreme or
> whatever its name is now). Both uses an incompatible license, and I
> think it is on purpose... so I don't know how to deal with this.
>
> The V8 parser is BSD so I think it is compatible, but it will be some
> work to convert it from c++ to rpython (
> http://code.google.com/p/v8/source/browse/trunk/src/parser.cc
>  )
>
> My final answer would be, "I don't know, what do you guys think?".
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090226/6d34d67f/attachment-0001.htm 

From santagada at gmail.com  Thu Feb 26 20:55:11 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 26 Feb 2009 16:55:11 -0300
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
	<d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
Message-ID: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>


On Feb 26, 2009, at 3:44 PM, Jim Baker wrote:

> Perhaps consider Antlr? We've had good success with this for Jython,  
> and it's now also used by NetBeans support for Python in general. I  
> took a look at http://www.antlr.org/grammar/list, and there are a  
> number of options for JS. Most importantly, Antlr supports the  
> parser chain in Python, so it's possible this could be more readily  
> converted to RPython.

Yep this could be done, if so I would use this grammar file http://research.xebic.com/es3/ 
  and then put the code for JS 1.5+ in it as we start supporting those  
features.


> Some potential issues:
> Chris Lambrou has a parser for EcmaScript 3.0. But there's no  
> license here, so you'd definitely have to contact him on this. Like  
> standalone grammars in general, it's rather unlikely to have been  
> extensively tested. With Jython we started with a reference parser  
> that Terrence Parr had made, and taking CPython as reference here as  
> to what the AST should be, over time we targeted that by explicitly  
> comparing ASTS. Some time later, including incremental parse support  
> and various syntax errors, we're pretty confident that it's  
> basically done.


The thing that would be great is if pypy and jython would use the same  
parser using antlr so the work to support python 3.0 (and 2.7, 2.8,  
etc) could be partially shared :)

--
Leonardo Santagada
santagada at gmail.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090226/34e15d27/attachment.htm 

From jbaker at zyasoft.com  Thu Feb 26 21:18:53 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Thu, 26 Feb 2009 13:18:53 -0700
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
	<d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
Message-ID: <d03bb4010902261218x2d86a75fie6999b989152ece7@mail.gmail.com>

Please feel free to use our Python parser, it's licensed under the original
BSD license from Terrence Parr.
https://jython.svn.sourceforge.net/svnroot/jython/trunk/jython/grammar/

I don't know where the 3.0 work is currently located, in any event it's
something we have put on hold for the moment as we get the 2.5 release out.
I cc-ed in Frank to ensure he is in the loop here.

In terms of the parser itself: there's a modest amount of Java code in the
actions, but that should be easy to convert. Supporting RPython generation
in Antlr then makes even more sense in this case: as I understand it, Antlr
uses TP's other project, StringTemplate, to simplify the construction of
multiple backends.

- Jim

On Thu, Feb 26, 2009 at 12:55 PM, Leonardo Santagada <santagada at gmail.com>wrote:

>
> On Feb 26, 2009, at 3:44 PM, Jim Baker wrote:
>
> Perhaps consider Antlr? We've had good success with this for Jython, and
> it's now also used by NetBeans support for Python in general. I took a look
> at http://www.antlr.org/grammar/list, and there are a number of options
> for JS. Most importantly, Antlr supports the parser chain in Python, so it's
> possible this could be more readily converted to RPython.
>
>
> Yep this could be done, if so I would use this grammar file
> http://research.xebic.com/es3/ and then put the code for JS 1.5+ in it as
> we start supporting those features.
>
>
> Some potential issues:
> Chris Lambrou has a parser for EcmaScript 3.0. But there's no license here,
> so you'd definitely have to contact him on this. Like standalone grammars in
> general, it's rather unlikely to have been extensively tested. With Jython
> we started with a reference parser that Terrence Parr had made, and taking
> CPython as reference here as to what the AST should be, over time we
> targeted that by explicitly comparing ASTS. Some time later, including
> incremental parse support and various syntax errors, we're pretty confident
> that it's basically done.
>
>
> The thing that would be great is if pypy and jython would use the same
> parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc)
> could be partially shared :)
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
>


-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090226/e2f668b3/attachment.htm 

From dalius.dobravolskas at gmail.com  Fri Feb 27 09:15:25 2009
From: dalius.dobravolskas at gmail.com (Dalius Dobravolskas)
Date: Fri, 27 Feb 2009 10:15:25 +0200
Subject: [pypy-dev] pypy sandbox
In-Reply-To: <98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
References: <98dd087e0902180501s3ea9bcf0i6596d5f243941da8@mail.gmail.com>
	<499C084E.6040806@gmail.com>
	<98dd087e0902181239w47b96c2cqe40e1f6cd49e24f4@mail.gmail.com>
Message-ID: <98dd087e0902270015q3d2b75fbq992308f3e0da032@mail.gmail.com>

Hello,

On Wed, Feb 18, 2009 at 10:39 PM, Dalius Dobravolskas
<dalius.dobravolskas at gmail.com> wrote:
> I will try to implement WSGI sample similar to Django (as mentioned
> here http://morepypy.blogspot.com/2009/02/wroclaw-2009-sprint-progress-report.html).

So here is final result:
http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox

Thank you everyone for your help. I think I will play around more with PyPy.

-- 
Dalius
http://blog.sandbox.lt

From holger at merlinux.eu  Fri Feb 27 12:45:20 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 27 Feb 2009 12:45:20 +0100
Subject: [pypy-dev] pypy/doc changes / py.test plugins
Message-ID: <20090227114520.GC4576@trillke.net>

Hi all, 

i just merged the pytest plugin branch to py/trunk which 
is used by pypy/trunk and branches.  I needed to fix
pypy/trunk/doc/* to work with the new pytest_restdoc 
plugin, otherwise i hope that everythings just continues 
to work.  But to avoid deprecation warnings you might
want to apply revision 62212 and 62219 to the branches. 

If you are interested, you might read up on pytest plugins here: 

http://tetamap.wordpress.com/2009/02/27/new-plugin-architecture-and-plugins-for-pytest/

cheers,
holger

-- 
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu 

From fwierzbicki at gmail.com  Fri Feb 27 15:42:06 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 27 Feb 2009 09:42:06 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
	<d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
Message-ID: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>

On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada <santagada at gmail.com> wrote:
> The thing that would be great is if pypy and jython would use the same
> parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc)
> could be partially shared :)
This would indeed be very cool.  Also, ANTLR 3.x supports a really
interesting form of grammar inheritance which would help us share a
base grammar (I remember it has a sort of diff-merge form of
inheritance, my google skills are failing me, I'll find a reference
today sometime I'm sure).  At the JVM Language summit last year, I met
ANTLR expert Jim Idle, and he expressed an interest in seeing if the
Jython grammar could be used as a grammar for CPython.  I've copied
Jim Idle on this email.

As a side note, it appears that Guido van Rossum has had some positive
experiences with ANTLR recently:

http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html

-Frank

From fwierzbicki at gmail.com  Fri Feb 27 15:46:41 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 27 Feb 2009 09:46:41 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
	<d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
Message-ID: <4dab5f760902270646r50c8e72id54b039ab5ac12de@mail.gmail.com>

On Thu, Feb 26, 2009 at 3:18 PM, Jim Baker <jbaker at zyasoft.com> wrote:
> I don't know where the 3.0 work is currently located, in any event it's
> something we have put on hold for the moment as we get the 2.5 release out.
> I cc-ed in Frank to ensure he is in the loop here.
The 3.0 grammar is here:
https://jython.svn.sourceforge.net/svnroot/jython/branches/jy3k/grammar

Some light testing with AST comparisons shows that it is pretty good,
but does not yet support the extended function syntax (like keyword
only args)

-Frank

From fwierzbicki at gmail.com  Fri Feb 27 15:59:31 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 27 Feb 2009 09:59:31 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<49A6D3F8.3020501@gmx.de>
	<8922F008-F165-4A9A-97B2-F3FCB541C78E@gmail.com>
	<d03bb4010902261044p121dfa6wd61990c742be6650@mail.gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
Message-ID: <4dab5f760902270659w5448a729nd768d33710143bd7@mail.gmail.com>

On Fri, Feb 27, 2009 at 9:42 AM, Frank Wierzbicki <fwierzbicki at gmail.com> wrote:
> (I remember it has a sort of diff-merge form of
> inheritance, my google skills are failing me, I'll find a reference
> today sometime I'm sure).
I found a paper (http://www.cs.vu.nl/icpc2008/docs/Parr.pdf) but I'm
not sure if this has made it into official 3.x yet.

-Frank

From jacob at openend.se  Fri Feb 27 19:19:03 2009
From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Fri, 27 Feb 2009 19:19:03 +0100
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
Message-ID: <200902271919.03717.jacob@openend.se>

fredagen den 27 februari 2009 skrev Frank Wierzbicki:
> On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada <santagada at gmail.com> 
wrote:
> > The thing that would be great is if pypy and jython would use the same
> > parser using antlr so the work to support python 3.0 (and 2.7, 2.8, etc)
> > could be partially shared :)
>
> This would indeed be very cool.  Also, ANTLR 3.x supports a really
> interesting form of grammar inheritance which would help us share a
> base grammar (I remember it has a sort of diff-merge form of
> inheritance, my google skills are failing me, I'll find a reference
> today sometime I'm sure).  At the JVM Language summit last year, I met
> ANTLR expert Jim Idle, and he expressed an interest in seeing if the
> Jython grammar could be used as a grammar for CPython.  I've copied
> Jim Idle on this email.
>
> As a side note, it appears that Guido van Rossum has had some positive
> experiences with ANTLR recently:
>
> http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html
>
> -Frank
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

Andrew Dalke, who is very thorough in his investigation of software, has 
written som interesting things about his experience with ANTLR as well as 
some other parsing projects. In short, he likes ANTLR as a tool, but in his 
application, it is considerably slower than some other alternatives.
He also has something called python4ply, which is a ready, MIT licensed
parser for Python.

You can find his articles on
http://www.dalkescientific.com/writings/diary/archive/


Jacob Hall?n

From santagada at gmail.com  Fri Feb 27 19:37:10 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 27 Feb 2009 15:37:10 -0300
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <200902271919.03717.jacob@openend.se>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
Message-ID: <693E7D78-65B6-4680-A34D-E4999915B172@gmail.com>


On Feb 27, 2009, at 3:19 PM, Jacob Hall?n wrote:

> fredagen den 27 februari 2009 skrev Frank Wierzbicki:
>> On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada <santagada at gmail.com 
>> >
> wrote:
>>> The thing that would be great is if pypy and jython would use the  
>>> same
>>> parser using antlr so the work to support python 3.0 (and 2.7,  
>>> 2.8, etc)
>>> could be partially shared :)
>>
>> This would indeed be very cool.  Also, ANTLR 3.x supports a really
>> interesting form of grammar inheritance which would help us share a
>> base grammar (I remember it has a sort of diff-merge form of
>> inheritance, my google skills are failing me, I'll find a reference
>> today sometime I'm sure).  At the JVM Language summit last year, I  
>> met
>> ANTLR expert Jim Idle, and he expressed an interest in seeing if the
>> Jython grammar could be used as a grammar for CPython.  I've copied
>> Jim Idle on this email.
>>
>> As a side note, it appears that Guido van Rossum has had some  
>> positive
>> experiences with ANTLR recently:
>>
>> http://www.antlr.org/pipermail/antlr-interest/2009-February/032783.html
>>
>> -Frank
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
> Andrew Dalke, who is very thorough in his investigation of software,  
> has
> written som interesting things about his experience with ANTLR as  
> well as
> some other parsing projects. In short, he likes ANTLR as a tool, but  
> in his
> application, it is considerably slower than some other alternatives.
> He also has something called python4ply, which is a ready, MIT  
> licensed
> parser for Python.
>
> You can find his articles on
> http://www.dalkescientific.com/writings/diary/archive/


The problem he might be having is with the python backend for ANTLR,  
wich neither us (we are going to have to create a rpython one) nor  
cpython (which would use a c89 one) would have. but this is just a  
guess as I have had no time to read his article yet

--
Leonardo Santagada
santagada at gmail.com




From fwierzbicki at gmail.com  Fri Feb 27 19:38:39 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 27 Feb 2009 13:38:39 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <200902271919.03717.jacob@openend.se>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
Message-ID: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com>

On Fri, Feb 27, 2009 at 1:19 PM, Jacob Hall?n <jacob at openend.se> wrote:
> Andrew Dalke, who is very thorough in his investigation of software, has
> written som interesting things about his experience with ANTLR as well as
> some other parsing projects. In short, he likes ANTLR as a tool, but in his
> application, it is considerably slower than some other alternatives.
> He also has something called python4ply, which is a ready, MIT licensed
> parser for Python.
>
> You can find his articles on
> http://www.dalkescientific.com/writings/diary/archive/
I see the part you are likely talking about:

"""
It looks like every character read incurs several Python function
calls, which are a lot more expensive in Python than in Java or C++.
There's no easy change for this, so I'm pretty sure the
ANTLR-generated parser always going to be slower than PLY.
"""

I wonder if producing a parser in C would work for PyPy if this is
unavoidable?  I know it misses the purpose of PyPy a bit -- just a
thought :)

-Frank

From fwierzbicki at gmail.com  Fri Feb 27 19:40:45 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Fri, 27 Feb 2009 13:40:45 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <693E7D78-65B6-4680-A34D-E4999915B172@gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
	<693E7D78-65B6-4680-A34D-E4999915B172@gmail.com>
Message-ID: <4dab5f760902271040o410aa987te0e62fd3f561e44b@mail.gmail.com>

On Fri, Feb 27, 2009 at 1:37 PM, Leonardo Santagada <santagada at gmail.com> wrote:
> The problem he might be having is with the python backend for ANTLR, wich
> neither us (we are going to have to create a rpython one) nor cpython (which
> would use a c89 one) would have.
Ah! You are thinking of generating an rpython backend -- interesting.

-Frank

From arigo at tunes.org  Fri Feb 27 20:19:46 2009
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 27 Feb 2009 20:19:46 +0100
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
	<4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com>
Message-ID: <20090227191945.GA4243@code0.codespeak.net>

Hi,

On Fri, Feb 27, 2009 at 01:38:39PM -0500, Frank Wierzbicki wrote:
> """
> It looks like every character read incurs several Python function
> calls, which are a lot more expensive in Python than in Java or C++.
> There's no easy change for this, so I'm pretty sure the
> ANTLR-generated parser always going to be slower than PLY.
> """
> 
> I wonder if producing a parser in C would work for PyPy if this is
> unavoidable?  I know it misses the purpose of PyPy a bit -- just a
> thought :)

No, our goal would be to generate an RPython parser, not just a Python
one.  A few function calls are not more of a problem for RPython than
they are for C.  So that should be fine.  In other words, if ANTLR
generates parsers expecting the speed of a Java or C++ kind of backend,
then an RPython backend is no problem either.


A bientot,

Armin.

From holger at merlinux.eu  Fri Feb 27 20:19:08 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 27 Feb 2009 20:19:08 +0100
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
	<4dab5f760902271038i162443b9g5fbab02292c8d162@mail.gmail.com>
Message-ID: <20090227191908.GL4576@trillke.net>

On Fri, Feb 27, 2009 at 13:38 -0500, Frank Wierzbicki wrote:
> On Fri, Feb 27, 2009 at 1:19 PM, Jacob Hall?n <jacob at openend.se> wrote:
> > Andrew Dalke, who is very thorough in his investigation of software, has
> > written som interesting things about his experience with ANTLR as well as
> > some other parsing projects. In short, he likes ANTLR as a tool, but in his
> > application, it is considerably slower than some other alternatives.
> > He also has something called python4ply, which is a ready, MIT licensed
> > parser for Python.
> >
> > You can find his articles on
> > http://www.dalkescientific.com/writings/diary/archive/
> I see the part you are likely talking about:
> 
> """
> It looks like every character read incurs several Python function
> calls, which are a lot more expensive in Python than in Java or C++.
> There's no easy change for this, so I'm pretty sure the
> ANTLR-generated parser always going to be slower than PLY.
> """
> 
> I wonder if producing a parser in C would work for PyPy if this is
> unavoidable?  I know it misses the purpose of PyPy a bit -- just a
> thought :)

RPython calls are not much more expensive than C calls. 
So an RPython backend would not suffer. 

holger

From jurgis.pralgauskis at gmail.com  Sat Feb 28 09:09:20 2009
From: jurgis.pralgauskis at gmail.com (Jurgis Pralgauskis)
Date: Sat, 28 Feb 2009 10:09:20 +0200
Subject: [pypy-dev] pypy translation error :\
Message-ID: <34f4097d0902280009p17e54432ked63e3542b271539@mail.gmail.com>

Hello,

maybe smb could give a hint, why:
http://files.akl.lt/users/jurgis/etc/pypy_translation_error.txt

ps.: I acted according to instructions on
http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox
my env: python 2.5.2, Ubuntu 8.04


Thanks in advance
-- 
Jurgis Pralgauskis
Don't worry, be happy and make things better ;)
http://sagemath.visiems.lt

From anto.cuni at gmail.com  Sat Feb 28 19:28:31 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 28 Feb 2009 19:28:31 +0100
Subject: [pypy-dev] holydays
Message-ID: <49A9824F.20706@gmail.com>

Hi all,
I'm going on holydays next week (skiing :-)), so I won't be online until the 9th.

ciao,
Anto

From nytrokiss at gmail.com  Sat Feb 28 20:03:37 2009
From: nytrokiss at gmail.com (James Matthews)
Date: Sat, 28 Feb 2009 21:03:37 +0200
Subject: [pypy-dev] holydays
In-Reply-To: <49A9824F.20706@gmail.com>
References: <49A9824F.20706@gmail.com>
Message-ID: <8a6b8e350902281103r76424ddfv50f7707a95a7b023@mail.gmail.com>

Holidays or Holy Days :)? Enjoy your vacation!

On Sat, Feb 28, 2009 at 8:28 PM, Antonio Cuni <anto.cuni at gmail.com> wrote:

> Hi all,
> I'm going on holydays next week (skiing :-)), so I won't be online until
> the 9th.
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.astorandblack.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090228/a3ff392b/attachment.htm 

From fijall at gmail.com  Sun Mar  1 16:50:57 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 1 Mar 2009 16:50:57 +0100
Subject: [pypy-dev] pypy translation error :\
In-Reply-To: <34f4097d0902280009p17e54432ked63e3542b271539@mail.gmail.com>
References: <34f4097d0902280009p17e54432ked63e3542b271539@mail.gmail.com>
Message-ID: <693bc9ab0903010750g40fc9df6w57a6a43dfb19e3df@mail.gmail.com>

thanks for the report, should be fixed by now.

On Sat, Feb 28, 2009 at 9:09 AM, Jurgis Pralgauskis
<jurgis.pralgauskis at gmail.com> wrote:
> Hello,
>
> maybe smb could give a hint, why:
> http://files.akl.lt/users/jurgis/etc/pypy_translation_error.txt
>
> ps.: I acted according to instructions on
> http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox
> my env: python 2.5.2, Ubuntu 8.04
>
>
> Thanks in advance
> --
> Jurgis Pralgauskis
> Don't worry, be happy and make things better ;)
> http://sagemath.visiems.lt
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From santagada at gmail.com  Mon Mar  2 16:00:03 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 2 Mar 2009 12:00:03 -0300
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <49A86AEB.6080902@temporal-wave.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
	<693E7D78-65B6-4680-A34D-E4999915B172@gmail.com>
	<49A86AEB.6080902@temporal-wave.com>
Message-ID: <6A9BE8C6-1FF1-4970-B319-59BBFBD0F571@gmail.com>


On Feb 27, 2009, at 7:36 PM, Jim Idle wrote:

> Leonardo Santagada wrote:
>>
>> On Feb 27, 2009, at 3:19 PM, Jacob Hall?n wrote:
>>
>>> fredagen den 27 februari 2009 skrev Frank Wierzbicki:
>>>> On Thu, Feb 26, 2009 at 2:55 PM, Leonardo Santagada <santagada at gmail.com 
>>>> >
>>>
>>> Andrew Dalke, who is very thorough in his investigation of  
>>> software, has
>>> written som interesting things about his experience with ANTLR as  
>>> well as
>>> some other parsing projects. In short, he likes ANTLR as a tool,  
>>> but in his
>>> application, it is considerably slower than some other alternatives.
>>> He also has something called python4ply, which is a ready, MIT  
>>> licensed
>>> parser for Python.
>>>
>>> You can find his articles on
>>> http://www.dalkescientific.com/writings/diary/archive/
>>
>>
>> The problem he might be having is with the python backend for  
>> ANTLR, wich neither us (we are going to have to create a rpython  
>> one) nor cpython (which would use a c89 one) would have. but this  
>> is just a guess as I have had no time to read his article yet,
> All I can find is info about using the Python backend.  
> Unfortunately, the Python backend is very slow. There was some  
> discussion between the Python runtime author and Guido about why - I  
> can't re-quote as it was private email, but basically the runtime  
> and the generated code are method-call heavy, which isn't a good  
> idea in Python. Also, as string handling  and other things are not  
> particularly quick, it is hard to get Python to perform when running  
> a parser using a design that wasn't specifically tailored for  
> Python. After all, Python wasn't really aimed at writing things like  
> lexers and parsers and is much better at things in other domains.  
> All that said, I think that the Python runtime will get better as  
> there will be more expansive choices for backend runtime authors in  
> the future. Then again, is the speed of the parser going to be a  
> factor?

Not much I think. Well I did take a look at both the support code and  
the generated code (and a fast look at the runtime). The support code  
in java is really crazy, at least for me I didn't get most of it (I  
should read the docs later).

Now the generated code seems to be following the java backend so far  
as being almost RPython. there is a problem that RPython doesn't have  
sets so it will take some time to make it work... but I think it is  
doable. Somehow my generated code has "pass" before every block of  
code on both parsers and lexers, the reason for that is still a  
mistery for me (do anyone knows why?).

Maybe in this weekend I will have more time to look/work more seriosly  
on this.


> The Java runtime is a lot quicker that the Python runtime and unless  
> there are Python translation units with 25,000 lines (there will be  
> somewhere ;-), performance would not be a factor.
>
> When I wrote the C runtime however, I did not make a blind copy of  
> the Java runtime, hence the performance is akin to hand written  
> code. For instance the GNU C parser written for ANTLR and running  
> with the C runtime, is almost the same speed as the GNU C parser  
> itself.

This is great, but is the code C89 or do you use something from a  
newer standard? Because the only way to have a shot at using it with  
cpython would be if it was C89... though using the same parser in  
jython and pypy would be cool enough.

> None of this would help (in terms of ANTLR) if you want a Python  
> parser that runs in Python of course :-)


Well knowing that the Java one is quick is a good indication that the  
rpython one can be quick also.

Thanks for all the info and for the quick response,
--
Leonardo Santagada
santagada at gmail.com




From arigo at tunes.org  Mon Mar  2 16:47:14 2009
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 2 Mar 2009 16:47:14 +0100
Subject: [pypy-dev] Leysin Winter Sprint
Message-ID: <20090302154714.GA21804@code0.codespeak.net>

Hi all!


=====================================================================
         PyPy Leysin Winter Sprint (14-21th April 2009)
=====================================================================
..                             image:: http://www.ermina.ch/002.JPG

The next PyPy sprint will be in Leysin, Switzerland, for the
sixth time.  This sprint will take place immediately after
Easter.  This is a fully public sprint: newcomers and topics
other than those proposed below are welcome.  

------------------------------
Goals and topics of the sprint
------------------------------

* The overall idea of the sprint is to continue working on making PyPy ready
  for general use.  There are a few tasks left in there.  In parallel, we
  will continue the work on the JIT, if there is general interest.  And as
  usual, we are ready to add any other task -- please mention on the mailing
  list what you would like to work on; the list of task is not really fixed.

* And as usual, the main side goal is to have fun in winter sports :-)
  We can take a day off for ski until Sunday, the 19th; afterwards, the
  installations close.  (There was quite a lot of snow this winter, so
  there should be some left even though it's relatively late in the season.)

-----------------------
Location & Accomodation
-----------------------

Leysin, Switzerland, "same place as before".  Let me refresh your
memory: both the sprint venue and the lodging will be in a very spacious
pair of chalets built specifically for bed & breakfast:
http://www.ermina.ch/.  The place has a good ADSL Internet connexion
with wireless installed.  You can of course arrange your own
lodging anywhere (so long as you are in Leysin, you cannot be more than a
15 minutes walk away from the sprint venue), but I definitely recommend
lodging there too -- you won't find a better view anywhere else (though you
probably won't get much worse ones easily, either :-)

Please *confirm* that you are coming so that we can adjust the reservations
as appropriate.  The rate so far has been around 60 CHF a night all included
in 2-person rooms, with breakfast.  There are larger rooms too (less
expensive) and maybe the possibility to get a single room if you really want
to.

Please register by svn:

  http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2009/people.txt

or on the pypy-sprint mailing list if you do not yet have check-in rights:

  http://codespeak.net/mailman/listinfo/pypy-sprint

You need a Swiss-to-(insert country here) power adapter.  There will be
some Swiss-to-EU adapters around -- bring a EU-format power strip if you
have one.

-----------
Exact times
-----------

Officially, 14-21 April 2009.  The idea is that the 14th should be the arrival
day, so we won't work too much until the 15th.

-----------------------------------------------------------------------

Armin

From fwierzbicki at gmail.com  Mon Mar  2 21:00:19 2009
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Mon, 2 Mar 2009 15:00:19 -0500
Subject: [pypy-dev] Parsing in PyPy (and runicode)
In-Reply-To: <49AC0F89.7040503@temporal-wave.com>
References: <EA020D2E-2057-4B9E-8343-B96B6BBBD959@gmail.com>
	<6CA22A60-3D3F-44A2-A121-23B0CA8FCC12@gmail.com>
	<4dab5f760902270642y526129e3s2fcee99eff45579b@mail.gmail.com>
	<200902271919.03717.jacob@openend.se>
	<693E7D78-65B6-4680-A34D-E4999915B172@gmail.com>
	<49A86AEB.6080902@temporal-wave.com>
	<6A9BE8C6-1FF1-4970-B319-59BBFBD0F571@gmail.com>
	<49AC0F89.7040503@temporal-wave.com>
Message-ID: <4dab5f760903021200j11c31023r3a768ec98d3c0994@mail.gmail.com>

On Mon, Mar 2, 2009 at 11:55 AM, Jim Idle <jimi at temporal-wave.com> wrote:
>> Well knowing that the Java one is quick is a good indication that the
>> rpython one can be quick also.
>
> Yes - I think so. Perhaps one way is to ignore the current runtime, which
> has been written more for functionality than performance as far as I can
> see, and write an RPython runtime that is strictly aimed at performance and
> RPython semantics etc. It is not a huge project to do this, but it is some
> work of course.
Indeed the Java parser has not been optimized for performance (yet) --
it was written with compatibility as the #1 concern.  In particular
I'm sure I've lost a bit of performance in order to support _ast.py
and ast.py.

-Frank

From santagada at gmail.com  Wed Mar  4 02:38:40 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 3 Mar 2009 22:38:40 -0300
Subject: [pypy-dev] Compiling RSDL and ExternalCompilationInfo
Message-ID: <C5296938-27EE-4CF4-9C3C-D020862534AB@gmail.com>

I've tried running some RSDL examples in my computer and they are not  
working, but if I try to compile the code by hand it works (without  
the framework and includes it does complain the same as translation on  
pypy):

Apfelstrudel:~/projects/pypy/pypy/rlib/rsdl/macosx-sdl-main santagada$  
gcc -I/Library/Frameworks/SDL.framework/Headers -framework Cocoa - 
framework SDL SDLMain.m
Undefined symbols:
   "_SDL_main", referenced from:
       -[SDLMain applicationDidFinishLaunching:] in ccCdkKBZ.o
ld: symbol(s) not found
collect2: ld returned 1 exit status

The error is because the supporting code does not have a main (I  
think). But when I try to translate something with pypy I get this  
other error on this paste http://paste.pocoo.org/show/106398/ wich  
ends with:

pypy.translator.platform.CompilationError: <CompilationError  
err='Undefined symbols:\n  "_CPSEnableForegroundOperation", referenced  
from:\n      _CustomApplicationMain in cckWdEVi.o\n...erenced from: 
\n      _NSApp$non_lazy_ptr in cckWdEVi.o\nld: symbol(s) not found 
\ncollect2: ld returned 1 exit status\n'>
Apfelstrudel:~/projects/pypy/pypy/rlib/rsdl santagada$

Using py.test to run the example it stops saying I don't have sdl  
installed (the error defined on the check_sdl_installation in eci.py.

The eci.py file should be defining it but besides Version/A/Headers  
(which I also tried changing on my WC) appears to be trying to define  
what I typed to gcc. What could be going on?

--
Leonardo Santagada
santagada at gmail.com




From pedronis at openend.se  Thu Mar  5 23:57:26 2009
From: pedronis at openend.se (Samuele Pedroni)
Date: Thu, 05 Mar 2009 23:57:26 +0100
Subject: [pypy-dev] cleaned builds with lots of failures,
	issues with nightly translations
Message-ID: <49B058D6.70206@openend.se>

I cleaned up the builds with lots of failures that happened on 3rd.

Btw build files are stored with names starting with build numbers in the 
builder directories (named like the builders) under 
/home/buildmaster/pypy on codespeak.

It seems that some our nightly translations by buildbot are timing out 
now, it may be legitimate and then we need to up the timeouts or 
something wrong may be going on,
I don't have time to investigate that right now.

Samuele


From holger at merlinux.eu  Fri Mar  6 10:22:52 2009
From: holger at merlinux.eu (holger krekel)
Date: Fri, 6 Mar 2009 10:22:52 +0100
Subject: [pypy-dev] cleaned builds with lots of failures,
	issues with	nightly translations
In-Reply-To: <49B058D6.70206@openend.se>
References: <49B058D6.70206@openend.se>
Message-ID: <20090306092252.GZ4576@trillke.net>

Hi Samuele, 

On Thu, Mar 05, 2009 at 23:57 +0100, Samuele Pedroni wrote:
> I cleaned up the builds with lots of failures that happened on 3rd.

great, thanks. 
 
> Btw build files are stored with names starting with build numbers in the 
> builder directories (named like the builders) under 
> /home/buildmaster/pypy on codespeak.

ok. what's the best way to find out the svn revision those build numbers 
belong to? 
 
> It seems that some our nightly translations by buildbot are timing out 
> now, it may be legitimate and then we need to up the timeouts or 
> something wrong may be going on,
> I don't have time to investigate that right now.
>

at least http://tinyurl.com/dcbn8v indicates   that our
lib-python tests ran against a recent pypy-c, with only three
failures out of 250 cpython regression test files. 
didn'T look further. 

holger

-- 
Metaprogramming, Python, Testing: http://tetamap.wordpress.com
Python, PyPy, pytest contracting: http://merlinux.eu 

From jurgis.pralgauskis at gmail.com  Sun Mar  8 00:24:27 2009
From: jurgis.pralgauskis at gmail.com (Jurgis Pralgauskis)
Date: Sun, 8 Mar 2009 01:24:27 +0200
Subject: [pypy-dev] sandboxed exec function with context passing?
Message-ID: <34f4097d0903071524n16ebbfet98f87a61a7bb48ff@mail.gmail.com>

I would like to use sandboxed pypy for online step by step tutorials...
1) I need ability to pass context then.
2) and also would be nice to get result as variable not just like output.

my experiments are based on explanations from
http://blog.sandbox.lt/en/WSGI%20and%20PyPy%20sandbox

maybe my question seems too trivial,
but I looked around  pypy_interact code, its class inherits from 2 calsses
I followed one of them and went till sandlib.py handle_until_return()
but here didn't see anything apparent and felt like lost...
could smb direct me ? :)

or just tell the functions which would act like usual eval/exec (just sandboxed)

Thanks in advance
-- 
Jurgis Pralgauskis

From arigo at tunes.org  Mon Mar  9 17:46:50 2009
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 9 Mar 2009 17:46:50 +0100
Subject: [pypy-dev] sandboxed exec function with context passing?
In-Reply-To: <34f4097d0903071524n16ebbfet98f87a61a7bb48ff@mail.gmail.com>
References: <34f4097d0903071524n16ebbfet98f87a61a7bb48ff@mail.gmail.com>
Message-ID: <20090309164650.GA5434@code0.codespeak.net>

Hi Jurgis,

On Sun, Mar 08, 2009 at 01:24:27AM +0200, Jurgis Pralgauskis wrote:
> I would like to use sandboxed pypy for online step by step tutorials...
> 1) I need ability to pass context then.
> 2) and also would be nice to get result as variable not just like output.

There is no way any security mechanism can allow arbitrary objects to be
passed in or out, as this would create a security hole.  In our case, it
is an extreme version of this constrain.  The subprocess is really a
different process, and you communicate with it using only the
(sandboxed) I/O functions.  That means that you must encode everything
as I/O operations (and maybe build nicer abstractions to use on top of
that, but these would have to use the low-level I/O method of exchanging
information).

There is nothing like that built so far, but you can easily use e.g. the
marshal module: marshal the data outside, and send it to the sandboxed
process via a pseudo "file read", and then unmarshal it there.  And the
same in the other direction with a "file write".


A bientot,

Armin.

From anto.cuni at gmail.com  Tue Mar 10 15:48:26 2009
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 10 Mar 2009 15:48:26 +0100
Subject: [pypy-dev] Leysin Winter Sprint
In-Reply-To: <20090302154714.GA21804@code0.codespeak.net>
References: <20090302154714.GA21804@code0.codespeak.net>
Message-ID: <49B67DBA.3060702@gmail.com>

Armin Rigo wrote:

> Please *confirm* that you are coming so that we can adjust the reservations
> as appropriate.  The rate so far has been around 60 CHF a night all included
> in 2-person rooms, with breakfast.  There are larger rooms too (less
> expensive) and maybe the possibility to get a single room if you really want
> to.

is 60 CHF per-person or per-room?

Anyone interested in sharing one room?

ciao,
Anto

From lac at openend.se  Wed Mar 11 15:01:34 2009
From: lac at openend.se (Laura Creighton)
Date: Wed, 11 Mar 2009 15:01:34 +0100
Subject: [pypy-dev] RunSnakeRun
Message-ID: <200903111401.n2BE1YA9004344@theraft.openend.se>

I'd never heard of this before, and thought it looked cool.

http://www.vrplumber.com/programming/runsnakerun/

Laura

From arigo at tunes.org  Wed Mar 11 15:10:59 2009
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 11 Mar 2009 15:10:59 +0100
Subject: [pypy-dev] Leysin Winter Sprint
In-Reply-To: <49B67DBA.3060702@gmail.com>
References: <20090302154714.GA21804@code0.codespeak.net>
	<49B67DBA.3060702@gmail.com>
Message-ID: <20090311141059.GA535@code0.codespeak.net>

Hi Anto,

On Tue, Mar 10, 2009 at 03:48:26PM +0100, Antonio Cuni wrote:
> is 60 CHF per-person or per-room?

It's the price per person.


A bientot,

Armin.

From irving at naml.us  Fri Mar 13 01:57:21 2009
From: irving at naml.us (Geoffrey Irving)
Date: Thu, 12 Mar 2009 20:57:21 -0400
Subject: [pypy-dev] pypy for typed languages?
Message-ID: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>

Hello,

I've come to the conclusion that JIT compilation is essential to avoid
abstract penalties even for typed languages.  The one sentence summary
is that the C++ inline keyword is exactly the opposite of what you
want: in a JIT language you can have a "flatten" function that
recursively inlines everything that a function calls (including
closures).

Thus, I'm curious about whether pypy could be practically applied to a
typed front end language, say with the following basic set of
features:

1. Packed storage such as C structs and arrays.
2. A Hindley Milner polymorphic type system with type classes.

In particular, what restrictions does pypy impose on storage layout?
For example, would it be able to handle dynamically-typed homogeneous
lists, represented as a single pointer to a type object and an array
of structs required to be of that type?

Also, if pypy can target LLVM, how do you imagine the two JIT systems
cooperating (if at all)?

Thanks!
Geoffrey

From william.leslie.ttg at gmail.com  Fri Mar 13 03:17:18 2009
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Fri, 13 Mar 2009 13:17:18 +1100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
Message-ID: <f45df7230903121917g6eae6c26g20b1948b01e39d0a@mail.gmail.com>

(sorry Geoffrey for the missend)

2009/3/13 Geoffrey Irving <irving at naml.us>:
> Hello,
Hi! and welcome.

> I'm curious about whether pypy could be practically applied to a
> typed front end language, say with the following basic set of
> features:
>
> 1. Packed storage such as C structs and arrays.

>From my limited understanding this shouldn't be too difficult. Packed storage is
provided at app level, for example, by the ctypes and rawffi modules, and at
interpreter level by using lltypes directly (which is what the JIT operates on).
You could evaluate defstruct by manipulating lltypes. Class definitions could
possibly use rpython.rclass (maybe with a few modifications if classes have
strict layout requirements too).


> 2. A Hindley Milner polymorphic type system with type classes.

I'll leave it to someone with a better knowledge of how the JIT takes advantage
of types to answer this.

William Leslie

From arigo at tunes.org  Fri Mar 13 12:13:02 2009
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 13 Mar 2009 12:13:02 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
Message-ID: <20090313111302.GA27894@code0.codespeak.net>

Hi Geoffrey,

On Thu, Mar 12, 2009 at 08:57:21PM -0400, Geoffrey Irving wrote:
> I've come to the conclusion that JIT compilation is essential to avoid
> abstract penalties even for typed languages.

Indeed: our approach to JIT compilation was first tried with machine
code (see Dynamo/DynamoRio) and then with Java.

> Thus, I'm curious about whether pypy could be practically applied to a
> typed front end language, say with the following basic set of
> features:

I suppose you can try.  For a direct application you need a language for
which it makes sense to write an interpreter in RPython.

> In particular, what restrictions does pypy impose on storage layout?
> For example, would it be able to handle dynamically-typed homogeneous
> lists, represented as a single pointer to a type object and an array
> of structs required to be of that type?

RPython doesn't have support for this.  If you go directly to the lltype
type system, then it's possible -- but our current JIT generator doesn't
support it.  And also, if you write an interpreter using the lltype type
system directly, you loose the abstraction level of writing an
interpreter in RPython in the first place -- e.g. it would not be
translatable any more to the ootype type system, thus to Java or CLI.

So all in all, PyPy could probably be subverted to do what you want, but
it's not really meant to (and we are a bit unlikely to give much
support, as far as I can see).

> Also, if pypy can target LLVM, how do you imagine the two JIT systems
> cooperating (if at all)?

That's a longer-term goal, but I can easily imagine that the code
produced by our JIT could be sent to LLVM at runtime for further
optimizations.


A bientot,

Armin.

From irving at naml.us  Fri Mar 13 15:04:41 2009
From: irving at naml.us (Geoffrey Irving)
Date: Fri, 13 Mar 2009 10:04:41 -0400
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <20090313111302.GA27894@code0.codespeak.net>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
Message-ID: <7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>

On Fri, Mar 13, 2009 at 7:13 AM, Armin Rigo <arigo at tunes.org> wrote:
>> In particular, what restrictions does pypy impose on storage layout?
>> For example, would it be able to handle dynamically-typed homogeneous
>> lists, represented as a single pointer to a type object and an array
>> of structs required to be of that type?
>
> RPython doesn't have support for this. ?If you go directly to the lltype
> type system, then it's possible -- but our current JIT generator doesn't
> support it. ?And also, if you write an interpreter using the lltype type
> system directly, you loose the abstraction level of writing an
> interpreter in RPython in the first place -- e.g. it would not be
> translatable any more to the ootype type system, thus to Java or CLI.
>
> So all in all, PyPy could probably be subverted to do what you want, but
> it's not really meant to (and we are a bit unlikely to give much
> support, as far as I can see).

Thanks for the reply.  It seems like targeting LLVM directly is the
way to go for such projects for now.

Geoffrey

From filippo at esaurito.net  Fri Mar 13 16:00:40 2009
From: filippo at esaurito.net (Filippo Giunchedi)
Date: Fri, 13 Mar 2009 16:00:40 +0100
Subject: [pypy-dev] distributed version control  mirror of trunk?
Message-ID: <20090313150040.GB4625@clamp.esaurito.net>

Hi,
regarding the switch to a DVCS discussed in
http://codespeak.net/pipermail/pypy-dev/2008q3/004816.html I was wondering if
there's a (one way) mirror of SVN trunk/branches to a DVCS such as git or if
such mirror is planned or wished.

thanks in advance,
filippo
--
Filippo Giunchedi - http://esaurito.net - 0x6B79D401

I never forget a face, but in your case I'll be glad to make an exception.
-- Groucho Marx
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20090313/88ce0b20/attachment.pgp 

From fijall at gmail.com  Fri Mar 13 16:19:28 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 13 Mar 2009 16:19:28 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
Message-ID: <693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>

On Fri, Mar 13, 2009 at 3:04 PM, Geoffrey Irving <irving at naml.us> wrote:
> On Fri, Mar 13, 2009 at 7:13 AM, Armin Rigo <arigo at tunes.org> wrote:
>>> In particular, what restrictions does pypy impose on storage layout?
>>> For example, would it be able to handle dynamically-typed homogeneous
>>> lists, represented as a single pointer to a type object and an array
>>> of structs required to be of that type?
>>
>> RPython doesn't have support for this. ?If you go directly to the lltype
>> type system, then it's possible -- but our current JIT generator doesn't
>> support it. ?And also, if you write an interpreter using the lltype type
>> system directly, you loose the abstraction level of writing an
>> interpreter in RPython in the first place -- e.g. it would not be
>> translatable any more to the ootype type system, thus to Java or CLI.
>>
>> So all in all, PyPy could probably be subverted to do what you want, but
>> it's not really meant to (and we are a bit unlikely to give much
>> support, as far as I can see).
>
> Thanks for the reply. ?It seems like targeting LLVM directly is the
> way to go for such projects for now.
>

I don't know, do you have any example?

From irving at naml.us  Fri Mar 13 16:48:39 2009
From: irving at naml.us (Geoffrey Irving)
Date: Fri, 13 Mar 2009 11:48:39 -0400
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
	<693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
Message-ID: <7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>

On Fri, Mar 13, 2009 at 11:19 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Fri, Mar 13, 2009 at 3:04 PM, Geoffrey Irving <irving at naml.us> wrote:
>> On Fri, Mar 13, 2009 at 7:13 AM, Armin Rigo <arigo at tunes.org> wrote:
>>>> In particular, what restrictions does pypy impose on storage layout?
>>>> For example, would it be able to handle dynamically-typed homogeneous
>>>> lists, represented as a single pointer to a type object and an array
>>>> of structs required to be of that type?
>>>
>>> RPython doesn't have support for this. ?If you go directly to the lltype
>>> type system, then it's possible -- but our current JIT generator doesn't
>>> support it. ?And also, if you write an interpreter using the lltype type
>>> system directly, you loose the abstraction level of writing an
>>> interpreter in RPython in the first place -- e.g. it would not be
>>> translatable any more to the ootype type system, thus to Java or CLI.
>>>
>>> So all in all, PyPy could probably be subverted to do what you want, but
>>> it's not really meant to (and we are a bit unlikely to give much
>>> support, as far as I can see).
>>
>> Thanks for the reply. ?It seems like targeting LLVM directly is the
>> way to go for such projects for now.
>
> I don't know, do you have any example?

At this point I'm just in the curiosity stage, so I don't have a
concrete example.  Roughly, I'm imagining a language based on System
CT [1] or the like, which is somewhat like duck typing for static
languages with homogeneous data structures.  Since all function can be
overloaded, at runtime functions would pass around large dictionaries
of overloads, which would need to be optimized away via JIT for
reasonable speed.

A JIT would have an easier job on a System CT-like language than with
python or java, due to the guarantee of homogeneity.  For example, in
the sum function

def sum(container):
    sum = zero
    for elem in container:
        sum = sum + elem
    return sum

a language with homogeneous containers knows that "+" is constant
throughout the loop even if it is impossible to know which constant it
is in advance.

Geoffrey

[1] http://homepages.dcc.ufmg.br/~camarao/CT

From santagada at gmail.com  Fri Mar 13 17:40:34 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 13 Mar 2009 13:40:34 -0300
Subject: [pypy-dev] distributed version control  mirror of trunk?
In-Reply-To: <20090313150040.GB4625@clamp.esaurito.net>
References: <20090313150040.GB4625@clamp.esaurito.net>
Message-ID: <3CE783A0-2E3D-471A-9521-C3271302977B@gmail.com>


On Mar 13, 2009, at 12:00 PM, Filippo Giunchedi wrote:

> Hi,
> regarding the switch to a DVCS discussed in
> http://codespeak.net/pipermail/pypy-dev/2008q3/004816.html I was  
> wondering if
> there's a (one way) mirror of SVN trunk/branches to a DVCS such as  
> git or if
> such mirror is planned or wished.

I think michael hudson did have something on launchpad at one time,  
but I don't think it is updated anymore. I wish there was a two way  
mirror to hg, but I do so little work on pypy that my vote doesnt  
count :) (and I really don't mind svn much).

--
Leonardo Santagada
santagada at gmail.com




From p.giarrusso at gmail.com  Sat Mar 14 05:28:32 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Sat, 14 Mar 2009 05:28:32 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
	<693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
	<7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>
Message-ID: <a4d0c86c0903132128i61530920o925e5df56075fe8c@mail.gmail.com>

On Fri, Mar 13, 2009 at 16:48, Geoffrey Irving <irving at naml.us> wrote:
> On Fri, Mar 13, 2009 at 11:19 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> On Fri, Mar 13, 2009 at 3:04 PM, Geoffrey Irving <irving at naml.us> wrote:
>>> On Fri, Mar 13, 2009 at 7:13 AM, Armin Rigo <arigo at tunes.org> wrote:
>>>>> In particular, what restrictions does pypy impose on storage layout?
>>>>> For example, would it be able to handle dynamically-typed homogeneous
>>>>> lists, represented as a single pointer to a type object and an array
>>>>> of structs required to be of that type?
>>>>
>>>> RPython doesn't have support for this. ?If you go directly to the lltype
>>>> type system, then it's possible -- but our current JIT generator doesn't
>>>> support it. ?And also, if you write an interpreter using the lltype type
>>>> system directly, you loose the abstraction level of writing an
>>>> interpreter in RPython in the first place -- e.g. it would not be
>>>> translatable any more to the ootype type system, thus to Java or CLI.
>>>>
>>>> So all in all, PyPy could probably be subverted to do what you want, but
>>>> it's not really meant to (and we are a bit unlikely to give much
>>>> support, as far as I can see).
>>>
>>> Thanks for the reply. ?It seems like targeting LLVM directly is the
>>> way to go for such projects for now.
>>
>> I don't know, do you have any example?
>
> At this point I'm just in the curiosity stage, so I don't have a
> concrete example. ?Roughly, I'm imagining a language based on System
> CT [1] or the like, which is somewhat like duck typing for static
> languages with homogeneous data structures. ?Since all function can be
> overloaded, at runtime functions would pass around large dictionaries
> of overloads, which would need to be optimized away via JIT for
> reasonable speed.

What is your plan for optimizing those? Are you already planning to
use inline caching for that (what I describe below)?

> A JIT would have an easier job on a System CT-like language than with
> python or java, due to the guarantee of homogeneity. ?For example, in
> the sum function
>
> def sum(container):
> ? ?sum = zero
> ? ?for elem in container:
> ? ? ? ?sum = sum + elem
> ? ?return sum
>
> a language with homogeneous containers knows that "+" is constant
> throughout the loop even if it is impossible to know which constant it
> is in advance.

That can be useful, but in modern JIT that's maybe not the key point,
although you can surely make also use of that.

Most JIT have a heuristic solution which optimizes late binding
anyway, without semantic restrictions, through inline caching or
aggressive inlining - around 90% percent of call sites always call the
same actual method (i.e. they are monomorphic), so code similar to:
if (dest->type == ACertainType)
  ACertainType::method(args); //Early bound call, possibly inlined.
else
  dest->method(args); //Late bound call
can be generated by the JIT. This avoids the indirect jump in most
cases. I'm omitting various details needed to make it actually go
fast, but they're described in the literature.

For that case, it'd be likely better to inline that function into the
caller to get monomorphic call sites (and by any standard that
function should be small enough for that).

Having a constant '+' doesn't help so much if it can have any value.
You save a complete method lookup over a hierarchy, in the Python case
(or in Java with inteface methods), but you still have a call through
a function pointer, which can be quite slow; applying the above
technique

Implementing the above idea in generated JVM bytecode made a
Jython-like JIT compiler for Python, written by another student at the
course, be much faster than CPython (they reported speedups of at
least 3x to 5x, with much room for improvements).

I think also PyPy does something similar for Python, but only through
a hash table. I'd love to see how much inline caching would help on
top of that, when I'll have time to start working on this.

Regards
-- 
Paolo Giarrusso

From arigo at tunes.org  Sat Mar 14 18:52:19 2009
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 14 Mar 2009 18:52:19 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <a4d0c86c0903132128i61530920o925e5df56075fe8c@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
	<693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
	<7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>
	<a4d0c86c0903132128i61530920o925e5df56075fe8c@mail.gmail.com>
Message-ID: <20090314175219.GA12036@code0.codespeak.net>

Hi Paolo,

On Sat, Mar 14, 2009 at 05:28:32AM +0100, Paolo Giarrusso wrote:
> I think also PyPy does something similar for Python, but only through
> a hash table. I'd love to see how much inline caching would help on
> top of that, when I'll have time to start working on this.

No.  PyPy does something very different, more similar to trace-based
JITs (goggle :-)  That does not involve hash tables (I don't know what
you are referring to here).


A bientot,

Armin.

From steve at shrogers.com  Tue Mar 17 03:36:15 2009
From: steve at shrogers.com (Steven H. Rogers)
Date: Mon, 16 Mar 2009 20:36:15 -0600
Subject: [pypy-dev] Speeding Up SimPy
Message-ID: <49BF0C9F.7030004@shrogers.com>

SimPy (http://simpy.sf.net) is a very nice discrete event simulation 
package written in Python.  It is, however, rather slow for large models 
and I'm looking at ways to speed it up.  SimPy should be a good test 
case for PyPy and when the JIT stabilizes, PyPy may provide faster 
execution times. 

As we'd like to see some immediate benefits, I've tried Psyco which 
provides a worthwhile 45% improvement on one model and I'll look at 
modifying SimPy to work better with it.  SimPy makes heavy use of 
generators, so I'm interested in the support for them planned for the 
next release.  How close is Psyco generator support to being useful?

Greenlets look like they could be a viable alternative to generators for 
simulating concurrency.  How expensive are greenlets compared to generators?

Regards,
Steve

From richard.m.tew at gmail.com  Tue Mar 17 04:18:53 2009
From: richard.m.tew at gmail.com (Richard Tew)
Date: Mon, 16 Mar 2009 23:18:53 -0400
Subject: [pypy-dev] Speeding Up SimPy
In-Reply-To: <49BF0C9F.7030004@shrogers.com>
References: <49BF0C9F.7030004@shrogers.com>
Message-ID: <952d92df0903162018u314e2c9sddb7bff4044265d4@mail.gmail.com>

On Mon, Mar 16, 2009 at 10:36 PM, Steven H. Rogers <steve at shrogers.com> wrote:
> Greenlets look like they could be a viable alternative to generators for
> simulating concurrency. ?How expensive are greenlets compared to generators?

I haven't heard of anyone who has benchmarked the difference.  I'd be
interested in seeing some results myself.

I could handwave based on indirect reports, for what it's worth, as
it's easier than actually benchmarking it myself :-)  The Concurrence
framework supports running on top of greenlets and on top of Stackless
Python, which greenlets are a spin off of.  It claims that there's
only a 10-25% difference in performance in Stackless' favour [1].
Stackless Python and a generator based solution were compared in a
recent benchmark [2] with generators being around 10 times slower than
Stackless.

Given that both reports are more or less accurate, I expect greenlets
to be considerably faster than generators.

Cheers,
Richard.

[1] http://opensource.hyves.org/concurrence/index.html
[2] http://entitycrisis.blogspot.com/2009/03/concurrent-scaling-benchmarks.html

From cumber at netspace.net.au  Tue Mar 17 10:03:42 2009
From: cumber at netspace.net.au (Ben Mellor)
Date: Tue, 17 Mar 2009 20:03:42 +1100
Subject: [pypy-dev] RPython I/O
Message-ID: <20090317200342.1426a1ec@intyalie>

Hi

Just wondering, what's the right way to do IO in an RPython program?

sys.stdin/sys.stdout don't work, neither does opening files using open(), and
raw_input doesn't work. All I can get is print. I can't find anything about
this in the docs, and I haven't been able to understand how the PyPy
interpreter is doing it from looking at the source code so far.

I've been playing around with using the PyPy framework to implement a small
interpreter, and this is currently blocking me from translating it.

-- Ben Mellor

From fijall at gmail.com  Tue Mar 17 10:08:11 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 17 Mar 2009 10:08:11 +0100
Subject: [pypy-dev] RPython I/O
In-Reply-To: <20090317200342.1426a1ec@intyalie>
References: <20090317200342.1426a1ec@intyalie>
Message-ID: <693bc9ab0903170208o322aaff7gcf57b559ee04e5cc@mail.gmail.com>

There are two official ways:

1. use directly os.read/os.open/os.write, rather tedious if you ask me
2. use interfaces that are present in pypy/rlib/streamio.py. I think
only documentation for that is in tests unfortunately :(

It has interface similar to python's streams, but adapted a bit to
accomodate rpythonism.

Cheers,
fijal

On Tue, Mar 17, 2009 at 10:03 AM, Ben Mellor <cumber at netspace.net.au> wrote:
> Hi
>
> Just wondering, what's the right way to do IO in an RPython program?
>
> sys.stdin/sys.stdout don't work, neither does opening files using open(), and
> raw_input doesn't work. All I can get is print. I can't find anything about
> this in the docs, and I haven't been able to understand how the PyPy
> interpreter is doing it from looking at the source code so far.
>
> I've been playing around with using the PyPy framework to implement a small
> interpreter, and this is currently blocking me from translating it.
>
> -- Ben Mellor
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From steve at shrogers.com  Tue Mar 17 13:55:45 2009
From: steve at shrogers.com (Steven H. Rogers)
Date: Tue, 17 Mar 2009 06:55:45 -0600
Subject: [pypy-dev] Speeding Up SimPy
In-Reply-To: <952d92df0903162018u314e2c9sddb7bff4044265d4@mail.gmail.com>
References: <49BF0C9F.7030004@shrogers.com>
	<952d92df0903162018u314e2c9sddb7bff4044265d4@mail.gmail.com>
Message-ID: <49BF9DD1.20304@shrogers.com>

Richard Tew wrote:
> On Mon, Mar 16, 2009 at 10:36 PM, Steven H. Rogers <steve at shrogers.com> wrote:
>   
>> Greenlets look like they could be a viable alternative to generators for
>> simulating concurrency.  How expensive are greenlets compared to generators?
>>     
>
> I haven't heard of anyone who has benchmarked the difference.  I'd be
> interested in seeing some results myself.
>
> I could handwave based on indirect reports, for what it's worth, as
> it's easier than actually benchmarking it myself :-)  ...
>   
Thanks Richard.  A little hand waving from someone with your experience 
provides good insight.  I'll start work on some benchmarks.

Does anyone know how well greenlets and Psyco play together?

Regards,
Steve

From jacob at openend.se  Tue Mar 17 19:50:35 2009
From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Tue, 17 Mar 2009 19:50:35 +0100
Subject: [pypy-dev] Processor simulators
Message-ID: <200903171950.35840.jacob@openend.se>

While catching up on todays events on the IRC channel, it struck me that we 
are getting to a point where using a simulator for the processor would be 
very useful.

Once upon a time I did a lot of development of embedded systems, and using a 
simulator was a really nice way of timing and debugging your code. You could 
also do neat tricks if you had the source code of the simulator, recreating 
rare events and generally break the notion of your hardware being an 
unchangeable constant.

I did a quick Goggle search and found that there are plenty of simulators for 
the X86 architecture, with several being fully Open Sourced. For instance,
at http://www.ptlsim.org/, there is a simulator that can run code natively on 
the machine and then jump into and out of simulation mode. This allows for 
precise timing of your selected pieces of code and the possibility of tracing 
exactly what happens at the machine level, with cache misses and whatnot.

I think it might be a useful tool for some of the current backend work for the 
JIT, and for future optimizations.

Jacob


From cumber at netspace.net.au  Wed Mar 18 07:45:33 2009
From: cumber at netspace.net.au (Ben Mellor)
Date: Wed, 18 Mar 2009 17:45:33 +1100
Subject: [pypy-dev] RPython I/O
In-Reply-To: <693bc9ab0903170208o322aaff7gcf57b559ee04e5cc@mail.gmail.com>
References: <20090317200342.1426a1ec@intyalie>
	<693bc9ab0903170208o322aaff7gcf57b559ee04e5cc@mail.gmail.com>
Message-ID: <20090318174533.583f749e@intyalie>

Thanks! Got it working now.

If I do:

stdin_fd = sys.stdin.fileno()

during import time and then use that with os.read (either directly or
indirectly through streams), will it also be translated correctly for the other
non-C backends?


> There are two official ways:
> 
> 1. use directly os.read/os.open/os.write, rather tedious if you ask me
> 2. use interfaces that are present in pypy/rlib/streamio.py. I think
> only documentation for that is in tests unfortunately :(
> 
> It has interface similar to python's streams, but adapted a bit to
> accomodate rpythonism.
> 
> Cheers,
> fijal
> 
> On Tue, Mar 17, 2009 at 10:03 AM, Ben Mellor <cumber at netspace.net.au> wrote:
> > Hi
> >
> > Just wondering, what's the right way to do IO in an RPython program?
> >
> > sys.stdin/sys.stdout don't work, neither does opening files using open(),
> > and raw_input doesn't work. All I can get is print. I can't find anything
> > about this in the docs, and I haven't been able to understand how the PyPy
> > interpreter is doing it from looking at the source code so far.
> >
> > I've been playing around with using the PyPy framework to implement a small
> > interpreter, and this is currently blocking me from translating it.
> >
> > -- Ben Mellor
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
> >

From lac at openend.se  Mon Mar 23 10:14:50 2009
From: lac at openend.se (Laura Creighton)
Date: Mon, 23 Mar 2009 10:14:50 +0100
Subject: [pypy-dev] EuroSciPy -- Place to find people who might like to try
	out the JIT?
Message-ID: <200903230914.n2N9Eo5a021471@theraft.openend.se>


just a thought,
Laura

------- Forwarded Message

Return-Path: python-announce-list-bounces+lac=openend.se at python.org
Delivery-Date: Mon Mar 23 09:32:52 2009
Date: Sun, 22 Mar 2009 20:59:56 +0100
From: =?ISO-8859-15?Q?Mike_M=FCller?= <mmueller at python-academy.de>
Organization: Python Academy
User-Agent: Thunderbird 2.0.0.19 (Windows/20081209)
MIME-Version: 1.0
To: python-announce-list at python.org
Subject: EuroSciPy 2009, Leipzig, Germany, July 25-26


EuroSciPy 2009
==============

We're pleased to announce the EuroSciPy 2009 Conference to be held in
Leipzig, Germany on July 25-26, 2009.

http://www.euroscipy.org

This is the second conference after the successful conference last
year. Again, EuroSciPy will be a venue for the European community of
users of the Python programming language in science.


Call for Participation
- ----------------------

If you are a scientist using Python for your computational work, we'd
love to have you formally present your results, methods or
experiences. To apply to present a talk at this year's EuroSciPy,
please submit an abstract of your talk as a PDF, MS Word or plain text
file to mmueller at python-academy dot de. The deadline for abstract
submission is April 30, 2009. Papers and/or presentation slides are 
acceptable and are due by June 15, 2009. Presentations will be allotted
30 minutes.


Registration
- ------------

Registration is open. The registration fee is 100.00 ? for early
registrants and will increase to 150.00 ? for late registration after 
June 15, 2009. Registration will include breakfast, snacks and lunch
for Saturday and Sunday.

Please register here:
http://www.euroscipy.org/presentations/index.html


Important Dates
- ---------------

March 21        Registration opens
April 30        Abstract submission deadline
May 15          Acceptance of presentations
May 30          Announcement of conference program
June 15         Early bird registration deadline
June 15         Paper/slides submission deadline
July 20 - 24    Pre-Conference courses
July 25/26      Conference


Venue
- -----

mediencampus
Poetenweg 28
04155 Leipzig
Germany

See http://www.euroscipy.org/venue.html for details.


Help Welcome
- ------------

You like to help make the EuroSciPy 2009 a success?
Here are some ways you can get involved:

* attend the conference
* submit an abstract for a presentation
* give a lightning talk
* make EuroSciPy known:
     - write about it on your website
     - in your blog
     - talk to friends about it
     - post to local e-mail lists
     - post to related forums
     - spread flyers and posters in your institution
     - make entries in relevant event calendars
     - anything you can think of
* inform potential sponsors about the event
* become a sponsor

If you're interested in volunteering to help organize things
or have some other idea that can help the conference, please
email us at mmueller at python-academy dot de.


Sponsorship
- -----------

Do you like to sponsor the conference?
There are several options available:

http://www.euroscipy.org/sponsors/become_a_sponsor.html


Pre-Conference Courses
- ----------------------

Would you like to learn Python or about some of the most used scientific
libraries in Python? Then the "Python Summer Course" [1] might be for
you. There are two parts to this course:

* a two-day course "Introduction to Python" [2] for people with
   programming experience in other languages and
* a three-day course "Python for Scientists and Engineers" [3] that
   introduces some of the most used Python tools for scientists and
   engineers such as NumPy, PyTables, and matplotlib

Both courses can be booked individually [4]. Of course, you can attend
the courses without registering for EuroSciPy.

[1] http://www.python-academy.com/courses/python_summer_course.html
[2] http://www.python-academy.com/courses/python_course_programmers.html
[3] http://www.python-academy.com/courses/python_course_scientists.html
[4] http://www.python-academy.com/courses/dates.html
- --
http://mail.python.org/mailman/listinfo/python-announce-list

        Support the Python Software Foundation:
        http://www.python.org/psf/donations.html

------- End of Forwarded Message


From p.giarrusso at gmail.com  Mon Mar 23 11:46:29 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Mon, 23 Mar 2009 11:46:29 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <20090314175219.GA12036@code0.codespeak.net>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
	<693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
	<7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>
	<a4d0c86c0903132128i61530920o925e5df56075fe8c@mail.gmail.com>
	<20090314175219.GA12036@code0.codespeak.net>
Message-ID: <a4d0c86c0903230346g1736b56ehc53e0efb0f089597@mail.gmail.com>

Hi Armin,

On Sat, Mar 14, 2009 at 18:52, Armin Rigo <arigo at tunes.org> wrote:
> Hi Paolo,

> On Sat, Mar 14, 2009 at 05:28:32AM +0100, Paolo Giarrusso wrote:
>> I think also PyPy does something similar for Python, but only through
>> a hash table. I'd love to see how much inline caching would help on
>> top of that, when I'll have time to start working on this.

> No. ?PyPy does something very different, more similar to trace-based
> JITs (goggle :-)
Ah OK, sorry for that.
> That does not involve hash tables (I don't know what
> you are referring to here).

I was thinking about the method cache in the interpreter; I'm not yet
familiar with the approaches you used for JIT compilation (neither the
one of PyPy 1.0 nor the one being prototyped right now), but I know
the general purpose "get a JIT from an interpreter", and I thought
that having a matching behaviour implied using the same lookup
algorithm in JITted code. I understand that's no more true, with the
ideas of trace-based JITs.

Greetings,
-- 
Paolo Giarrusso

From vincent.legoll at gmail.com  Tue Mar 24 11:24:05 2009
From: vincent.legoll at gmail.com (Vincent Legoll)
Date: Tue, 24 Mar 2009 11:24:05 +0100
Subject: [pypy-dev] [OT] PyPy/python newb - need dbm other than dumbdbm
	anddbm
Message-ID: <4727185d0903240324p475a6d6fv9c0ffbd4e700fc1f@mail.gmail.com>

Hello,

have you tried tailor (http://progetti.arstecnica.it/tailor)

I've had good experience with it converting perforce to
svn, a branch with about 220 K changesets on mid-end
workstation HW (4 GB Ram / Dual-proc dual core Xeon
1.8 GHz)

I can't remeber the total time it took, but it was in the
2-4 days range. tailor keeps state of where it is in the
process of migration, so it can restart at the same
place in case a problem needs to be manually solved.

The process was more IO bound than CPU bound, I
don't know how well it handles cvs though.

-- 
Vincent Legoll

From arigo at tunes.org  Tue Mar 24 11:48:16 2009
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 24 Mar 2009 11:48:16 +0100
Subject: [pypy-dev] pypy for typed languages?
In-Reply-To: <a4d0c86c0903230346g1736b56ehc53e0efb0f089597@mail.gmail.com>
References: <7f9d599f0903121757k37421a77kaa707214d0ce291c@mail.gmail.com>
	<20090313111302.GA27894@code0.codespeak.net>
	<7f9d599f0903130704g74e62e68lad65d735c85eb9e6@mail.gmail.com>
	<693bc9ab0903130819y1a686216h8cdada85fca59d17@mail.gmail.com>
	<7f9d599f0903130848m282b371fk231e4b17bb9cff34@mail.gmail.com>
	<a4d0c86c0903132128i61530920o925e5df56075fe8c@mail.gmail.com>
	<20090314175219.GA12036@code0.codespeak.net>
	<a4d0c86c0903230346g1736b56ehc53e0efb0f089597@mail.gmail.com>
Message-ID: <20090324104816.GA1990@code0.codespeak.net>

Hi Paolo,

On Mon, Mar 23, 2009 at 11:46:29AM +0100, Paolo Giarrusso wrote:
> I was thinking about the method cache in the interpreter; I'm not yet
> familiar with the approaches you used for JIT compilation (neither the
> one of PyPy 1.0 nor the one being prototyped right now), but I know
> the general purpose "get a JIT from an interpreter", and I thought
> that having a matching behaviour implied using the same lookup
> algorithm in JITted code. I understand that's no more true, with the
> ideas of trace-based JITs.

Ah, sorry.  I may have misread your original post.  The fact is that
both the PyPy 1.0 JIT and the current prototype JIT (and all other
prototypes inbetween) are similar, in that they are derived
automatically from the interpreter.  If this interpreter happens to
contain a hash table lookup, as the Python interpreter of PyPy
optionally does, then the JIT will contain it too.  So we are left with
experimenting with which interpreter features are useful for the JIT
too, and which ones give bad results.  This experimentation is both easy
and not quite doable right now because we are still developping the JIT
generation basics.


A bientot,

Armin.

From igor_trindade at yahoo.com.br  Tue Mar 24 16:08:28 2009
From: igor_trindade at yahoo.com.br (Igor Trindade Oliveira)
Date: Tue, 24 Mar 2009 11:08:28 -0400
Subject: [pypy-dev] GSoC proposal - C++ binding
Message-ID: <200903241108.29227.igor_trindade@yahoo.com.br>

Hi guys,

i was talking with some pypy guys about work in this summer in GSoC with C++ 
bindings to Pypy and i wrote a proposal and i like that you review it.

Igor

PROPOSAL:

Problem

Pypy are becoming one of the most used Python interpreter because it is easy 
and flexible, but we still have a big problem, it does not have any binding to 
any Graphical User Interface(GUI) and we have the second problem, many GUIs 
are wrote in C++(like Qt or WxWidgets) and Pypy can not access C++ code 
because it does not have any C API, it just can access C code using ctypes.

Solution

Many solutions had been argued but the main solution is use Reflex[2],which is 
developed at CERN (which has an incredible amount of C++ libraries). It is not 
mainly intended for writing Python bindings for C++ libraries but instead 
provides reflection capabilities for C++. The idea is that for every C++ shared 
library, an additional shared library is produced, which allows together with 
Reflex to introspect properties of C++ classes, methods, etc. at runtime. These 
facilities are then used for writing a small generic CPython extension module, 
that allows CPython to use any C++ library for which this reflection 
information was generated [3].
This approach is a bit similar to the ctypes module, apart from the fact that 
ctypes does not use any reflection information, but the user has to specify the 
data structures that occur in the C code herself. This makes it sometimes 
rather burdensome to write cross-platform library bindings.
For PyPy the approach seems rather fitting: We would need to implement only the 
generic extension module(a C language interface to Reflex) and could then use 
any number of C++ libraries.

Goals

The main goal of this project is to do a C reflex interface and make an easy 
API using ctypes to RPython communicate with that interface. More specific, the 
goals of the project are:
1.create an interface to reflex in C;
2.Use RPython ctypes to communicate with it;
3.create an API to RPython ctypes have an easy way to communicate with C 
reflex.
4.Begin a binding to Qt;

Timeline

The Google Summer of Code is subdivided in two parts. First part plans:
1.work in reflex to create a C interface for it.
2.Begin work on RPython ctypes;

Second part plans:

1.create an API to RPython ctypes communicate with C reflex;
2.Begin Qt binding;

Bio

My name is Igor Trindade Oliveira, I am a Undergraduate Computer Engineering 
student in Brazil, in the fourth year.
I work also in KDE project in KDE-PIM projects[1] and have experience with 
microcontrollers like 8051, MSP430, Mips r4000 and Arm 9. 
And with KDE I have very experience with C++ code.

[1] http://techbase.kde.org/Projects/PIM/Akonadi/Testing
[2] http://root.cern.ch/drupal/content/reflex
[3] http://morepypy.blogspot.com/2008/10/sprint-discussions-c-library-
bindings.html

From ndbecker2 at gmail.com  Tue Mar 24 18:18:14 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 24 Mar 2009 13:18:14 -0400
Subject: [pypy-dev] GSoC proposal - C++ binding
References: <200903241108.29227.igor_trindade@yahoo.com.br>
Message-ID: <gqb4km$nor$1@ger.gmane.org>

Igor Trindade Oliveira wrote:

> Hi guys,
> 
> i was talking with some pypy guys about work in this summer in GSoC with
> C++ bindings to Pypy and i wrote a proposal and i like that you review it.
> 
> Igor
> 
> PROPOSAL:
> 
> Problem
> 
> Pypy are becoming one of the most used Python interpreter because it is
> easy and flexible, but we still have a big problem, it does not have any
> binding to any Graphical User Interface(GUI) and we have the second
> problem, many GUIs are wrote in C++(like Qt or WxWidgets) and Pypy can not
> access C++ code because it does not have any C API, it just can access C
> code using ctypes.
> 
> Solution
> 
> Many solutions had been argued but the main solution is use
> Reflex[2],which is developed at CERN (which has an incredible amount of
> C++ libraries). It is not mainly intended for writing Python bindings for
> C++ libraries but instead provides reflection capabilities for C++. The
> idea is that for every C++ shared library, an additional shared library is
> produced, which allows together with Reflex to introspect properties of
> C++ classes, methods, etc. at runtime. These facilities are then used for
> writing a small generic CPython extension module, that allows CPython to
> use any C++ library for which this reflection information was generated
> [3]. This approach is a bit similar to the ctypes module, apart from the
> fact that ctypes does not use any reflection information, but the user has
> to specify the data structures that occur in the C code herself. This
> makes it sometimes rather burdensome to write cross-platform library
> bindings. For PyPy the approach seems rather fitting: We would need to
> implement only the generic extension module(a C language interface to
> Reflex) and could then use any number of C++ libraries.
> 
...
I wonder if py++ would help?



From santagada at gmail.com  Tue Mar 24 19:13:25 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 24 Mar 2009 15:13:25 -0300
Subject: [pypy-dev] GSoC proposal - C++ binding
In-Reply-To: <gqb4km$nor$1@ger.gmane.org>
References: <200903241108.29227.igor_trindade@yahoo.com.br>
	<gqb4km$nor$1@ger.gmane.org>
Message-ID: <68532EF2-A9AC-439D-AC3B-85EC03EC6545@gmail.com>


On Mar 24, 2009, at 2:18 PM, Neal Becker wrote:

> Igor Trindade Oliveira wrote:
>
>> Hi guys,
>>
>> i was talking with some pypy guys about work in this summer in GSoC  
>> with
>> C++ bindings to Pypy and i wrote a proposal and i like that you  
>> review it.
>>
>> Igor
>>
>> PROPOSAL:
>>
>> Problem
>>
>> Pypy are becoming one of the most used Python interpreter because  
>> it is
>> easy and flexible, but we still have a big problem, it does not  
>> have any
>> binding to any Graphical User Interface(GUI) and we have the second
>> problem, many GUIs are wrote in C++(like Qt or WxWidgets) and Pypy  
>> can not
>> access C++ code because it does not have any C API, it just can  
>> access C
>> code using ctypes.
>>
>> Solution
>>
>> Many solutions had been argued but the main solution is use
>> Reflex[2],which is developed at CERN (which has an incredible  
>> amount of
>> C++ libraries). It is not mainly intended for writing Python  
>> bindings for
>> C++ libraries but instead provides reflection capabilities for C++.  
>> The
>> idea is that for every C++ shared library, an additional shared  
>> library is
>> produced, which allows together with Reflex to introspect  
>> properties of
>> C++ classes, methods, etc. at runtime. These facilities are then  
>> used for
>> writing a small generic CPython extension module, that allows  
>> CPython to
>> use any C++ library for which this reflection information was  
>> generated
>> [3]. This approach is a bit similar to the ctypes module, apart  
>> from the
>> fact that ctypes does not use any reflection information, but the  
>> user has
>> to specify the data structures that occur in the C code herself. This
>> makes it sometimes rather burdensome to write cross-platform library
>> bindings. For PyPy the approach seems rather fitting: We would need  
>> to
>> implement only the generic extension module(a C language interface to
>> Reflex) and could then use any number of C++ libraries.
>>
> ...
> I wonder if py++ would help?


When people were looking at what would be best to use they considered  
py++ (and boost.python).

One of the reasons to not use it is that if I remember correctly py++  
uses gcc-xml and it needs the source of the C++ library before  
generating some cpython module that needs to be compiled also. One of  
the good points of Reflex is that after the generation of the  
reflection library you don't need the source of the package neither a  
compiler installed, and reflex supports c++ also (which would be  
interesting if we want to convince OS people of including reflex  
modules of c++ libraries).

Could Reflex be made to generate a reflex module of c packages also,  
so you don't have to declare them again in ctypes in python and all  
languages that support libffi? That would be cool.

--
Leonardo Santagada
santagada at gmail.com




From igor_trindade at yahoo.com.br  Tue Mar 24 20:27:27 2009
From: igor_trindade at yahoo.com.br (Igor Trindade Oliveira)
Date: Tue, 24 Mar 2009 15:27:27 -0400
Subject: [pypy-dev] GSoC proposal - C++ binding
In-Reply-To: <68532EF2-A9AC-439D-AC3B-85EC03EC6545@gmail.com>
References: <200903241108.29227.igor_trindade@yahoo.com.br>
	<gqb4km$nor$1@ger.gmane.org>
	<68532EF2-A9AC-439D-AC3B-85EC03EC6545@gmail.com>
Message-ID: <200903241527.27794.igor_trindade@yahoo.com.br>

On Tuesday 24 March 2009 14:13:25 Leonardo Santagada wrote:
> On Mar 24, 2009, at 2:18 PM, Neal Becker wrote:
> > Igor Trindade Oliveira wrote:
> >> Hi guys,
> >>
> >> i was talking with some pypy guys about work in this summer in GSoC
> >> with
> >> C++ bindings to Pypy and i wrote a proposal and i like that you
> >> review it.
> >>
> >> Igor
> >>
> >> PROPOSAL:
> >>
> >> Problem
> >>
> >> Pypy are becoming one of the most used Python interpreter because
> >> it is
> >> easy and flexible, but we still have a big problem, it does not
> >> have any
> >> binding to any Graphical User Interface(GUI) and we have the second
> >> problem, many GUIs are wrote in C++(like Qt or WxWidgets) and Pypy
> >> can not
> >> access C++ code because it does not have any C API, it just can
> >> access C
> >> code using ctypes.
> >>
> >> Solution
> >>
> >> Many solutions had been argued but the main solution is use
> >> Reflex[2],which is developed at CERN (which has an incredible
> >> amount of
> >> C++ libraries). It is not mainly intended for writing Python
> >> bindings for
> >> C++ libraries but instead provides reflection capabilities for C++.
> >> The
> >> idea is that for every C++ shared library, an additional shared
> >> library is
> >> produced, which allows together with Reflex to introspect
> >> properties of
> >> C++ classes, methods, etc. at runtime. These facilities are then
> >> used for
> >> writing a small generic CPython extension module, that allows
> >> CPython to
> >> use any C++ library for which this reflection information was
> >> generated
> >> [3]. This approach is a bit similar to the ctypes module, apart
> >> from the
> >> fact that ctypes does not use any reflection information, but the
> >> user has
> >> to specify the data structures that occur in the C code herself. This
> >> makes it sometimes rather burdensome to write cross-platform library
> >> bindings. For PyPy the approach seems rather fitting: We would need
> >> to
> >> implement only the generic extension module(a C language interface to
> >> Reflex) and could then use any number of C++ libraries.
> >
> > ...
> > I wonder if py++ would help?
>
> When people were looking at what would be best to use they considered
> py++ (and boost.python).
>
> One of the reasons to not use it is that if I remember correctly py++
> uses gcc-xml and it needs the source of the C++ library before
> generating some cpython module that needs to be compiled also. One of
> the good points of Reflex is that after the generation of the
> reflection library you don't need the source of the package neither a
> compiler installed, and reflex supports c++ also (which would be
> interesting if we want to convince OS people of including reflex
> modules of c++ libraries).
>
> Could Reflex be made to generate a reflex module of c packages also,
> so you don't have to declare them again in ctypes in python and all
> languages that support libffi? That would be cool.

right now i dont know but i can check it, but i believe that it just works 
with C++ packages.

Igor Trindade Oliveira
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev


From chris.nicholls at some.ox.ac.uk  Wed Mar 25 14:14:26 2009
From: chris.nicholls at some.ox.ac.uk (Chris Nicholls)
Date: Wed, 25 Mar 2009 13:14:26 +0000
Subject: [pypy-dev] Google Summer of code
Message-ID: <198166e70903250614t297d1d65hca5a7c797ccded3b@mail.gmail.com>

Hello,

I'm a student interested in taking part in Google summer of code with pypy.
I've been following pypy for a while now and this seems like a good
opportunity to get involved, Is there a list of ideas anywhere? I'm not
particularly sure on where to start... I found last years list
here<http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html>but I
expect its quite out of date.

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090325/22680910/attachment.htm 

From tismer at stackless.com  Thu Mar 26 01:27:33 2009
From: tismer at stackless.com (Christian Tismer)
Date: Thu, 26 Mar 2009 01:27:33 +0100
Subject: [pypy-dev] http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
Message-ID: <49CACBF5.4020904@stackless.com>

Hi friends,

please have a look at this.
http://code.google.com/p/unladen-swallow/wiki/ProjectPlan

is this YAIPAP ?
Yet Another Ignorant Python Acceleration Project

I see them mentioning H?lzle and Self, and I see a reference to
Psyco where they want to steal from, but PyPy does not exist.

IMHO, this is totally GAGA  - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From cumber at netspace.net.au  Thu Mar 26 01:41:27 2009
From: cumber at netspace.net.au (Ben Mellor)
Date: Thu, 26 Mar 2009 11:41:27 +1100
Subject: [pypy-dev] RPython rendering of % operator
Message-ID: <20090326114127.147ff5ac@intyalie>

Does anyone know why

lambda x, y: x % y

is being converted to a flow graph as:

v2 = int_mod(x_0, y_0)
v1 = int_add(v2, [int_mul([int_and([cast_bool_to_int([int_le([int_xor(x_0,
y_0)], (0))])], [cast_bool_to_int([int_ne(v2, (0))])])], y_0)])

(v1 is returned)


What's wrong with just v1 = int_mod(x_0, y_0)?

-- Ben

From santagada at gmail.com  Thu Mar 26 02:42:32 2009
From: santagada at gmail.com (Leonardo Santagada)
Date: Wed, 25 Mar 2009 22:42:32 -0300
Subject: [pypy-dev]
	http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
In-Reply-To: <49CACBF5.4020904@stackless.com>
References: <49CACBF5.4020904@stackless.com>
Message-ID: <9B54C41A-EF31-4BB1-927F-6D02B88A7B41@gmail.com>


On Mar 25, 2009, at 9:27 PM, Christian Tismer wrote:

> Hi friends,
>
> please have a look at this.
> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>
> is this YAIPAP ?
> Yet Another Ignorant Python Acceleration Project
>
> I see them mentioning H?lzle and Self, and I see a reference to
> Psyco where they want to steal from, but PyPy does not exist.
>
> IMHO, this is totally GAGA  - chris


I was reading it earlier, the simpler ideas, like making pickle faster  
and most of q1 deliverables seems nice, and could really help python  
right now, but those seems more like the things Need For Speed sprint  
was doing. Not the LLVM-JIT, new GC, eliminate the GIL seems  
unrealistic, even the pace of the project seems to be counting on tons  
of new developers joining the project after the Q1 milestone.

--
Leonardo Santagada
santagada at gmail.com




From amauryfa at gmail.com  Thu Mar 26 09:36:00 2009
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 26 Mar 2009 09:36:00 +0100
Subject: [pypy-dev] RPython rendering of % operator
In-Reply-To: <20090326114127.147ff5ac@intyalie>
References: <20090326114127.147ff5ac@intyalie>
Message-ID: <e27efe130903260136i7b46dc70s9245ac3d9886534d@mail.gmail.com>

Hello,

On Thu, Mar 26, 2009 at 01:41, Ben Mellor <cumber at netspace.net.au> wrote:
> Does anyone know why
>
> lambda x, y: x % y
>
> is being converted to a flow graph as:
>
> v2 = int_mod(x_0, y_0)
> v1 = int_add(v2, [int_mul([int_and([cast_bool_to_int([int_le([int_xor(x_0,
> y_0)], (0))])], [cast_bool_to_int([int_ne(v2, (0))])])], y_0)])
>
> (v1 is returned)
>
>
> What's wrong with just v1 = int_mod(x_0, y_0)?

You are looking at the graph of the function:
    pypy.rpython.lltypesystem.opimpl.op_int_mod

The difference is certainly because C and python handle modulus
differently when one argument is negative:
(-5 % 3) returns -2 in C, but 1 with python.

Now, I don't know whether RPython should absolutely implement python
semantics here.
RPython does not check for overflow for example.

-- 
Amaury Forgeot d'Arc

From cfbolz at gmx.de  Thu Mar 26 09:51:39 2009
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 26 Mar 2009 09:51:39 +0100
Subject: [pypy-dev] RPython rendering of % operator
In-Reply-To: <e27efe130903260136i7b46dc70s9245ac3d9886534d@mail.gmail.com>
References: <20090326114127.147ff5ac@intyalie>
	<e27efe130903260136i7b46dc70s9245ac3d9886534d@mail.gmail.com>
Message-ID: <49CB421B.4050600@gmx.de>

Amaury Forgeot d'Arc wrote:
> Hello,
> 
> On Thu, Mar 26, 2009 at 01:41, Ben Mellor <cumber at netspace.net.au> wrote:
>> Does anyone know why
>>
>> lambda x, y: x % y
>>
>> is being converted to a flow graph as:
>>
>> v2 = int_mod(x_0, y_0)
>> v1 = int_add(v2, [int_mul([int_and([cast_bool_to_int([int_le([int_xor(x_0,
>> y_0)], (0))])], [cast_bool_to_int([int_ne(v2, (0))])])], y_0)])
>>
>> (v1 is returned)
>>
>>
>> What's wrong with just v1 = int_mod(x_0, y_0)?
> 
> You are looking at the graph of the function:
>     pypy.rpython.lltypesystem.opimpl.op_int_mod
> 
> The difference is certainly because C and python handle modulus
> differently when one argument is negative:
> (-5 % 3) returns -2 in C, but 1 with python.
> 
> Now, I don't know whether RPython should absolutely implement python
> semantics here.
> RPython does not check for overflow for example.

The semantics of % in RPython _are_ the same as in Python. It's just
that the semantics of int_mod aren't that of Python, therefore % must be
mapped to something more complex. The reason why int_mod has the C
semantics is that it makes the life of all backends easier.

Cheers,

Carl Friedrich


From fijall at gmail.com  Thu Mar 26 13:05:35 2009
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 26 Mar 2009 13:05:35 +0100
Subject: [pypy-dev] Google Summer of code
In-Reply-To: <198166e70903250614t297d1d65hca5a7c797ccded3b@mail.gmail.com>
References: <198166e70903250614t297d1d65hca5a7c797ccded3b@mail.gmail.com>
Message-ID: <693bc9ab0903260505p3d081950m33b21b5878df8bb9@mail.gmail.com>

Hi Chris.

There is no up-to-date list (we were too busy with other stuff and
conferences etc.), but you can surely come to #pypy on IRC and ask a
bit. It depends what are you interested in of course.

Cheers,
fijal

2009/3/25 Chris Nicholls <chris.nicholls at some.ox.ac.uk>:
> Hello,
>
> I'm a student interested in taking part in Google summer of code with pypy.
> I've been following pypy for a while now and this seems like a good
> opportunity to get involved, Is there a list of ideas anywhere? I'm not
> particularly sure on where to start... I found last years list here but I
> expect its quite out of date.
>
> Chris
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From p.giarrusso at gmail.com  Mon Mar 30 01:28:31 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Mon, 30 Mar 2009 00:28:31 +0100
Subject: [pypy-dev]
	http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
In-Reply-To: <9B54C41A-EF31-4BB1-927F-6D02B88A7B41@gmail.com>
References: <49CACBF5.4020904@stackless.com>
	<9B54C41A-EF31-4BB1-927F-6D02B88A7B41@gmail.com>
Message-ID: <a4d0c86c0903291628l4caef283s96a0adc7d26e6b54@mail.gmail.com>

On Thu, Mar 26, 2009 at 02:42, Leonardo Santagada <santagada at gmail.com> wrote:
>
> On Mar 25, 2009, at 9:27 PM, Christian Tismer wrote:
>
> > Hi friends,
> >
> > please have a look at this.
> > http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
> >
> > is this YAIPAP ?
> > Yet Another Ignorant Python Acceleration Project
> >
> > I see them mentioning H?lzle and Self, and I see a reference to
> > Psyco where they want to steal from, but PyPy does not exist.
> >
> > IMHO, this is totally GAGA ?- chris

First, I suggest you have a look at the FAQ - when I did, I discovered
the developers are full-time Google engineers. Having said that, the
Google V8 group does not seem to be involved, and that's quite stupid.
However, I studied with Lars Bak and the base ideas he had are the
same.

Missing knowledge of PyPy is also stupid obviously, but I wonder why
nobody proposed "hey, let's tell them"; fighting another project.
They'd sure benefit, for instance, from CALL_METHOD and
CALL_LIKELY_BUILTIN ideas (they mention among their purposes fixing
the performance problems addressed by CALL_LIKELY_BUILTIN, that's why
I mention these).

Also, they're already started with releasing. Have a look at their benchmarks:
http://code.google.com/p/unladen-swallow/wiki/Releases

Did you look at that because declaring it GAGA?

> I was reading it earlier, the simpler ideas, like making pickle faster
> and most of q1 deliverables seems nice, and could really help python
> right now, but those seems more like the things Need For Speed sprint
> was doing.

I don't know about Need For Sprint, but my first guess is that this
maybe happens because, as pointed out various ways, CPython has lots
of "obvious" deficiencies.

> Not the LLVM-JIT, new GC, eliminate the GIL seems
> unrealistic, even the pace of the project seems to be counting on tons
> of new developers joining the project after the Q1 milestone.

Well, the milestones seem crazy but they did achieve something notable
in their Q1 deliverable, but most of the ideas seem correct.
"Eliminate the GIL" is not hard by itself, the problem (IMHO, no hard
numbers for that) is that it's impossible with refcounting (because of
the cost of atomic refcount manipulation). The author of the old "free
threading" patch mentioned only another problem, that is locking
around name lookups in mutable name dictionaries (for looking up
object members, globals, etc.), which can also be approached (and I
think refcount manipulation is a bigger issue).

Regards
--
Paolo Giarrusso

From tismer at stackless.com  Mon Mar 30 13:48:39 2009
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 30 Mar 2009 13:48:39 +0200
Subject: [pypy-dev]
	http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
In-Reply-To: <a4d0c86c0903291628l4caef283s96a0adc7d26e6b54@mail.gmail.com>
References: <49CACBF5.4020904@stackless.com>	
	<9B54C41A-EF31-4BB1-927F-6D02B88A7B41@gmail.com>
	<a4d0c86c0903291628l4caef283s96a0adc7d26e6b54@mail.gmail.com>
Message-ID: <49D0B197.5070509@stackless.com>

On 3/30/09 1:28 AM, Paolo Giarrusso wrote:
> On Thu, Mar 26, 2009 at 02:42, Leonardo Santagada<santagada at gmail.com>  wrote:
>> On Mar 25, 2009, at 9:27 PM, Christian Tismer wrote:
>>
>>> Hi friends,
>>>
>>> please have a look at this.
>>> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>>>
>>> is this YAIPAP ?
>>> Yet Another Ignorant Python Acceleration Project
>>>
>>> I see them mentioning H?lzle and Self, and I see a reference to
>>> Psyco where they want to steal from, but PyPy does not exist.
>>>
>>> IMHO, this is totally GAGA  - chris
>
> First, I suggest you have a look at the FAQ - when I did, I discovered
> the developers are full-time Google engineers. Having said that, the
> Google V8 group does not seem to be involved, and that's quite stupid.
> However, I studied with Lars Bak and the base ideas he had are the
> same.

Interesting. Yes, I read the FAQ, but I don't know the people.

> Missing knowledge of PyPy is also stupid obviously, but I wonder why
> nobody proposed "hey, let's tell them"; fighting another project.
> They'd sure benefit, for instance, from CALL_METHOD and
> CALL_LIKELY_BUILTIN ideas (they mention among their purposes fixing
> the performance problems addressed by CALL_LIKELY_BUILTIN, that's why
> I mention these).

Well, we could have contacted them.
But (I'm speaking for myself) if they have decided to totally
ignore a project like PyPy, then I think it is for a reason.
Not knowing about PyPy means to close more than two eyes.
Therefore I see no point in approaching them.

> Also, they're already started with releasing. Have a look at their benchmarks:
> http://code.google.com/p/unladen-swallow/wiki/Releases
>
> Did you look at that because declaring it GAGA?

My first perception of the project was a bit distorted.
I saw this as an attempt to replace PyPy with something better,
and it seemed GAGA to me to do that ignoring prior work.

Now that I realize that the project has much smaller
goals, it becomes more realistic.

The Q1 goals are relatively doable without doubt. The current
achievements speedwise remind me of the anxient Python2C project.
It showed the typical acceleration by a factor of around 2, which
is what you can expect when eliminating the interpreter loop.

>> I was reading it earlier, the simpler ideas, like making pickle faster
>> and most of q1 deliverables seems nice, and could really help python
>> right now, but those seems more like the things Need For Speed sprint
>> was doing.

Yes, Need for Speed did small, doable things.

>> Not the LLVM-JIT, new GC, eliminate the GIL seems
>> unrealistic, even the pace of the project seems to be counting on tons
>> of new developers joining the project after the Q1 milestone.
>
> Well, the milestones seem crazy but they did achieve something notable
> in their Q1 deliverable, but most of the ideas seem correct.

Q1 is fine, but it does by no means give any proof that the next
milestones can be achieved.

> "Eliminate the GIL" is not hard by itself, the problem (IMHO, no hard
> numbers for that) is that it's impossible with refcounting (because of
> the cost of atomic refcount manipulation). The author of the old "free
> threading" patch mentioned only another problem, that is locking
> around name lookups in mutable name dictionaries (for looking up
> object members, globals, etc.), which can also be approached (and I
> think refcount manipulation is a bigger issue).

As I remember that patch, the overhead was around 40%, mostly because
of reference counting. I guess nobody actually goes this path,
because it is such a waste, compared to multiple processes, and doing
it "right" (where I'm referring to some Java achievements I heard of)
is pretty much of a total re-write of Python.

I'm pretty much wondering if the latter makes sense, given the
existence of PyPy.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From p.giarrusso at gmail.com  Mon Mar 30 20:38:31 2009
From: p.giarrusso at gmail.com (Paolo Giarrusso)
Date: Mon, 30 Mar 2009 20:38:31 +0200
Subject: [pypy-dev]
	http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
In-Reply-To: <49D0B197.5070509@stackless.com>
References: <49CACBF5.4020904@stackless.com>
	<9B54C41A-EF31-4BB1-927F-6D02B88A7B41@gmail.com>
	<a4d0c86c0903291628l4caef283s96a0adc7d26e6b54@mail.gmail.com>
	<49D0B197.5070509@stackless.com>
Message-ID: <a4d0c86c0903301138v1cd4d7d5lf294eefd13fd5e81@mail.gmail.com>

On Mon, Mar 30, 2009 at 13:48, Christian Tismer <tismer at stackless.com> wrote:
> On 3/30/09 1:28 AM, Paolo Giarrusso wrote:
>>
>> On Thu, Mar 26, 2009 at 02:42, Leonardo Santagada<santagada at gmail.com>
>> ?wrote:
>>>
>>> On Mar 25, 2009, at 9:27 PM, Christian Tismer wrote:
>>>
>>>> Hi friends,
>>>>
>>>> please have a look at this.
>>>> http://code.google.com/p/unladen-swallow/wiki/ProjectPlan
>>>>
>>>> is this YAIPAP ?
>>>> Yet Another Ignorant Python Acceleration Project
>>>>
>>>> I see them mentioning H?lzle and Self, and I see a reference to
>>>> Psyco where they want to steal from, but PyPy does not exist.
>>>>
>>>> IMHO, this is totally GAGA ?- chris
>>
>> First, I suggest you have a look at the FAQ - when I did, I discovered
>> the developers are full-time Google engineers. Having said that, the
>> Google V8 group does not seem to be involved, and that's quite stupid.
>> However, I studied with Lars Bak and the base ideas he had are the
>> same.
>
> Interesting. Yes, I read the FAQ, but I don't know the people.
>
>> Missing knowledge of PyPy is also stupid obviously, but I wonder why
>> nobody proposed "hey, let's tell them"; fighting another project.
>> They'd sure benefit, for instance, from CALL_METHOD and
>> CALL_LIKELY_BUILTIN ideas (they mention among their purposes fixing
>> the performance problems addressed by CALL_LIKELY_BUILTIN, that's why
>> I mention these).
>
> Well, we could have contacted them.
> But (I'm speaking for myself) if they have decided to totally
> ignore a project like PyPy, then I think it is for a reason.
> Not knowing about PyPy means to close more than two eyes.
> Therefore I see no point in approaching them.

Well, there is a mention of this project on the PyPy blog (just
noticed), and they talk in term of "friendly competition". Since they
seem to know about PyPy, I wonder why it's not listed on the website.
But it's anyway by far incomplete (lots of interesting questions
weren't answered by those pages).

>> Also, they're already started with releasing. Have a look at their
>> benchmarks:
>> http://code.google.com/p/unladen-swallow/wiki/Releases
>>
>> Did you look at that because declaring it GAGA?
>
> My first perception of the project was a bit distorted.
> I saw this as an attempt to replace PyPy with something better,
> and it seemed GAGA to me to do that ignoring prior work.
>
> Now that I realize that the project has much smaller
> goals, it becomes more realistic.
>
> The Q1 goals are relatively doable without doubt. The current
> achievements speedwise remind me of the anxient Python2C project.
> It showed the typical acceleration by a factor of around 2, which
> is what you can expect when eliminating the interpreter loop.
>
>>> I was reading it earlier, the simpler ideas, like making pickle faster
>>> and most of q1 deliverables seems nice, and could really help python
>>> right now, but those seems more like the things Need For Speed sprint
>>> was doing.
>
> Yes, Need for Speed did small, doable things.
>
>>> Not the LLVM-JIT, new GC, eliminate the GIL seems
>>> unrealistic, even the pace of the project seems to be counting on tons
>>> of new developers joining the project after the Q1 milestone.
>>
>> Well, the milestones seem crazy but they did achieve something notable
>> in their Q1 deliverable, but most of the ideas seem correct.
>
> Q1 is fine, but it does by no means give any proof that the next
> milestones can be achieved.
>
>> "Eliminate the GIL" is not hard by itself, the problem (IMHO, no hard
>> numbers for that) is that it's impossible with refcounting (because of
>> the cost of atomic refcount manipulation). The author of the old "free
>> threading" patch mentioned only another problem, that is locking
>> around name lookups in mutable name dictionaries (for looking up
>> object members, globals, etc.), which can also be approached (and I
>> think refcount manipulation is a bigger issue).
>
> As I remember that patch, the overhead was around 40%, mostly because
> of reference counting.

You're right, but I couldn't find someone stating that it was due to
reference counting, on the MLs or on FAQs. The mention of dictionaries
(which are probably also a problem) was from the author of the patch,
Greg Stein:

http://mail.python.org/pipermail/python-dev/2001-August/017099.html

> I guess nobody actually goes this path,
> because it is such a waste, compared to multiple processes, and doing
> it "right" (where I'm referring to some Java achievements I heard of)
> is pretty much of a total re-write of Python.

Well, converting each and every C module from refcounting to garbage
collection is _not_ a total rewrite of Python. I can understand why
you claim it has the same costs. But eXtreme Programming on one side,
and the daily activity of the Linux kernel on another side, show that
given enough manpower such things can be done.

I mean, why CPython doesn't allow having multiple interpreters in the
same process? Because it would take time to move all statics into the
thread state. Well, much bigger changes do take place in the Linux
kernel much more than once per release. Having had experience working
there, I see either a severe lack of manpower, or a lack of trust into
what developers can do.

And even if I say "enough manpower", it still doesn't amount at all to
a complete rewrite. For a human programmer, the conversion is mostly
mechanical: remove (or make conditional) refcount stuff, then in all
functions, register all pointers with the GC (or, in C++, turn them
into handles). If you don't do that over night, you'll get bugs only
in very particular situations. It's obviously boring though, but
programming is not always fun.

> I'm pretty much wondering if the latter makes sense, given the
> existence of PyPy.

Because PyPy still needs lots of time, because the PyPy idea _is_
research, and because if they meet their deadlines (I'm not sure if
it's conceivably possible) Google will save computing power much
earlier. Especially because their C API will be more compatible with
the current one. Well, they actually claim full compatibility, but
this point _is_ IMHO crazy, and they'll notice when they'll have to
remove refcounting.

About the "PyPy is research" statement, just a quote:
http://morepypy.blogspot.com/2008/10/sprint-discussions-jit-generator.html

"Partial evaluation (the basis for our JIT generator) is a 30 years
old technique that was always just promising and never really
successful, so the fact that we think we can solve its problems in a
few years is very much hubris anyway :-). On the positive side, we
think that we now know these problems much better than ever before and
that we have a plan that has a chance to succeed."

I didn't read the post about tracing JITs yet, so I don't know if that
still holds true (I mean, if this is still about partial evaluation).
Anyway, at most it's research approaching completion and production
status, but IMHO nobody imagines getting production stability by the
end of 2009 (that's IMHO though, I might very well be wrong, and I
hope so).

Regards
-- 
Paolo Giarrusso

From lac at openend.se  Tue Mar 31 18:37:57 2009
From: lac at openend.se (Laura Creighton)
Date: Tue, 31 Mar 2009 18:37:57 +0200
Subject: [pypy-dev] the next time we get frustrated with buildbot
Message-ID: <200903311637.n2VGbvFX009404@theraft.openend.se>


Something Open Source that somebody says is better....
I'd never heard of it  before ...

Laura

- ------- Forwarded Message

Return-Path: testing-in-python-bounces at lists.idyll.org
Delivery-Date: Tue Mar 31 18:16:27 2009
Return-Path: <testing-in-python-bounces at lists.idyll.org>
From: Kumar McMillan <kumar.mcmillan at gmail.com>
To: titus at idyll.org
Cc:tip at lists.idyll.org
Subject: Re: [TIP] Everybody wants a pony!
X-BeenThere: testing-in-python at lists.idyll.org

On Tue, Mar 31, 2009 at 9:32 AM, C. Titus Brown <ctb at msu.edu> wrote:
> Hi all,
>
> one thing that became clear at PyCon this year is that everyone wants a
> pony. ?Preferably a pink pony.
>
> (You had to be there.)
>
> Anyway, along the lines of a pony, I am proposing to build or extend a
> replacement for buildbot that will serve those of you who need a simpler
> and more easily-installable continuous interation framework. ?I have a
> few basic ideas, but I'd love to hear what everyone else has to say.

I hate to be "that guy;" I love Python, really I do, but what I love
more is when a great open source tool with great documentation and a
community of maintainers solves a problem better than any other tool.
It makes the decision process easy.

It's called Hudson:
https://hudson.dev.java.net/

Yes, it is written in Java but Java is ubiquitous and Hudson is light
years beyond Buildbot.  Anyone (i.e. non-programmers) can configure
builds and workflows very easily.

If not obvious from the documentation, you can build and test
anything, it doesn't have to be Java.  At Leapfrog we have converted
all of our Python and Ruby buildbots to Hudson (oh did I say that out
loud?) and have never been happier or more efficient in how we do
continuous integration.  You don't need anything special in your tests
to use Hudson but you get some extra details (that buildbot never
provided) if you use xunit style test output.  There are several nose
plugins for this like Nosexunit -- the androgynous plugin.  Example:
nosetests --with-nosexunit

Titus, if you *must* spend your precious spare time (your hourly rate
is what now? priceless?) on this then at the very least make a Python
clone of Hudson :)  But seriously, give it a fair try and I think
you'll like it.

It has a plugin system too.  I assume one could write plugins using Jython even.


>
> Roughly speaking, I'd like to have pony-build do the following, relative
> to buildbot:
>
> ?- "push" results from the client back to the server over an RPC
> ? connection, rather than have the server control the client.
> ? (That is, base it on a polling model rather than a remote control
> ? model.)
>
> ?- Be trivially installable on clients (a single "easy_install" that
> ? works, even for Steve Holden).
>
> ?- A separable and flexible reporting server to support RPC queries and
> ? programmable workflows.
>
> ?- A separable and flexible scheduling/config server to support build
> ? requests and request builds on specific servers.
>
> ?- "Anonymous" push of results, so that Joe Blow users can build,
> ? compile, test, and send the results back to the server for your
> ? examination, without breaking a sweat.
>
> ?- Push of metadata to the reporting server for e.g. communication of
> ? coverage data.
>
> ?- distutils/setuptools and configure/make support.
>
> A lot of these ideas come from comparing DART with buildbot, and having
> good (and bad) experiences with both.
>
> We do have a prototype up and running and I'll try to get a screencast
> up within a week or two.
>
> Your thoughts welcome!
>
> cheers,
> --titus
> --
> C. Titus Brown, ctb at msu.edu
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>

_______________________________________________
testing-in-python mailing list
testing-in-python at lists.idyll.org
http://lists.idyll.org/listinfo/testing-in-python

------- End of Forwarded Message


From jbaker at zyasoft.com  Tue Mar 31 20:36:48 2009
From: jbaker at zyasoft.com (Jim Baker)
Date: Tue, 31 Mar 2009 12:36:48 -0600
Subject: [pypy-dev] the next time we get frustrated with buildbot
In-Reply-To: <200903311637.n2VGbvFX009404@theraft.openend.se>
References: <200903311637.n2VGbvFX009404@theraft.openend.se>
Message-ID: <d03bb4010903311136n502df53wb425e415c4c52646@mail.gmail.com>

Hudson works very well for Jython, so I'd certainly recommend it. Our
version of regrtest has some minor modifications to support JUnit XML, see
there, test.test_support, and test.junit_xml for this code. In particular,
by breaking out our test cases we can more readily observe a change in time
of a specific failing case, so it's quite useful.

You can see it in action here: http://bob.underboss.org:8080/job/jython/

On Tue, Mar 31, 2009 at 10:37 AM, Laura Creighton <lac at openend.se> wrote:

>
> Something Open Source that somebody says is better....
> I'd never heard of it  before ...
>
> Laura
>
> - ------- Forwarded Message
>
> Return-Path: testing-in-python-bounces at lists.idyll.org
> Delivery-Date: Tue Mar 31 18:16:27 2009
> Return-Path: <testing-in-python-bounces at lists.idyll.org>
> From: Kumar McMillan <kumar.mcmillan at gmail.com>
> To: titus at idyll.org
> Cc:tip at lists.idyll.org <Cc%3Atip at lists.idyll.org>
> Subject: Re: [TIP] Everybody wants a pony!
> X-BeenThere: testing-in-python at lists.idyll.org
>
> On Tue, Mar 31, 2009 at 9:32 AM, C. Titus Brown <ctb at msu.edu> wrote:
> > Hi all,
> >
> > one thing that became clear at PyCon this year is that everyone wants a
> > pony.  Preferably a pink pony.
> >
> > (You had to be there.)
> >
> > Anyway, along the lines of a pony, I am proposing to build or extend a
> > replacement for buildbot that will serve those of you who need a simpler
> > and more easily-installable continuous interation framework.  I have a
> > few basic ideas, but I'd love to hear what everyone else has to say.
>
> I hate to be "that guy;" I love Python, really I do, but what I love
> more is when a great open source tool with great documentation and a
> community of maintainers solves a problem better than any other tool.
> It makes the decision process easy.
>
> It's called Hudson:
> https://hudson.dev.java.net/
>
> Yes, it is written in Java but Java is ubiquitous and Hudson is light
> years beyond Buildbot.  Anyone (i.e. non-programmers) can configure
> builds and workflows very easily.
>
> If not obvious from the documentation, you can build and test
> anything, it doesn't have to be Java.  At Leapfrog we have converted
> all of our Python and Ruby buildbots to Hudson (oh did I say that out
> loud?) and have never been happier or more efficient in how we do
> continuous integration.  You don't need anything special in your tests
> to use Hudson but you get some extra details (that buildbot never
> provided) if you use xunit style test output.  There are several nose
> plugins for this like Nosexunit -- the androgynous plugin.  Example:
> nosetests --with-nosexunit
>
> Titus, if you *must* spend your precious spare time (your hourly rate
> is what now? priceless?) on this then at the very least make a Python
> clone of Hudson :)  But seriously, give it a fair try and I think
> you'll like it.
>
> It has a plugin system too.  I assume one could write plugins using Jython
> even.
>
>
> >
> > Roughly speaking, I'd like to have pony-build do the following, relative
> > to buildbot:
> >
> >  - "push" results from the client back to the server over an RPC
> >   connection, rather than have the server control the client.
> >   (That is, base it on a polling model rather than a remote control
> >   model.)
> >
> >  - Be trivially installable on clients (a single "easy_install" that
> >   works, even for Steve Holden).
> >
> >  - A separable and flexible reporting server to support RPC queries and
> >   programmable workflows.
> >
> >  - A separable and flexible scheduling/config server to support build
> >   requests and request builds on specific servers.
> >
> >  - "Anonymous" push of results, so that Joe Blow users can build,
> >   compile, test, and send the results back to the server for your
> >   examination, without breaking a sweat.
> >
> >  - Push of metadata to the reporting server for e.g. communication of
> >   coverage data.
> >
> >  - distutils/setuptools and configure/make support.
> >
> > A lot of these ideas come from comparing DART with buildbot, and having
> > good (and bad) experiences with both.
> >
> > We do have a prototype up and running and I'll try to get a screencast
> > up within a week or two.
> >
> > Your thoughts welcome!
> >
> > cheers,
> > --titus
> > --
> > C. Titus Brown, ctb at msu.edu
> >
> > _______________________________________________
> > testing-in-python mailing list
> > testing-in-python at lists.idyll.org
> > http://lists.idyll.org/listinfo/testing-in-python
> >
>
> _______________________________________________
> testing-in-python mailing list
> testing-in-python at lists.idyll.org
> http://lists.idyll.org/listinfo/testing-in-python
>
> ------- End of Forwarded Message
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
Jim Baker
jbaker at zyasoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20090331/eabac779/attachment-0001.htm