From fijall at gmail.com  Sun Jan  3 10:31:42 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 3 Jan 2010 10:31:42 +0100
Subject: [pypy-dev] RuPy talk is online
Message-ID: <693bc9ab1001030131v1c81982cp25a03a3faf3db10@mail.gmail.com>

My RuPy talk is available online:

http://blip.tv/file/3014306

A sidenote - it was an extremely early talk (8:30 or so), so I'm more
asleep than usual :-)

Cheers,
fijal

From ademan555 at gmail.com  Thu Jan  7 08:59:03 2010
From: ademan555 at gmail.com (Dan Roberts)
Date: Wed, 06 Jan 2010 23:59:03 -0800
Subject: [pypy-dev] Applevel types
Message-ID: <1262851143.28722.7.camel@StormEagle>

    I want to make multiple interpreter-level types appear as a single
application-level type.  Armin got me started in the right direction,
pointing out pypy/objspace/std/{small,}int{type,object}.py to me.  This
made it clear that sharing a typedef will accomplish this, however in
the case of integer objects, only __new__ is registered with the
typedef.  How are other methods registered for object instances?  I
assumed that only methods provided to TypeDef would be visible at the
application-level on instances, however this must not be the case, are
any methods on the interpreter-level classes automatically wrapped?  Can
anyone provide more complex examples or explain exactly how the int
functionality works?

Thanks,
Dan


From amauryfa at gmail.com  Thu Jan  7 09:18:29 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 7 Jan 2010 09:18:29 +0100
Subject: [pypy-dev] Applevel types
In-Reply-To: <1262851143.28722.7.camel@StormEagle>
References: <1262851143.28722.7.camel@StormEagle>
Message-ID: <e27efe131001070018x45679167me990bf0cb5bcb534@mail.gmail.com>

Hi,

2010/1/7 Dan Roberts <ademan555 at gmail.com>:
> ? ?I want to make multiple interpreter-level types appear as a single
> application-level type. ?Armin got me started in the right direction,
> pointing out pypy/objspace/std/{small,}int{type,object}.py to me. ?This
> made it clear that sharing a typedef will accomplish this, however in
> the case of integer objects, only __new__ is registered with the
> typedef. ?How are other methods registered for object instances? ?I
> assumed that only methods provided to TypeDef would be visible at the
> application-level on instances, however this must not be the case, are
> any methods on the interpreter-level classes automatically wrapped? ?Can
> anyone provide more complex examples or explain exactly how the int
> functionality works?

I think this is because int objects have no methods outside the special slots
__float__ __add__ __repr__... which are implemented as MultiMethods:
float__Int in intobject.py, for example.

The magic that converts these functions to type methods is in
pypy.objspace.std.register_all()

-- 
Amaury Forgeot d'Arc

From pedronis at openend.se  Fri Jan  8 15:18:55 2010
From: pedronis at openend.se (Samuele Pedroni)
Date: Fri, 08 Jan 2010 15:18:55 +0100
Subject: [pypy-dev] 1.2 release schedule
Message-ID: <4B473ECF.7000807@openend.se>

Hello, I will send the following mail to pypy-dev later in the day,
please review

Hi, here is the tentative schedule for the upcoming 1.2 release:

- 19th of January: feature freeze (features under consideration are
listed in extradoc/planning/jit.txt with a reference to the release)
- 22th-29th of January: release work (stability, documentation and
packaging) to produce a release candidate on the 29th
- 5th of February: 1.2 release

One of the goals of the release is to gather feedback:
http://morepypy.blogspot.com/2009/12/planning-next-release-of-pypy.html

it would be good if in the period up to finalizing the release people
could try already and think of possible ways to stress out bugs from the
JIT trying it on small/medium examples (CPython own tests which we use
don't tend to loop enough to really use the JIT a lot).

regards

From pedronis at openend.se  Fri Jan  8 15:41:35 2010
From: pedronis at openend.se (Samuele Pedroni)
Date: Fri, 08 Jan 2010 15:41:35 +0100
Subject: [pypy-dev] 1.2 release schedule
In-Reply-To: <4B473ECF.7000807@openend.se>
References: <4B473ECF.7000807@openend.se>
Message-ID: <4B47441F.5090403@openend.se>

Samuele Pedroni wrote:
> Hello, I will send the following mail to pypy-dev later in the day,
> please review
>   
of course ignore these lines, the reviewing I was seeking has happened 
and the result sent

From exarkun at twistedmatrix.com  Sat Jan  9 23:37:49 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sat, 09 Jan 2010 22:37:49 -0000
Subject: [pypy-dev] Module hijinx
Message-ID: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain>

Hello,

On a PyPy I translated this morning, I'm seeing weird behavior importing
a module that messes with sys.modules (not a nice thing to do, but I
would still expect it to work):

exarkun at boson:/tmp$ ls -lR trickypackage/
trickypackage/:
total 4
-rw-r--r-- 1 exarkun exarkun 61 2010-01-09 11:39 foo.py
-rw-r--r-- 1 exarkun exarkun  0 2010-01-09 11:40 __init__.py
exarkun at boson:/tmp$ cat trickypackage/foo.py
import sys
sys.modules['trickypackage.foo'] = "Hello, world"
exarkun at boson:/tmp$ python
Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>from trickypackage import foo
>>>foo
'Hello, world'
>>>
exarkun at boson:/tmp$ 
~/Projects/pypy/trunk/pypy/translator/goal/pypy-c-70469
Python 2.5.2 (70469, Jan 09 2010, 15:24:34)
[PyPy 1.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``PyPy doesn't change the
fundamental physics constants''
>>>from trickypackage import foo
>>>foo
<module 'trickypackage.foo' from '/tmp/trickypackage/foo.py'>
>>>

Anyone have any idea why this might happen?  The real problem this is 
causing for me is with the twisted.internet.reactor module, of course.

Jean-Paul

From exarkun at twistedmatrix.com  Sat Jan  9 23:38:28 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sat, 09 Jan 2010 22:38:28 -0000
Subject: [pypy-dev] Parallel translation?
In-Reply-To: <693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com>
References: <4B192FC9.5040203@lutzhaase.com>
	<1afaf6160912040836m617685ean15d3fdada5a1d7bc@mail.gmail.com>
	<4B194455.6000406@gmail.com>
	<20091205154445.GA18139@code0.codespeak.net>
	<693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com>
Message-ID: <20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain>

On 5 Dec 2009, 04:49 pm, fijall at gmail.com wrote:
>On Sat, Dec 5, 2009 at 4:44 PM, Armin Rigo <arigo at tunes.org> wrote:
>>Hi,
>>
>>On Fri, Dec 04, 2009 at 06:18:13PM +0100, Antonio Cuni wrote:
>>>I agree that at this point in time we cannot or don't want to make
>>>annotation/rtyping/backend parallelizable, but it should definitely 
>>>be
>>>possible to just pass the -j flag to 'make' in an automatic way.
>>
>>Of course, that is full of open problems too. ?The main one is that 
>>each
>>gcc process consumes potentially a lot of RAM, so just passing "-j" is
>>not a great idea, as all gccs are started in parallel. ?It looks like
>>some obscure tweak is needed, like setting -j to a number that depends
>>not only on the number of CPUs (as is classically done) but also on 
>>the
>>total RAM of the system...
>>
>>
>>A bientot,
>>
>>Armin.
>
>I guess the original idea was to have a translation option that is
>passed as -j flag to make, so one can specify what number of jobs he
>wants, instead of trying to guess it automatically.

I poked around on this front a bit.  I couldn't find any code in PyPy 
which invokes make.  I did find 
pypy.translator.platform.distutils_platform.DistutilsPlatform._build, 
though.  This seems to be where lists of C files are sent for 
compilation.  Is that right?

I thought about how to make this parallel.  The cheesy solution, of 
course, would be to start a few threads and have them do the compilation 
(which should be sufficiently parallel, since it's another process 
that's doing the actual work).  This is a bit complicated by the chdir 
calls in the code, though.  Also, maybe distutils isn't threadsafe.

I dunno if I'll think about this any further, but I thought I'd 
summarize what little I did figure out.

Jean-Paul
>
>Cheers,
>fijal
>_______________________________________________
>pypy-dev at codespeak.net
>http://codespeak.net/mailman/listinfo/pypy-dev

From benjamin at python.org  Sun Jan 10 00:51:00 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 9 Jan 2010 17:51:00 -0600
Subject: [pypy-dev] Module hijinx
In-Reply-To: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain>
References: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain>
Message-ID: <1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com>

2010/1/9  <exarkun at twistedmatrix.com>:
> Anyone have any idea why this might happen? ?The real problem this is
> causing for me is with the twisted.internet.reactor module, of course.

Amaury's refactorings may have broken this. I've checked in a fix now.



-- 
Regards,
Benjamin

From exarkun at twistedmatrix.com  Sun Jan 10 04:21:49 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 10 Jan 2010 03:21:49 -0000
Subject: [pypy-dev] Module hijinx
In-Reply-To: <1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com>
References: <20100109223749.8677.2120488026.divmod.xquotient.451@localhost.localdomain>
	<1afaf6161001091551o1b259282g5ea083f51dc41100@mail.gmail.com>
Message-ID: <20100110032149.8677.331083332.divmod.xquotient.453@localhost.localdomain>

On 9 Jan, 11:51 pm, benjamin at python.org wrote:
>2010/1/9  <exarkun at twistedmatrix.com>:
>>Anyone have any idea why this might happen? ?The real problem this is
>>causing for me is with the twisted.internet.reactor module, of course.
>
>Amaury's refactorings may have broken this. I've checked in a fix now.

Thanks, your fix appears to have worked.

Jean-Paul

From amauryfa at gmail.com  Sun Jan 10 15:49:15 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Sun, 10 Jan 2010 15:49:15 +0100
Subject: [pypy-dev] Parallel translation?
In-Reply-To: <20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain>
References: <4B192FC9.5040203@lutzhaase.com>
	<1afaf6160912040836m617685ean15d3fdada5a1d7bc@mail.gmail.com>
	<4B194455.6000406@gmail.com>
	<20091205154445.GA18139@code0.codespeak.net>
	<693bc9ab0912050849m3f9441cpf6e6e2b53a08857a@mail.gmail.com>
	<20100109223828.8677.116612598.divmod.xquotient.452@localhost.localdomain>
Message-ID: <e27efe131001100649x67083a72s441439d956997294@mail.gmail.com>

Hello,

2010/1/9  <exarkun at twistedmatrix.com>:
> On 5 Dec 2009, 04:49 pm, fijall at gmail.com wrote:
>> I guess the original idea was to have a translation option that is
>> passed as -j flag to make, so one can specify what number of jobs he
>> wants, instead of trying to guess it automatically.
>
> I poked around on this front a bit. ?I couldn't find any code in PyPy which
> invokes make. ?I did find
> pypy.translator.platform.distutils_platform.DistutilsPlatform._build,
> though. ?This seems to be where lists of C files are sent for compilation.
> ?Is that right?

PyPy does generate a makefile (gen_makefile() in
http://codespeak.net/svn/pypy/trunk/pypy/translator/c/genc.py )

but it is not used in all configurations:
In the same file, see the call to execute_makefile().
-Ojit uses the makefile, though.

-- 
Amaury Forgeot d'Arc

From exarkun at twistedmatrix.com  Sun Jan 10 17:55:22 2010
From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com)
Date: Sun, 10 Jan 2010 16:55:22 -0000
Subject: [pypy-dev] os.openpty
Message-ID: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain>

Hi all,

I just checked in an implementation of os.openpty:

https://codespeak.net/viewvc/?view=rev&revision=70477

It's not all that exciting (unless you were missing it before), but it's 
my first PyPy commit in a while so I thought I'd point it out in case 
there's anything horribly wrong with it.  I did add a test and make sure 
translation on Linux 32 still works.  Hopefully I didn't introduce any 
problems on any other platforms.

Jean-Paul

From cfbolz at gmx.de  Tue Jan 12 15:17:57 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Tue, 12 Jan 2010 15:17:57 +0100
Subject: [pypy-dev] os.openpty
In-Reply-To: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain>
References: <20100110165522.8677.302848211.divmod.xquotient.462@localhost.localdomain>
Message-ID: <4B4C8495.7070802@gmx.de>

Hi Jean-Paul,

On 01/10/2010 05:55 PM, exarkun at twistedmatrix.com wrote:
> I just checked in an implementation of os.openpty:
>
> https://codespeak.net/viewvc/?view=rev&revision=70477
>
> It's not all that exciting (unless you were missing it before), but it's
> my first PyPy commit in a while so I thought I'd point it out in case
> there's anything horribly wrong with it.  I did add a test and make sure
> translation on Linux 32 still works.  Hopefully I didn't introduce any
> problems on any other platforms.

Not that I have a clue what the thing does, but the commit looks fine to 
me :-). The nightly tests also don't show anything suspicious.

Thanks for doing this!

Carl Friedrich

From fijall at gmail.com  Sat Jan 23 02:23:40 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 23 Jan 2010 02:23:40 +0100
Subject: [pypy-dev] profiling thoughts
Message-ID: <693bc9ab1001221723t4b8c45bfo96cb08a0c11bb278@mail.gmail.com>

As discussed with Samuele & others, we have to decide what to do with
profiling, since cProfile information is sort of pointless. Unladen
swallow people seem (with success) to interface oprofile, which also
has hooks for JVM, so it's already easier.

Links:

http://oprofile.sourceforge.net/doc/devel/index.html
http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ExecutionEngine/JIT/OProfileJITEventListener.cpp?view=markup

any thoughts?

Cheers,
fijal

From fijall at gmail.com  Sun Jan 24 20:12:47 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 24 Jan 2010 20:12:47 +0100
Subject: [pypy-dev] New URL for buildbot
Message-ID: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com>

We have much nicer url for buildbot these days:

http://buildbot.pypy.org

Cheers,
fijal

From cfbolz at gmx.de  Mon Jan 25 15:01:46 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 25 Jan 2010 15:01:46 +0100
Subject: [pypy-dev] Call for Papers: DYLA @ TOOLS 2010
Message-ID: <4B5DA44A.4070302@gmx.de>

... apologies for cross-posting and multiple copies!

4th Workshop on Dynamic Languages and Applications (DYLA 2010)

categories: programming languages, software engineering, computer science

When: Jun 28, 2010
Where: Malaga, Spain
Submission Deadline: Mar 31, 2010
Notification Due: May 14, 2010
Final Version Due: May 31, 2010
link: http://scg.unibe.ch/wiki/events/dyla2010

Call For Papers

The DYLA Workshop series focuses on the revival of dynamic languages.
These days, dynamic languages (like Ruby, Python, JavaScript, Lua,
etc?) are getting ever more popular. This is a call to arms for
academia! We need to explore the future of dynamic languages through
its human aspects and technical issues. We also ought to look back and
pick up solutions from existing dynamic languages (such as Scheme,
Smalltalk, or Self) to be rediscovered and spread around.

Goal and Topics

The goal of this workshop is to act as a forum where we can discuss
dynamic languages and their applications. Topics of interest include
any topic relevant in applying and/or supporting dynamic languages:
Ruby, Python, Groovy, JavaScript, Lua, Clojure, Lisp, Scheme,
Smalltalk, Self, ABCL, Prolog, and many more?

Human aspects of dynamic languages, such as
- empirical studies about the application of dynamic languages
- best practices and patterns specific to dynamic languages
- program correctness through unit testing (as opposed to types)
- improved or novel IDE support for dynamic languages
- use of dynamic features by library & framework developers
- scripting of static application with dynamic languages
- reverse engineering and analysis of dynamic applications

Technical aspects of dynamic languages, such as
- what features make a language a dynamic one?
- agents, actors, active object, distribution, concurrency and mobility
- delegation, prototypes, mix-ins, traits
- first-class closures, continuations, environments
- reflection and meta-programming
- automated reasoning about programs written in dynamic languages
- (concurrent/distributed/mobile/aspect) virtual machines
- optimization of dynamic languages
- multi-paradigm & static/dynamic-marriages
- type systems for dynamic languages
- (dynamic) aspects for dynamic languages
- higher-order objects & messages
- ?other exotic dynamic features

Submissions

The workshop will have a demo-oriented style. The idea is to allow
participants to demonstrate new and interesting features and discuss
what they feel is relevant for the dynamic language community.
Participants need to submit a 2?4 page position paper of their work in
ACM, sig-alternate.cls format. At the workshop, participants will be
asked to give 10-minute ?lightning demos? of their contributions.

Submission page is https://www.easychair.org/login.cgi?conf=dyla10

Organizers

- Alexandre Bergel, Univ of Chile,
- Carl Friedrich Bolz, Heinrich-Heine-Univ, D?sseldorf, Germany
- Simon Denier, INRIA Lille, France
- Michael Haupt, HPI Potsdam, Germany
- Adrian Kuhn, Univ of Bern, Switzerland

Program Committee

- Tom Dinkelaker, Technische Univ Darmstadt, Germany
- Johan Fabry, Univ of Chile
- Matthew Flatt, Univ of Utah, USA
- Stephan Herrmann, TU Berlin, Germany
- Abram Hindle, Univ of Waterloo, Canada
- Kasper Lund, Google, Denmark.
- Michael Perscheid, HPI Potsdam, Germany
- Rodolfo Toledo, Univ of Chile
- Niko Schwarz, Univ of Bern, Switzerland
- Peter Sommerlad, HSR Rapperswil, Switzerland
- Alessandro Warth, Viewpoints Research Institute, USA
- Vadim Zaytsev, Univ of Koblenz, Germany

For further information, please visit our website or follow us on twitter
- http://bit.ly/dyla2010
- http://twitter.com/dyla2010


Carl Friedrich Bolz

From fijall at gmail.com  Sun Jan 31 15:31:20 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 31 Jan 2010 15:31:20 +0100
Subject: [pypy-dev] [pypy-svn] r70985 - pypy/branch/multijit
In-Reply-To: <20100130145843.3A7DC1680AB@codespeak.net>
References: <20100130145843.3A7DC1680AB@codespeak.net>
Message-ID: <693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com>

Hello.

My proposition would be not to add new features to tracing until we
fix bugs. Bugs being segfaults when you try running stuff on top of
pypy-c-jit like apptests or translate.py (smalltalk target crashes
every few tries).

Cheers,
fijal

On Sat, Jan 30, 2010 at 3:58 PM,  <arigo at codespeak.net> wrote:
> Author: arigo
> Date: Sat Jan 30 15:58:41 2010
> New Revision: 70985
>
> Added:
> ? pypy/branch/multijit/
> ? ? ?- copied from r70984, pypy/trunk/
> Log:
> A branch in which to support having multiple JITs
> in the same RPython program.
>
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>

From arigo at tunes.org  Sun Jan 31 17:21:37 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 31 Jan 2010 17:21:37 +0100
Subject: [pypy-dev] New URL for buildbot
In-Reply-To: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com>
References: <693bc9ab1001241112o3dbe41e1p6b892c1a45fdadae@mail.gmail.com>
Message-ID: <20100131162137.GA31984@code0.codespeak.net>

Hi,

On Sun, Jan 24, 2010 at 08:12:47PM +0100, Maciej Fijalkowski wrote:
> We have much nicer url for buildbot these days:
> 
> http://buildbot.pypy.org

For reference, the "older" plots are at:
    http://codespeak.net/pypy/jitplots.html

I mention them again because they give more information: basically you
can get the graph of every number printed at the end of a pypy-c-jit
run, like how much time was spent in the backend or how often it aborted
tracing for various reasons.


A bientot,

Armin.

From arigo at tunes.org  Sun Jan 31 17:25:06 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 31 Jan 2010 17:25:06 +0100
Subject: [pypy-dev] [pypy-svn] r70985 - pypy/branch/multijit
In-Reply-To: <693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com>
References: <20100130145843.3A7DC1680AB@codespeak.net>
	<693bc9ab1001310631k90af4dcj1c7a20a9c6ab61a3@mail.gmail.com>
Message-ID: <20100131162506.GB31984@code0.codespeak.net>

Hi Maciej,

On Sun, Jan 31, 2010 at 03:31:20PM +0100, Maciej Fijalkowski wrote:
> My proposition would be not to add new features to tracing until we
> fix bugs. Bugs being segfaults when you try running stuff on top of
> pypy-c-jit like apptests or translate.py (smalltalk target crashes
> every few tries).

Sorry, it's already done :-)  We can now translate RPython programs
containing several JITting interpreters.  There are of course details to
fix but basically it just works.


A bientot,

Armin.

From eva_maia at sapo.pt  Tue Feb  2 15:06:45 2010
From: eva_maia at sapo.pt (eva_maia at sapo.pt)
Date: Tue, 02 Feb 2010 14:06:45 +0000
Subject: [pypy-dev] Type inference of PyPy
Message-ID: <20100202140645.92365nra6hdfzwcg@w18.mail.sapo.pt>


Hi,

  I'm a Masters student in CS at University of Porto and the purpose  
of my thesis is the development of a Why module for Python.

  Why is a software verification platform (VCG). There is already such  
tools for C (frama-c) and Java (krakatoa).

To that end, I need to infer the type of program variables. I'd like  
to use type inference of PyPy (for  RPython). I know there is a  
statement (x.knowntype) that given the type of the arguments it gives  
the type of return variable, but I want to know if it is also possible  
to access the type of local variables.


Thanks,

Eva.

From arigo at tunes.org  Tue Feb  2 17:17:51 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Feb 2010 17:17:51 +0100
Subject: [pypy-dev] Type inference of PyPy
In-Reply-To: <20100202140645.92365nra6hdfzwcg@w18.mail.sapo.pt>
References: <20100202140645.92365nra6hdfzwcg@w18.mail.sapo.pt>
Message-ID: <20100202161751.GA10351@code0.codespeak.net>

Hi,

On Tue, Feb 02, 2010 at 02:06:45PM +0000, eva_maia at sapo.pt wrote:
> To that end, I need to infer the type of program variables. I'd like  
> to use type inference of PyPy (for  RPython).

Two warnings before you go any further (I guess you already know about
it but it doesn't hurt repeating it):

1. You can't infer the type of most program variables in Python:
http://codespeak.net/pypy/dist/pypy/doc/faq.html#can-pypy-compile-normal-python-programs

2. PyPy's type inference only works for RPython, not for Python
   programs.

> I know there is a  statement (x.knowntype) that given the type of the
> arguments it gives  the type of return variable, but I want to know if
> it is also possible  to access the type of local variables.

In the RPythonAnnotator instance there is a dictionary called 'bindings'
that maps local variables to the corresponding SomeXxx instances.  What
you don't get "out of the box" is the mapping from the real local
variables in your Python source code -- only from Variable instances,
which are produced by the flow object space.


A bientot,

Armin.

From reynaudd at loria.fr  Mon Feb  8 20:52:14 2010
From: reynaudd at loria.fr (Daniel Reynaud)
Date: Mon, 8 Feb 2010 20:52:14 +0100
Subject: [pypy-dev] virtual machine translation tutorial
Message-ID: <6dbefc261002081152m8fd8a88ofa7cbc2076b77136@mail.gmail.com>

Hello pypy folk,

For the past 2 days, I have struggled with virtual machine translation
and JIT'ing. I relied heavily on the patience of everyone on #pypy, so
I tried to aggregate the accumulated knowledge in a blog post.

Here it is: http://indefinitestudies.org/2010/02/08/creating-a-toy-virtual-machine-with-pypy/

Cheers,
dan

From cfbolz at gmx.de  Thu Feb 11 13:15:19 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 11 Feb 2010 13:15:19 +0100
Subject: [pypy-dev] =?iso-8859-1?q?cobra_server_at_the_Uni_D=FCsseldorf?=
Message-ID: <4B73F4D7.1040508@gmx.de>

Hi all,

we need to do some server restructuring at the university and in the 
process need to use Cobra for other things (we will get some other 
machine back after the reshuffling). Sorry about the disturbance. Could 
people please check their home directories for things that need to be 
kept and move them elsewhere? We plan to shut Cobra down on the 19th of 
February, if somebody cannot make it till then, please send me a mail.

Cheers,

Carl Friedrich

From ncbray at gmail.com  Thu Feb 18 21:36:39 2010
From: ncbray at gmail.com (Nick Bray)
Date: Thu, 18 Feb 2010 14:36:39 -0600
Subject: [pypy-dev] Modeling attributes in the annotator
Message-ID: <5d66556e1002181236g2b3ec989kbeeee713430ab90a@mail.gmail.com>

I'm doing a lit review on Python analysis techniques.  Right now I'm
trying to understand how PyPy's annotator models object attributes.
Are the descriptions on the web page accurate?  There are two corner
cases that I do not understand how the annotator, as described, deals
with.

1)
o.f = 1
??? = o.__dict__['f']

2) Any attribute lookup involving a descriptor that is not a function object.

I realize the annotator is only intended to work for RPython. Given
the information I have, however, I do not see what restrictions in
RPython would disallow these cases.  ("They just are" is an acceptable
answer, I just want to make sure I'm not being particularly dense and
missing something.)

From benjamin at python.org  Thu Feb 18 22:12:58 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 18 Feb 2010 15:12:58 -0600
Subject: [pypy-dev] Modeling attributes in the annotator
In-Reply-To: <5d66556e1002181236g2b3ec989kbeeee713430ab90a@mail.gmail.com>
References: <5d66556e1002181236g2b3ec989kbeeee713430ab90a@mail.gmail.com>
Message-ID: <1afaf6161002181312p427d10aco474f339216f65133@mail.gmail.com>

2010/2/18 Nick Bray <ncbray at gmail.com>:
> I'm doing a lit review on Python analysis techniques. ?Right now I'm
> trying to understand how PyPy's annotator models object attributes.
> Are the descriptions on the web page accurate? ?There are two corner
> cases that I do not understand how the annotator, as described, deals
> with.
>
> 1)
> o.f = 1
> ??? = o.__dict__['f']
>
> 2) Any attribute lookup involving a descriptor that is not a function object.
>
> I realize the annotator is only intended to work for RPython. Given
> the information I have, however, I do not see what restrictions in
> RPython would disallow these cases. ?("They just are" is an acceptable
> answer, I just want to make sure I'm not being particularly dense and
> missing something.)

They just are. __dict__ isn't RPython.



-- 
Regards,
Benjamin

From fijall at gmail.com  Thu Feb 18 22:58:20 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 18 Feb 2010 16:58:20 -0500
Subject: [pypy-dev] Modeling attributes in the annotator
In-Reply-To: <1afaf6161002181312p427d10aco474f339216f65133@mail.gmail.com>
References: <5d66556e1002181236g2b3ec989kbeeee713430ab90a@mail.gmail.com> 
	<1afaf6161002181312p427d10aco474f339216f65133@mail.gmail.com>
Message-ID: <693bc9ab1002181358r132078b6x4c02dc6b5216fc7@mail.gmail.com>

On Thu, Feb 18, 2010 at 4:12 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/2/18 Nick Bray <ncbray at gmail.com>:
>> I'm doing a lit review on Python analysis techniques. ?Right now I'm
>> trying to understand how PyPy's annotator models object attributes.
>> Are the descriptions on the web page accurate? ?There are two corner
>> cases that I do not understand how the annotator, as described, deals
>> with.
>>
>> 1)
>> o.f = 1
>> ??? = o.__dict__['f']
>>
>> 2) Any attribute lookup involving a descriptor that is not a function object.
>>
>> I realize the annotator is only intended to work for RPython. Given
>> the information I have, however, I do not see what restrictions in
>> RPython would disallow these cases. ?("They just are" is an acceptable
>> answer, I just want to make sure I'm not being particularly dense and
>> missing something.)
>
> They just are. __dict__ isn't RPython.
>
>
>
> --
> Regards,
> Benjamin
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

There is a bit of documentation on RPython.

http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html#restricted-python

From arigo at tunes.org  Mon Feb 22 16:05:12 2010
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 22 Feb 2010 16:05:12 +0100
Subject: [pypy-dev] Graphs moved: cobra->wyvern
Message-ID: <20100222150511.GA27169@code0.codespeak.net>

Hi all,

The 'cobra' machine is going down, so I moved the graph plot servers on
'wyvern'.  For a reminder, they are:

    without JIT: http://wyvern.cs.uni-duesseldorf.de:8098/index.html

    with JIT: http://codespeak.net/pypy/jitplots.html


A bientot,

Armin.

From arigo at tunes.org  Thu Feb 25 10:18:04 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Feb 2010 10:18:04 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
Message-ID: <20100225091804.GA15891@code0.codespeak.net>

Hi Miquel,

May I point out a couple of comments about http://speed.pypy.org/ ?
The first is that it looks great, indeed; thank you very much for doing
such a site! :-)

The comments are mostly about the graphics in /timeline: 

* they should also display, or allow to display, the speed of CPython
  for comparison;

* they should have a longer maximum history than 100, to see more
  clearly the long-term evolution.


All in all it looks great!

A bientot,
Armin.

From santagada at gmail.com  Thu Feb 25 14:39:12 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 25 Feb 2010 10:39:12 -0300
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <20100225091804.GA15891@code0.codespeak.net>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>
Message-ID: <F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>

I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).

- When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).

- Overview window comments:
	- rename the column "result" to "time" (or timing?);
	- create a new column called "previous time" with the last time of the previous revision.
	- rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%);
	- with the last two ones it will make understanding the improvement and trends a lot easier;
	- trend should have the number of revisions on the label (and the values reversed like the "improvement" column);
	- rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup";
	- the graph should say what it is graphing;
	- maybe invert the order of results.

- Timeline window comments:
	- showing all benchmarks in the same page would be good
	- Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly;
	- see all benchmarks together, so you don't have to hunt around (maybe have a detailed view);
	- describe all axis;
	- as armin said have a selected by default cpython comparison so people can have a baseline.


I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.


On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:

> Hi Miquel,
> 
> May I point out a couple of comments about http://speed.pypy.org/ ?
> The first is that it looks great, indeed; thank you very much for doing
> such a site! :-)
> 
> The comments are mostly about the graphics in /timeline: 
> 
> * they should also display, or allow to display, the speed of CPython
>  for comparison;
> 
> * they should have a longer maximum history than 100, to see more
>  clearly the long-term evolution.
> 
> 
> All in all it looks great!
> 
> A bientot,
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

--
Leonardo Santagada
santagada at gmail.com




From tobami at googlemail.com  Thu Feb 25 16:10:16 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Feb 2010 16:10:16 +0100
Subject: [pypy-dev] speed.pypy.org launched
Message-ID: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>

As some of you already know, there is a new performance site online:
speed.pypy.org.

In Autumn last year I offered my help with a benchmarking website, and
eventually for a new main site for the project. What you can see now
is a work in progress, which I hope to improve in the next months. I
called it Codespeed, and in time it could become a framework for other
open source projects to use (for example javascript implementation
development, connected to a forge, etc...).

But right now, speed has two goals:
- To have a powerful tool for the devs to analyse performance and
detect regressions in the different pypy interpreters.
- To improve the visibility of the project by letting python
developers(users) have a look themselves at current performance and to
compare pypy to other implementations, not having to wait for some dev
to manually put together the graphs and post them on the blog.

To that end, there are currently two views.
The Overview does what it's name says, and allows a developer to have
a broad view of how last revisions are affecting performance, and
immediately spot regressions. If the user wants to closer examine the
changes in the performance of a given benchmark, just clicking on it
takes you to the the timeline for the selected benchmark.

Both views will get some more features, and I have further plans down
the pipeline:
- A Comparison tab: graph bars directly comparing performance of
different releases (pypy 1.2, trunk, cpython 2.x, unladen swallow,
Jython, etc.)
- A statistics tab: Go through regression test literature to see if we
can have good metrics shown.
- RSS feed/email alerts (for big regressions)
- svn info integration
- etc...

In the end it is a site for the pypy devs, so I hope to get a lot of
feedback from you so that it caters exactly to your needs.

Have a nice day!
Miquel

PD: When it is a bit more complete I'll write a post for the PyPy
status blog so it gets wider known.

From tobami at googlemail.com  Thu Feb 25 16:00:28 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Feb 2010 16:00:28 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>
	<F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
Message-ID: <c561346d1002250700v5c7706f5of44c9ec73dd8e669@mail.gmail.com>

Thanks for the nice comments.

I was waiting for the test hook to be implemented before emailing
pypy-dev about it, but seeing as It already got out I may do it now as
well ;-)

@Armin
>* they should also display, or allow to display, the speed of CPython
 >for comparison;
Already planned.

>* they should have a longer maximum history than 100, to see more
 >clearly the long-term evolution.
Done

@Leonardo
phew! those were a lot of suggestions, thanks. I will start
implementing some of them, though I would like the changes to be
discussed so that they get done right and all agree on what's best (in
the cases where it can't be an option).

I already implemented a couple of changes suggested by fijal:
logarithmic scale for the overview bars and variable width so it fits
on small screens, down to 980px width. I hope he is happy now :-)

I'll start a new thread for the explanation and plans.
Cheers!

2010/2/25 Leonardo Santagada <santagada at gmail.com>:
> I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
>
> - When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
>
> - Overview window comments:
> ? ? ? ?- rename the column "result" to "time" (or timing?);
> ? ? ? ?- create a new column called "previous time" with the last time of the previous revision.
> ? ? ? ?- rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%);
> ? ? ? ?- with the last two ones it will make understanding the improvement and trends a lot easier;
> ? ? ? ?- trend should have the number of revisions on the label (and the values reversed like the "improvement" column);
> ? ? ? ?- rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup";
> ? ? ? ?- the graph should say what it is graphing;
> ? ? ? ?- maybe invert the order of results.
>
> - Timeline window comments:
> ? ? ? ?- showing all benchmarks in the same page would be good
> ? ? ? ?- Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly;
> ? ? ? ?- see all benchmarks together, so you don't have to hunt around (maybe have a detailed view);
> ? ? ? ?- describe all axis;
> ? ? ? ?- as armin said have a selected by default cpython comparison so people can have a baseline.
>
>
> I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
>
>
> On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
>
>> Hi Miquel,
>>
>> May I point out a couple of comments about http://speed.pypy.org/ ?
>> The first is that it looks great, indeed; thank you very much for doing
>> such a site! :-)
>>
>> The comments are mostly about the graphics in /timeline:
>>
>> * they should also display, or allow to display, the speed of CPython
>> ?for comparison;
>>
>> * they should have a longer maximum history than 100, to see more
>> ?clearly the long-term evolution.
>>
>>
>> All in all it looks great!
>>
>> A bientot,
>> Armin.
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
>

From tobami at googlemail.com  Thu Feb 25 18:05:16 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Feb 2010 18:05:16 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>
	<F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
Message-ID: <c561346d1002250905k2f112efemc5ed84c695581088@mail.gmail.com>

I have implemented some of Leonardo's suggestions:

>- Overview window comments:
 >      - rename the column "result" to "time" (or timing?);
done
>       - trend should have the number of revisions on the label (and the values reversed like the "improvement" column);
done
>       - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup";
>       - the graph should say what it is graphing;
it is clearer now, plus there are some tooltips on hover.

>- Timeline window comments:
>       - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly;
only interpreter name shown now. (order implemented but not commited).

I'll see about the rest when I have time.

Miquel


2010/2/25 Leonardo Santagada <santagada at gmail.com>:
> I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
>
> - When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
>
> - Overview window comments:
> ? ? ? ?- rename the column "result" to "time" (or timing?);
> ? ? ? ?- create a new column called "previous time" with the last time of the previous revision.
> ? ? ? ?- rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%);
> ? ? ? ?- with the last two ones it will make understanding the improvement and trends a lot easier;
> ? ? ? ?- trend should have the number of revisions on the label (and the values reversed like the "improvement" column);
> ? ? ? ?- rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup";
> ? ? ? ?- the graph should say what it is graphing;
> ? ? ? ?- maybe invert the order of results.
>
> - Timeline window comments:
> ? ? ? ?- showing all benchmarks in the same page would be good
> ? ? ? ?- Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly;
> ? ? ? ?- see all benchmarks together, so you don't have to hunt around (maybe have a detailed view);
> ? ? ? ?- describe all axis;
> ? ? ? ?- as armin said have a selected by default cpython comparison so people can have a baseline.
>
>
> I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
>
>
> On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
>
>> Hi Miquel,
>>
>> May I point out a couple of comments about http://speed.pypy.org/ ?
>> The first is that it looks great, indeed; thank you very much for doing
>> such a site! :-)
>>
>> The comments are mostly about the graphics in /timeline:
>>
>> * they should also display, or allow to display, the speed of CPython
>> ?for comparison;
>>
>> * they should have a longer maximum history than 100, to see more
>> ?clearly the long-term evolution.
>>
>>
>> All in all it looks great!
>>
>> A bientot,
>> Armin.
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
>

From cfbolz at gmx.de  Thu Feb 25 18:38:16 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Thu, 25 Feb 2010 18:38:16 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
Message-ID: <4B86B588.4000601@gmx.de>

Hi Miquel,

On 02/25/2010 04:10 PM, Miquel Torres wrote:
> As some of you already know, there is a new performance site online:
> speed.pypy.org.
[...]

I'm quite impressed, this is very cool work and a good improvement over 
the current plots. Thanks for doing this!

One thing that I would really find useful are error bars in the timeline 
view. This would help us judge whether up-and-down movements are within 
the margin of randomness, or whether it is a real effect. I don't know 
how annoying they are to implement though, no clue whether the plot lib 
supports that. There should be enough information about errors in the 
json files, as far as I remember.

Cheers,

Carl Friedrich

From tobami at googlemail.com  Thu Feb 25 22:10:29 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Feb 2010 22:10:29 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <4B86B588.4000601@gmx.de>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de>
Message-ID: <c561346d1002251310g563c244cp1bc2846a12bf7f65@mail.gmail.com>

Hi Carl,

error bars are certainly doable and supported by the plot lib. The
only problem is that my backend model doesn't include such a field, so
at the moment the data (which is certainly included in the json files)
is not being saved. I will need to change the DB models in order to
acomodate that kind of information.

I'll add this to the wish list.

Cheers,
Miquel


2010/2/25 Carl Friedrich Bolz <cfbolz at gmx.de>:
> Hi Miquel,
>
> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>> As some of you already know, there is a new performance site online:
>> speed.pypy.org.
> [...]
>
> I'm quite impressed, this is very cool work and a good improvement over
> the current plots. Thanks for doing this!
>
> One thing that I would really find useful are error bars in the timeline
> view. This would help us judge whether up-and-down movements are within
> the margin of randomness, or whether it is a real effect. I don't know
> how annoying they are to implement though, no clue whether the plot lib
> supports that. There should be enough information about errors in the
> json files, as far as I remember.
>
> Cheers,
>
> Carl Friedrich
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From jacob at openend.se  Thu Feb 25 23:39:28 2010
From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Thu, 25 Feb 2010 23:39:28 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002251310g563c244cp1bc2846a12bf7f65@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de>
	<c561346d1002251310g563c244cp1bc2846a12bf7f65@mail.gmail.com>
Message-ID: <201002252339.28334.jacob@openend.se>

Thursday 25 February 2010 you wrote:
> Hi Carl,
> 
> error bars are certainly doable and supported by the plot lib. The
> only problem is that my backend model doesn't include such a field, so
> at the moment the data (which is certainly included in the json files)
> is not being saved. I will need to change the DB models in order to
> acomodate that kind of information.
> 
> I'll add this to the wish list.

How do you actually determine the error? Multiple runs or sliding average of 
historic data?

Jacob

From stephen at thorne.id.au  Fri Feb 26 01:04:21 2010
From: stephen at thorne.id.au (Stephen Thorne)
Date: Fri, 26 Feb 2010 10:04:21 +1000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
Message-ID: <20100226000421.GA2304@thorne.id.au>

On 2010-02-25, Miquel Torres wrote:
> To that end, there are currently two views.
> The Overview does what it's name says, and allows a developer to have
> a broad view of how last revisions are affecting performance, and
> immediately spot regressions. If the user wants to closer examine the
> changes in the performance of a given benchmark, just clicking on it
> takes you to the the timeline for the selected benchmark.

I have a simple question :) 

On the 'Timeline' view there is a graph. I realise the x axis is
'revision', but what is the y axis? It is not labelled and has no units.

-- 
Regards,
Stephen Thorne
Development Engineer
Netbox Blue

From renesd at gmail.com  Fri Feb 26 09:26:48 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 26 Feb 2010 08:26:48 +0000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
Message-ID: <64ddb72c1002260026i74b49b85xdb4a0789dc526ccb@mail.gmail.com>

Very cool :)

pypy is showing great progress!

From cfbolz at gmx.de  Fri Feb 26 10:54:01 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 26 Feb 2010 10:54:01 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <201002252339.28334.jacob@openend.se>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de>
	<c561346d1002251310g563c244cp1bc2846a12bf7f65@mail.gmail.com>
	<201002252339.28334.jacob@openend.se>
Message-ID: <4B879A39.2040606@gmx.de>

On 02/25/2010 11:39 PM, Jacob Hall?n wrote:
> Thursday 25 February 2010 you wrote:
>> Hi Carl,
>>
>> error bars are certainly doable and supported by the plot lib. The
>> only problem is that my backend model doesn't include such a field, so
>> at the moment the data (which is certainly included in the json files)
>> is not being saved. I will need to change the DB models in order to
>> acomodate that kind of information.
>>
>> I'll add this to the wish list.

Great, thanks.

> How do you actually determine the error? Multiple runs or sliding average of
> historic data?

We use the Unladen Swallow benchmark runner, which runs each benchmark 
multiple times and computes the standard deviation. I then computes a 
95% confidence interval using, I think, Student's T distribution.

Cheers,

Carl Friedrich

From stefan_ml at behnel.de  Fri Feb 26 10:59:49 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 26 Feb 2010 10:59:49 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <4B86B588.4000601@gmx.de>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de>
Message-ID: <hm862m$cer$1@dough.gmane.org>

Carl Friedrich Bolz, 25.02.2010 18:38:
> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>> As some of you already know, there is a new performance site online:
>> speed.pypy.org.
> [...]
> 
> I'm quite impressed, this is very cool work and a good improvement over 
> the current plots. Thanks for doing this!
> 
> One thing that I would really find useful are error bars in the timeline 
> view. This would help us judge whether up-and-down movements are within 
> the margin of randomness, or whether it is a real effect. I don't know 
> how annoying they are to implement though, no clue whether the plot lib 
> supports that. There should be enough information about errors in the 
> json files, as far as I remember.

The standard deviation is commonly considered meaningless for benchmark
results. All that really matters is the fastest run, everything else is
just fog.

Read the docs on timeit, for example.

Stefan


From tobami at googlemail.com  Fri Feb 26 11:05:32 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Fri, 26 Feb 2010 11:05:32 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <hm862m$cer$1@dough.gmane.org>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
Message-ID: <c561346d1002260205k2aeed582i1bfc545a6530f856@mail.gmail.com>

You may also consider that a benchmark that varies greatly between
runs may be a flawed benchmark.

I think it should be considered, but only on the running side, and act
accordingly (too high a deviation: discard run, reconsider benchmark,
reconsider environment or whatever).

Miquel


2010/2/26 Stefan Behnel <stefan_ml at behnel.de>:
> Carl Friedrich Bolz, 25.02.2010 18:38:
>> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>>> As some of you already know, there is a new performance site online:
>>> speed.pypy.org.
>> [...]
>>
>> I'm quite impressed, this is very cool work and a good improvement over
>> the current plots. Thanks for doing this!
>>
>> One thing that I would really find useful are error bars in the timeline
>> view. This would help us judge whether up-and-down movements are within
>> the margin of randomness, or whether it is a real effect. I don't know
>> how annoying they are to implement though, no clue whether the plot lib
>> supports that. There should be enough information about errors in the
>> json files, as far as I remember.
>
> The standard deviation is commonly considered meaningless for benchmark
> results. All that really matters is the fastest run, everything else is
> just fog.
>
> Read the docs on timeit, for example.
>
> Stefan
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From cfbolz at gmx.de  Fri Feb 26 11:25:50 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 26 Feb 2010 11:25:50 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <hm862m$cer$1@dough.gmane.org>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	<4B86B588.4000601@gmx.de>
	<hm862m$cer$1@dough.gmane.org>
Message-ID: <4B87A1AE.90903@gmx.de>

On 02/26/2010 10:59 AM, Stefan Behnel wrote:
> Carl Friedrich Bolz, 25.02.2010 18:38:
>> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>>> As some of you already know, there is a new performance site online:
>>> speed.pypy.org.
>> [...]
>>
>> I'm quite impressed, this is very cool work and a good improvement over
>> the current plots. Thanks for doing this!
>>
>> One thing that I would really find useful are error bars in the timeline
>> view. This would help us judge whether up-and-down movements are within
>> the margin of randomness, or whether it is a real effect. I don't know
>> how annoying they are to implement though, no clue whether the plot lib
>> supports that. There should be enough information about errors in the
>> json files, as far as I remember.
>
> The standard deviation is commonly considered meaningless for benchmark
> results. All that really matters is the fastest run, everything else is
> just fog.

I disagree with this, please read the following paper:

http://buytaert.net/files/oopsla07-georges.pdf

Cheers,

Carl Friedrich


From stefan_ml at behnel.de  Fri Feb 26 11:30:11 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 26 Feb 2010 11:30:11 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002260205k2aeed582i1bfc545a6530f856@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	<4B86B588.4000601@gmx.de>
	<hm862m$cer$1@dough.gmane.org>
	<c561346d1002260205k2aeed582i1bfc545a6530f856@mail.gmail.com>
Message-ID: <hm87rn$jqc$1@dough.gmane.org>

Miquel Torres, 26.02.2010 11:05:
> You may also consider that a benchmark that varies greatly between
> runs may be a flawed benchmark.
> 
> I think it should be considered, but only on the running side, and act
> accordingly (too high a deviation: discard run, reconsider benchmark,
> reconsider environment or whatever).

Right, there might even have been a cron job running at the same time.
There are various reasons why benchmark numbers can vary.

Especially in a JIT environment, you'd normally expect the benchmark
numbers to decrease over a certain time, or to stay constantly high for a
while, then show a peak when the compiler kicks in, and then continue at a
lower level (e.g. with the Sun JVM's hotspot JIT or incremental JIT
compilers in general). I assume that the benchmarking machinery handles
this, but it's yet another reason why highly differing timings can occur
within a single run, and why it's only the best run that really matters.

You could even go one step further: ignore deviating results in the history
graph and only present them when they are still reproducible (preferably
with the same source revision) an hour later.

Stefan


From bea at changemaker.nu  Fri Feb 26 11:44:20 2010
From: bea at changemaker.nu (Bea During)
Date: Fri, 26 Feb 2010 11:44:20 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <20100225091804.GA15891@code0.codespeak.net>
References: <20091002114028.GK15455@trillke.net>	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>
Message-ID: <4B87A604.9010601@changemaker.nu>

Hi there

Armin Rigo wrote:
> Hi Miquel,
>
> May I point out a couple of comments about http://speed.pypy.org/ ?
> The first is that it looks great, indeed; thank you very much for doing
> such a site! :-)
>
>   
Yes - it looks both nice and very useful indeed!

Do we have a link from PyPys "main page"
http://codespeak.net/pypy/dist/pypy/doc/  to this
page so people can find it more easily?

Or did I miss that?

Cheers

Bea

> The comments are mostly about the graphics in /timeline: 
>
> * they should also display, or allow to display, the speed of CPython
>   for comparison;
>
> * they should have a longer maximum history than 100, to see more
>   clearly the long-term evolution.
>
>
> All in all it looks great!
>
> A bientot,
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
>   


From anto.cuni at gmail.com  Fri Feb 26 12:08:39 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 26 Feb 2010 12:08:39 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <4B87A1AE.90903@gmx.de>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	<4B86B588.4000601@gmx.de>	<hm862m$cer$1@dough.gmane.org>
	<4B87A1AE.90903@gmx.de>
Message-ID: <4B87ABB7.6050005@gmail.com>

Carl Friedrich Bolz wrote:

> I disagree with this, please read the following paper:
> 
> http://buytaert.net/files/oopsla07-georges.pdf

+1 for this paper. I was about to link it as well, but cf has been faster.
I looked at the unladen swallow runner when I was writing my thesis, and I can 
confirm that it does the "statistically right" thing as described in the paper 
to compute error bars.

ciao,
Anto

From tobami at googlemail.com  Fri Feb 26 12:30:35 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Fri, 26 Feb 2010 12:30:35 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <4B87ABB7.6050005@gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
	<4B87A1AE.90903@gmx.de> <4B87ABB7.6050005@gmail.com>
Message-ID: <c561346d1002260330x48f97884g6a0d913550bb17f8@mail.gmail.com>

The paper is right, and the unladen swallow runner does the right thing.

What I meant was: use the statistically right method (like we are
doing now!), but don't show deviation bars if the deviation is
acceptable. Check after the run whether the deviation is not
"acceptable". If it isn't, rerun later, check that nothing in the
background is affecting performance, reevaluate reproducibility of the
given benchmark, etc.

But it doesn't change the fact that speed could save the deviation
data for later use.

What do you think?
Miquel

2010/2/26 Antonio Cuni <anto.cuni at gmail.com>:
> Carl Friedrich Bolz wrote:
>
>> I disagree with this, please read the following paper:
>>
>> http://buytaert.net/files/oopsla07-georges.pdf
>
> +1 for this paper. I was about to link it as well, but cf has been faster.
> I looked at the unladen swallow runner when I was writing my thesis, and I can
> confirm that it does the "statistically right" thing as described in the paper
> to compute error bars.
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From cfbolz at gmx.de  Fri Feb 26 13:13:15 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 26 Feb 2010 13:13:15 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <c561346d1002260330x48f97884g6a0d913550bb17f8@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>	
	<4B87A1AE.90903@gmx.de> <4B87ABB7.6050005@gmail.com>
	<c561346d1002260330x48f97884g6a0d913550bb17f8@mail.gmail.com>
Message-ID: <4B87BADB.8080708@gmx.de>

On 02/26/2010 12:30 PM, Miquel Torres wrote:
> The paper is right, and the unladen swallow runner does the right thing.
>
> What I meant was: use the statistically right method (like we are
> doing now!), but don't show deviation bars if the deviation is
> acceptable. Check after the run whether the deviation is not
> "acceptable". If it isn't, rerun later, check that nothing in the
> background is affecting performance, reevaluate reproducibility of the
> given benchmark, etc.

I think an important point is that the deviations don't need to come 
from anything in the background. We don't use threads in the benchmarks 
(yet) which would obviously insert non-determinism, but even currently 
there is enough randomness in the interpreter itself. The GC can start 
at different points, the JIT could decide (late in the process) that 
something else should be compiled, there are cache-effects, etc. This 
randomness is not a bad thing, but I think we should try to at least 
evaluate it, by showing the error bars. We should do that even if the 
errors are small, because that is a good result worth mentioning.

I guess around 20 or even 10 years ago you could attribute a "correct" 
running time to a program, but nowadays there is noise on all levels of 
the system and it is not really possible to ignore that. Also, there are 
really a lot more levels too :-).

> But it doesn't change the fact that speed could save the deviation
> data for later use.

Yip.

Cheers,

Carl Friedrich

From stefan_ml at behnel.de  Fri Feb 26 13:30:25 2010
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 26 Feb 2010 13:30:25 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <4B87A1AE.90903@gmx.de>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	<4B86B588.4000601@gmx.de>	<hm862m$cer$1@dough.gmane.org>
	<4B87A1AE.90903@gmx.de>
Message-ID: <hm8et2$d9p$1@dough.gmane.org>

Carl Friedrich Bolz, 26.02.2010 11:25:
> http://buytaert.net/files/oopsla07-georges.pdf

It's sad that the paper doesn't try to understand *why* others use
different ways to benchmark. They even admit at the end that their
statistical approach is only really interesting when the differences are
small enough, not mentioning at that point that the system must be complex
enough also, such as the Sun JVM. However, if the differences are small and
the benchmarked system is complex, it's best to question the benchmark in
the first place, rather than the statistics that lead to its results.

Anyway, I agree that, given the complexity of at least some of the
benchmarks in the suite, and given the requirement to do continuous
benchmarking to find both small and large differences, taking statistically
relevant run lines makes sense.

Stefan


From cfbolz at gmx.de  Fri Feb 26 13:43:41 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 26 Feb 2010 13:43:41 +0100
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <hm8et2$d9p$1@dough.gmane.org>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>	<4B86B588.4000601@gmx.de>	<hm862m$cer$1@dough.gmane.org>	<4B87A1AE.90903@gmx.de>
	<hm8et2$d9p$1@dough.gmane.org>
Message-ID: <4B87C1FD.3040806@gmx.de>

On 02/26/2010 01:30 PM, Stefan Behnel wrote:
> Carl Friedrich Bolz, 26.02.2010 11:25:
>> http://buytaert.net/files/oopsla07-georges.pdf
>
> It's sad that the paper doesn't try to understand *why* others use
> different ways to benchmark.

I guess those others should write their own papers (or blog posts or 
whatever) :-). If you know any well-written ones, I would be very 
interested.

> They even admit at the end that their
> statistical approach is only really interesting when the differences are
> small enough, not mentioning at that point that the system must be complex
> enough also, such as the Sun JVM. However, if the differences are small and
> the benchmarked system is complex, it's best to question the benchmark in
> the first place, rather than the statistics that lead to its results.
[...]

In my opinion there are probably not really many non-complex systems 
around nowadays, at least if we are talking about typical "desktop" 
systems. There is also a lot of noise on the CPU level, with caches, out 
of order execution, not even talking about the OS. And while PyPy is not 
quite as complex as a JVM, it is certainly moving in this direction. So 
even if your benchmark itself is a simple piece of Python code, the 
whole system that you invoke is still complex.

Cheers,

Carl Friedrich

From fijall at gmail.com  Fri Feb 26 16:05:19 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 26 Feb 2010 10:05:19 -0500
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <4B87A604.9010601@changemaker.nu>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com> 
	<20100225091804.GA15891@code0.codespeak.net>
	<4B87A604.9010601@changemaker.nu>
Message-ID: <693bc9ab1002260705o3bc25dd4lc92c3ed5d0e0e0f8@mail.gmail.com>

On Fri, Feb 26, 2010 at 5:44 AM, Bea During <bea at changemaker.nu>
wrote:> Hi there
>
> Armin Rigo wrote:
>> Hi Miquel,
>>
>> May I point out a couple of comments about http://speed.pypy.org/ ?
>> The first is that it looks great, indeed; thank you very much for doing
>> such a site! :-)
>>
>>
> Yes - it looks both nice and very useful indeed!
>
> Do we have a link from PyPys "main page"
> http://codespeak.net/pypy/dist/pypy/doc/ ?to this
> page so people can find it more easily?
>
> Or did I miss that?
>

Our main website is a mess right now. I would like to have it
redesigned, so we have actually interesting links from there, and
obviously the one to graphs (we have two of those now) would be great.
Note that for now speed.pypy.org is not automatically updated - it
requires adjustments to buildbot run.

Cheers,
fijal

From con at samuelreis.de  Fri Feb 26 16:21:16 2010
From: con at samuelreis.de (Samuel Reis)
Date: Fri, 26 Feb 2010 16:21:16 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <693bc9ab1002260705o3bc25dd4lc92c3ed5d0e0e0f8@mail.gmail.com>
References: <20091002114028.GK15455@trillke.net>	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>	<4B87A604.9010601@changemaker.nu>
	<693bc9ab1002260705o3bc25dd4lc92c3ed5d0e0e0f8@mail.gmail.com>
Message-ID: <4B87E6EC.6010302@samuelreis.de>

Am 26.02.10 16:05, schrieb Maciej Fijalkowski:
> On Fri, Feb 26, 2010 at 5:44 AM, Bea During<bea at changemaker.nu>
> wrote:>  Hi there
>    
>> Armin Rigo wrote:
>>      
>>> Hi Miquel,
>>>
>>> May I point out a couple of comments about http://speed.pypy.org/ ?
>>> The first is that it looks great, indeed; thank you very much for doing
>>> such a site! :-)
>>>
>>>
>>>        
>> Yes - it looks both nice and very useful indeed!
>>
>> Do we have a link from PyPys "main page"
>> http://codespeak.net/pypy/dist/pypy/doc/  to this
>> page so people can find it more easily?
>>
>> Or did I miss that?
>>
>>      
> Our main website is a mess right now. I would like to have it
> redesigned, so we have actually interesting links from there, and
> obviously the one to graphs (we have two of those now) would be great.
> Note that for now speed.pypy.org is not automatically updated - it
> requires adjustments to buildbot run.
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
If there's the need I could to do a redesign of the layout and (if 
needed) structure.
So you guys could focus on pypy development and just answer a few 
questions I come up with while working.

Greetings -- Samuel

From fijall at gmail.com  Fri Feb 26 16:58:46 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 26 Feb 2010 10:58:46 -0500
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <4B87E6EC.6010302@samuelreis.de>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com> 
	<20100225091804.GA15891@code0.codespeak.net>
	<4B87A604.9010601@changemaker.nu> 
	<693bc9ab1002260705o3bc25dd4lc92c3ed5d0e0e0f8@mail.gmail.com> 
	<4B87E6EC.6010302@samuelreis.de>
Message-ID: <693bc9ab1002260758y558882dapb98c0e37f8048ff7@mail.gmail.com>

On Fri, Feb 26, 2010 at 10:21 AM, Samuel Reis <con at samuelreis.de> wrote:
> Am 26.02.10 16:05, schrieb Maciej Fijalkowski:
>> On Fri, Feb 26, 2010 at 5:44 AM, Bea During<bea at changemaker.nu>
>> wrote:> ?Hi there
>>
>>> Armin Rigo wrote:
>>>
>>>> Hi Miquel,
>>>>
>>>> May I point out a couple of comments about http://speed.pypy.org/ ?
>>>> The first is that it looks great, indeed; thank you very much for doing
>>>> such a site! :-)
>>>>
>>>>
>>>>
>>> Yes - it looks both nice and very useful indeed!
>>>
>>> Do we have a link from PyPys "main page"
>>> http://codespeak.net/pypy/dist/pypy/doc/ ?to this
>>> page so people can find it more easily?
>>>
>>> Or did I miss that?
>>>
>>>
>> Our main website is a mess right now. I would like to have it
>> redesigned, so we have actually interesting links from there, and
>> obviously the one to graphs (we have two of those now) would be great.
>> Note that for now speed.pypy.org is not automatically updated - it
>> requires adjustments to buildbot run.
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
> If there's the need I could to do a redesign of the layout and (if
> needed) structure.
> So you guys could focus on pypy development and just answer a few
> questions I come up with while working.

Great! I have tons of ideas and no actual time to do all this (and no
skills as well). How about meeting on IRC at some point? It's easier
to discuss stuff that on mail
I'm fijal on #pypy at freenode.

>
> Greetings -- Samuel
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From tobami at googlemail.com  Fri Feb 26 19:24:51 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Fri, 26 Feb 2010 19:24:51 +0100
Subject: [pypy-dev] new speed.pypy.org site?
In-Reply-To: <F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
References: <20091002114028.GK15455@trillke.net>
	<c561346d0910021005m1cd4e8ddpe59738a41c0c3d47@mail.gmail.com>
	<20100225091804.GA15891@code0.codespeak.net>
	<F13C91A2-5030-427D-9047-E982D4A97D16@gmail.com>
Message-ID: <c561346d1002261024s7507dd6ag947b4336fad99ac5@mail.gmail.com>

I have commited a bunch of changes:

- Added cpython baseline to timeline
- Changed timeline defaults to both interpreters selected (plus baseline)
- Performance improvements in Timeline's ajax: due to above changes, I
worried that for a lot of interpreters selected the queries could be
too slow. Even though it is known that Premature optimization is the
root of all evil (tm), I had a look and optimized django database
calls a little bit (or more precisely made my implementation suck
less). So instead of nearly 200 ms per interpreter, it now takes less
than 100ms (measured locally, add >100ms for network latency). Plot
rerendering should feel faster now. I find it interesting how quick
django is. Sometimes you may even forget that the data is being pulled
from the server each time a different plot is selected.
- Timeline: Added labels for the axes
- pypy-c-jit appears now listed first
- Added tooltips for Host info and for some benchmarks. Copied the
descriptions for the benchmarks listed at the unladen site.

Cheers,
Miquel


2010/2/25 Leonardo Santagada <santagada at gmail.com>:
> I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
>
> - When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
>
> - Overview window comments:
> ? ? ? ?- rename the column "result" to "time" (or timing?);
> ? ? ? ?- create a new column called "previous time" with the last time of the previous revision.
> ? ? ? ?- rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%);
> ? ? ? ?- with the last two ones it will make understanding the improvement and trends a lot easier;
> ? ? ? ?- trend should have the number of revisions on the label (and the values reversed like the "improvement" column);
> ? ? ? ?- rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup";
> ? ? ? ?- the graph should say what it is graphing;
> ? ? ? ?- maybe invert the order of results.
>
> - Timeline window comments:
> ? ? ? ?- showing all benchmarks in the same page would be good
> ? ? ? ?- Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly;
> ? ? ? ?- see all benchmarks together, so you don't have to hunt around (maybe have a detailed view);
> ? ? ? ?- describe all axis;
> ? ? ? ?- as armin said have a selected by default cpython comparison so people can have a baseline.
>
>
> I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
>
>
> On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
>
>> Hi Miquel,
>>
>> May I point out a couple of comments about http://speed.pypy.org/ ?
>> The first is that it looks great, indeed; thank you very much for doing
>> such a site! :-)
>>
>> The comments are mostly about the graphics in /timeline:
>>
>> * they should also display, or allow to display, the speed of CPython
>> ?for comparison;
>>
>> * they should have a longer maximum history than 100, to see more
>> ?clearly the long-term evolution.
>>
>>
>> All in all it looks great!
>>
>> A bientot,
>> Armin.
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
>

From renesd at gmail.com  Mon Mar  1 11:15:22 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Mon, 1 Mar 2010 10:15:22 +0000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <hm862m$cer$1@dough.gmane.org>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
Message-ID: <64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>

On Fri, Feb 26, 2010 at 9:59 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Carl Friedrich Bolz, 25.02.2010 18:38:
>> On 02/25/2010 04:10 PM, Miquel Torres wrote:
>>> As some of you already know, there is a new performance site online:
>>> speed.pypy.org.
>> [...]
>>
>> I'm quite impressed, this is very cool work and a good improvement over
>> the current plots. Thanks for doing this!
>>
>> One thing that I would really find useful are error bars in the timeline
>> view. This would help us judge whether up-and-down movements are within
>> the margin of randomness, or whether it is a real effect. I don't know
>> how annoying they are to implement though, no clue whether the plot lib
>> supports that. There should be enough information about errors in the
>> json files, as far as I remember.
>
> The standard deviation is commonly considered meaningless for benchmark
> results. All that really matters is the fastest run, everything else is
> just fog.
>
> Read the docs on timeit, for example.
>
> Stefan
>


hi,

For some sets of problems, the first run is very important.  For
example, where you only want to process the particular data once.

For others the Worst performance is the most important.  For example
'real time' like programs which require the runtime only takes at
maximum N where N is measured.

Running something 10 times and removing outliers fails for these two
important cases.


cya.

From renesd at gmail.com  Mon Mar  1 11:18:07 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Mon, 1 Mar 2010 10:18:07 +0000
Subject: [pypy-dev] speed.pypy.org launched
In-Reply-To: <64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>
References: <c561346d1002250710h5df95a1ao8eaaf314ef34d963@mail.gmail.com>
	<4B86B588.4000601@gmx.de> <hm862m$cer$1@dough.gmane.org>
	<64ddb72c1003010215t50a3b620n1f541c851894684b@mail.gmail.com>
Message-ID: <64ddb72c1003010218l73a9779j17709e7b4275be23@mail.gmail.com>

On Mon, Mar 1, 2010 at 10:15 AM, Ren? Dudfield <renesd at gmail.com> wrote:
>
> For others the Worst performance is the most important. ?For example
> 'real time' like programs which require the runtime only takes at
> maximum N where N is measured.

um, this made no sense at all... oops!  Sorry, let me try again...


For others the Worst performance is the most important.  For example
'real time' like programs which require the runtime only takes at
maximum N.  Where N is an allocated time budget for that code.

From fijall at gmail.com  Tue Mar  2 05:50:36 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 1 Mar 2010 21:50:36 -0700
Subject: [pypy-dev] Release and tags
Message-ID: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>

Hello.

Can we copy trunk -> dist for the release, so we don't block any new
development because of the release? There was the point of dist, even
if we forgot it completely.

Cheers,
fijal

From renesd at gmail.com  Tue Mar  2 16:51:38 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Tue, 2 Mar 2010 15:51:38 +0000
Subject: [pypy-dev] gsoc for pypy...
Message-ID: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>

hi,

gsoc is on again... for your information the python mailing list for it is
here:
    http://mail.python.org/mailman/listinfo/soc2010-mentors

It would be good to have a wiki page(or blog post? ... err a text file
available via http) of project ideas for pypy... if there are any pypy
people mentoring that is.  With the pygame project last year, we found
having a project page for suggestions of acceptable projects helped students
pick things they were interested in, and things the project would be happy
with.



cu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100302/c14e67a2/attachment.htm 

From renesd at gmail.com  Tue Mar  2 19:06:25 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Tue, 2 Mar 2010 18:06:25 +0000
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
Message-ID: <64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>

hello again.

To start the gsoc ideas list off...

- port jit to 64 bit (maybe too short for 3 month project, if so start on
ARM after amd64?).
- speed up ctypes wrapper.
- GIL removal work.
- python2.6/2.7/py3k features.
- ctypes bindings for database adaptors (would be good for ironpython,
jython, and even cpython too).
- faster ctypes module.  (by using jit?)
- cpython bridge module.  Either load pypy as cpython extension or the
otherway around.  To allow gradual porting.
- ironclad port to pypy.  To allow loading cpython C extensions.  eg,
ironpython can run numpy with this.
- revive javascript/flex backend.
- improvements to java/.net backend.
- AOT compilation research - to allow compiling to iphone for example.

I guess this list could also be put on a page looking for sponsors?  To give
sponsors something concrete they can pay a contract for.  eg, 'approximately
2-3 months work(24,000 euros) to get X benefit, as well as a logo+link on
our sponsors page... etc.'


Anyway... those are some of the areas I can think of from past discussions.
I'm sure pypy project members have a better idea of what would be good for
gsoc students to work on?


cu,




On Tue, Mar 2, 2010 at 3:51 PM, Ren? Dudfield <renesd at gmail.com> wrote:

> hi,
>
> gsoc is on again... for your information the python mailing list for it is
> here:
>     http://mail.python.org/mailman/listinfo/soc2010-mentors
>
> It would be good to have a wiki page(or blog post? ... err a text file
> available via http) of project ideas for pypy... if there are any pypy
> people mentoring that is.  With the pygame project last year, we found
> having a project page for suggestions of acceptable projects helped students
> pick things they were interested in, and things the project would be happy
> with.
>
>
>
> cu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100302/89e0d4bf/attachment-0001.htm 

From fijall at gmail.com  Tue Mar  2 19:38:33 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 11:38:33 -0700
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com> 
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
Message-ID: <693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>

Hey.

Thanks for posting that! I think it makes sense to discuss a bit
whether stuff makes or does not make sense.

On Tue, Mar 2, 2010 at 11:06 AM, Ren? Dudfield <renesd at gmail.com> wrote:
> hello again.
>
> To start the gsoc ideas list off...
>
> - port jit to 64 bit (maybe too short for 3 month project, if so start on
> ARM after amd64?).

Not sure if it's too short. It's too short for core dev 3 month
project, but for student it's ideal.

> - speed up ctypes wrapper.

A bit hard, but also JIT-cooperation.

> - GIL removal work.

Hard/researchy

> - python2.6/2.7/py3k features.
> - ctypes bindings for database adaptors (would be good for ironpython,
> jython, and even cpython too).

that's not strictly pypy-related. I would like such projects to be
under PSF umbrella. Also there is a question of maintainability.

> - cpython bridge module.? Either load pypy as cpython extension or the
> otherway around.? To allow gradual porting.
> - ironclad port to pypy.? To allow loading cpython C extensions.? eg,
> ironpython can run numpy with this.

This is probably interesting, might also be a bit hard.

> - revive javascript/flex backend.

please no :)

> - improvements to java/.net backend.

Honestly, we had problems with maintability of new backends. Also .net
is sort of done (except .net bindings).

> - AOT compilation research - to allow compiling to iphone for example.

that's research, possibly interesting though.

>
> I guess this list could also be put on a page looking for sponsors?? To give
> sponsors something concrete they can pay a contract for.? eg, 'approximately
> 2-3 months work(24,000 euros) to get X benefit, as well as a logo+link on
> our sponsors page... etc.'

Sure. Although note that so far we did not get any feedback on
sponsoring blog post.

>
>
> Anyway... those are some of the areas I can think of from past discussions.
> I'm sure pypy project members have a better idea of what would be good for
> gsoc students to work on?

How about investigating (again) llvm jit backend?

Thanks for starting the discussion!

Cheers,
fijal

From fijall at gmail.com  Tue Mar  2 20:01:23 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 12:01:23 -0700
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com> 
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com> 
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com> 
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
Message-ID: <693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>

On Tue, Mar 2, 2010 at 11:56 AM, Benoit Boissinot
<bboissin+pypy at gmail.com> wrote:
> On Tue, Mar 2, 2010 at 7:38 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Hey.
>>
>>> Anyway... those are some of the areas I can think of from past discussions.
>>> I'm sure pypy project members have a better idea of what would be good for
>>> gsoc students to work on?
>>
>> How about investigating (again) llvm jit backend?
>
> Is there a summary of what the problems were? I guess the lack of
> maturity of the JIT engine was one (as experienced by
> unladed-swallow).
>
> cheers,
>
> Benoit
>

Not written. Bugs and immaturity, yes. The precise issue we run into
was tail calls not working, but this one is supposed to be fixed by
now. Having US team on LLVM JIT team is certainly a big advantage.

I suppose next step is to try to push it and see what's next.

Cheers,
fijal

From bboissin+pypy at gmail.com  Tue Mar  2 19:56:59 2010
From: bboissin+pypy at gmail.com (Benoit Boissinot)
Date: Tue, 2 Mar 2010 19:56:59 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
Message-ID: <40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>

On Tue, Mar 2, 2010 at 7:38 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hey.
>
>> Anyway... those are some of the areas I can think of from past discussions.
>> I'm sure pypy project members have a better idea of what would be good for
>> gsoc students to work on?
>
> How about investigating (again) llvm jit backend?

Is there a summary of what the problems were? I guess the lack of
maturity of the JIT engine was one (as experienced by
unladed-swallow).

cheers,

Benoit

From arigo at tunes.org  Tue Mar  2 20:22:51 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:22:51 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
Message-ID: <20100302192251.GA9739@code0.codespeak.net>

Hi,

On Tue, Mar 02, 2010 at 12:01:23PM -0700, Maciej Fijalkowski wrote:
> >> How about investigating (again) llvm jit backend?
> (...)
> I suppose next step is to try to push it and see what's next.

Yes, but I bet from experience that we're going to run after a few days'
work into the next issue.  We've already done that 3 or 4 times with
llvm, so now I'm honestly a bit fed up.  For comparison, gcc gives code
that is a bit slower but at least it is really a solid bug-free project.


A bientot,

Armin.

From arigo at tunes.org  Tue Mar  2 20:35:48 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:35:48 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <20100302192251.GA9739@code0.codespeak.net>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
Message-ID: <20100302193547.GA12258@code0.codespeak.net>

Re-hi,

On Tue, Mar 02, 2010 at 08:22:51PM +0100, Armin Rigo wrote:
> Yes, but I bet from experience that we're going to run after a few days'
> work into the next issue.  We've already done that 3 or 4 times with
> llvm, so now I'm honestly a bit fed up.  For comparison, gcc gives code
> that is a bit slower but at least it is really a solid bug-free project.

Let me make this statement more precise: I mean for the non-JIT backends
of PyPy, using the llvm backend of PyPy gaves results that were
typically a little bit faster than using the C backend, when compiling
with llvm-gcc.  However, with the various generations of our JIT, using
a JIT llvm backend always ran into llvm bugs.


A bientot,

Armin.

From bboissin+pypy at gmail.com  Tue Mar  2 20:40:49 2010
From: bboissin+pypy at gmail.com (Benoit Boissinot)
Date: Tue, 2 Mar 2010 20:40:49 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <20100302193547.GA12258@code0.codespeak.net>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
	<20100302193547.GA12258@code0.codespeak.net>
Message-ID: <40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>

On Tue, Mar 2, 2010 at 8:35 PM, Armin Rigo <arigo at tunes.org> wrote:
> Re-hi,
>
> On Tue, Mar 02, 2010 at 08:22:51PM +0100, Armin Rigo wrote:
>> Yes, but I bet from experience that we're going to run after a few days'
>> work into the next issue. ?We've already done that 3 or 4 times with
>> llvm, so now I'm honestly a bit fed up. ?For comparison, gcc gives code
>> that is a bit slower but at least it is really a solid bug-free project.
>
> Let me make this statement more precise: I mean for the non-JIT backends
> of PyPy, using the llvm backend of PyPy gaves results that were
> typically a little bit faster than using the C backend, when compiling
> with llvm-gcc. ?However, with the various generations of our JIT, using
> a JIT llvm backend always ran into llvm bugs.

On the other hand, it looks like the US guys made it production
quality, isn't it?
(At least it seems they generate correct code)

Benoit

From arigo at tunes.org  Tue Mar  2 20:51:35 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 20:51:35 +0100
Subject: [pypy-dev] gsoc for pypy...
In-Reply-To: <40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>
References: <64ddb72c1003020751r2bc5d26hfd0f3f0eff80c47a@mail.gmail.com>
	<64ddb72c1003021006r6dc70afdj23e804490909ea@mail.gmail.com>
	<693bc9ab1003021038h6ac15c0dqcbe25ca07413026f@mail.gmail.com>
	<40f323d01003021056l7badac4ft99f57c971a8507e6@mail.gmail.com>
	<693bc9ab1003021101s5ea358f8s3705fb40bf489271@mail.gmail.com>
	<20100302192251.GA9739@code0.codespeak.net>
	<20100302193547.GA12258@code0.codespeak.net>
	<40f323d01003021140y5ee5f5a6i3ca65a9f4ddf2b4b@mail.gmail.com>
Message-ID: <20100302195134.GA13608@code0.codespeak.net>

Hi,

On Tue, Mar 02, 2010 at 08:40:49PM +0100, Benoit Boissinot wrote:
> On the other hand, it looks like the US guys made it production
> quality, isn't it?
> (At least it seems they generate correct code)

Yes.  Well, I suppose it should be ok if a student wants to work on it,
though I remain a bit skeptical (maybe wrongly).


A bientot,

Armin.

From benjamin at python.org  Tue Mar  2 22:46:24 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 2 Mar 2010 15:46:24 -0600
Subject: [pypy-dev] Release and tags
In-Reply-To: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
Message-ID: <1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>

2010/3/1 Maciej Fijalkowski <fijall at gmail.com>:
> Hello.
>
> Can we copy trunk -> dist for the release, so we don't block any new
> development because of the release? There was the point of dist, even
> if we forgot it completely.

Maybe we should copy it to branch/release-1.2maint or some such name?
We could back port critical bug fixes.



-- 
Regards,
Benjamin

From fijall at gmail.com  Tue Mar  2 22:51:33 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 14:51:33 -0700
Subject: [pypy-dev] Release and tags
In-Reply-To: <1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com> 
	<1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
Message-ID: <693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>

This is already done. However, I wonder what's the purpose of dist.

On Tue, Mar 2, 2010 at 2:46 PM, Benjamin Peterson <benjamin at python.org> wrote:
> 2010/3/1 Maciej Fijalkowski <fijall at gmail.com>:
>> Hello.
>>
>> Can we copy trunk -> dist for the release, so we don't block any new
>> development because of the release? There was the point of dist, even
>> if we forgot it completely.
>
> Maybe we should copy it to branch/release-1.2maint or some such name?
> We could back port critical bug fixes.
>
>
>
> --
> Regards,
> Benjamin
>

From arigo at tunes.org  Tue Mar  2 22:56:33 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 2 Mar 2010 22:56:33 +0100
Subject: [pypy-dev] Release and tags
In-Reply-To: <693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>
References: <693bc9ab1003012050q39f4a2a8odd009c58b9375c67@mail.gmail.com>
	<1afaf6161003021346x5c4af40dxf411226f816f096c@mail.gmail.com>
	<693bc9ab1003021351k55248aa9mdf29cec3b92690db@mail.gmail.com>
Message-ID: <20100302215633.GA23901@code0.codespeak.net>

Hi Maciej,

On Tue, Mar 02, 2010 at 02:51:33PM -0700, Maciej Fijalkowski wrote:
> This is already done. However, I wonder what's the purpose of dist.

I'll copy from release/1.2.x into both release/1.2.0 and dist when
we really do the release.  That's supposedly the purpose of dist,
I guess.


A bientot,

Armin.

From fijall at gmail.com  Wed Mar  3 06:54:43 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 2 Mar 2010 22:54:43 -0700
Subject: [pypy-dev] buildbot question
Message-ID: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>

any particular reason why pypy-c tests are run twice each night? once
for app tests and once for pypy-c jit tests?

From arigo at tunes.org  Wed Mar  3 09:40:36 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Mar 2010 09:40:36 +0100
Subject: [pypy-dev] buildbot question
In-Reply-To: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>
References: <693bc9ab1003022154g5f181018y1b4600f33271b0a8@mail.gmail.com>
Message-ID: <20100303084036.GA8881@code0.codespeak.net>

Hi Maciek,

On Tue, Mar 02, 2010 at 10:54:43PM -0700, Maciej Fijalkowski wrote:
> any particular reason why pypy-c tests are run twice each night? once
> for app tests and once for pypy-c jit tests?

Once with a pypy-c and once with a pypy-c-jit, obviously.  It used to
make sense, and probably still does a bit.


Armin

From arigo at tunes.org  Wed Mar  3 10:14:35 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 3 Mar 2010 10:14:35 +0100
Subject: [pypy-dev] =?iso-8859-1?q?cobra_server_at_the_Uni_D=FCsseldorf?=
In-Reply-To: <4B73F4D7.1040508@gmx.de>
References: <4B73F4D7.1040508@gmx.de>
Message-ID: <20100303091434.GA11020@code0.codespeak.net>

Hi Carl Friedrich,

On Thu, Feb 11, 2010 at 01:15:19PM +0100, Carl Friedrich Bolz wrote:
> kept and move them elsewhere? We plan to shut Cobra down on the 19th of 
> February, if somebody cannot make it till then, please send me a mail.

Cobra is still around, but wyvern's hard drive seems to have died
yesterday.  At least it ended up disconnecting itself from the buildbot
metaserver.  You (cfbolz) can always try restarting the machine and see
if it helps, but I doubt it a bit.  We need a solution for that issue; I
suppose that waiting for the upcoming server at OpenEnd would work.  In
the meantime I listed cobra as an alternate server on which to run these
tests (it was listed on some builds, but not all).


A bientot,

Armin.

From cfbolz at gmx.de  Wed Mar  3 10:47:10 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 03 Mar 2010 10:47:10 +0100
Subject: [pypy-dev] =?iso-8859-1?q?cobra_server_at_the_Uni_D=FCsseldorf?=
In-Reply-To: <20100303091434.GA11020@code0.codespeak.net>
References: <4B73F4D7.1040508@gmx.de>
	<20100303091434.GA11020@code0.codespeak.net>
Message-ID: <4B8E301E.30303@gmx.de>

Hi Armin,

On 03/03/2010 10:14 AM, Armin Rigo wrote:
> On Thu, Feb 11, 2010 at 01:15:19PM +0100, Carl Friedrich Bolz wrote:
>> kept and move them elsewhere? We plan to shut Cobra down on the 19th of
>> February, if somebody cannot make it till then, please send me a mail.
>
> Cobra is still around, but wyvern's hard drive seems to have died
> yesterday.  At least it ended up disconnecting itself from the buildbot
> metaserver.  You (cfbolz) can always try restarting the machine and see
> if it helps, but I doubt it a bit.  We need a solution for that issue; I
> suppose that waiting for the upcoming server at OpenEnd would work.  In
> the meantime I listed cobra as an alternate server on which to run these
> tests (it was listed on some builds, but not all).

I asked Jens to wait a bit with the Cobra migration, so that we can do 
the release and have at least one machines that still runs our tests 
nightly. We should try to get a new hard drive for wyvern and reinstall 
it, but I guess only after the release.

Cheers,

Carl Friedrich

From arigo at tunes.org  Thu Mar  4 13:18:13 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 4 Mar 2010 13:18:13 +0100
Subject: [pypy-dev] next Leysin sprint?
Message-ID: <20100304121813.GA896@code0.codespeak.net>

Hi all,

This is a quick poll to know if there would be people interested in
attending the next sprint in Switzerland, one week after Easter, roughly
10-17th April 2010.

I guess the conditions for it to happen are that I am not the only "core
developer" (cfbolz?), and there are at least a couple of extra people
with whom to sprint.  The sprint topics are completely open.  The basic
idea is to do the sprint in Leysin, but I am also open to the idea of
making a 2nd sprint in Bern with a topic like "let's apply the JIT to
the Smalltalk interpreter" :-)


A bientot,

Armin.

From fuzzyman at gmail.com  Thu Mar  4 14:42:25 2010
From: fuzzyman at gmail.com (Michael Foord)
Date: Thu, 4 Mar 2010 13:42:25 +0000
Subject: [pypy-dev] next Leysin sprint?
In-Reply-To: <20100304121813.GA896@code0.codespeak.net>
References: <20100304121813.GA896@code0.codespeak.net>
Message-ID: <6f4025011003040542u6fdbe0fet1554bd78bba39802@mail.gmail.com>

On 4 March 2010 12:18, Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> This is a quick poll to know if there would be people interested in
> attending the next sprint in Switzerland, one week after Easter, roughly
> 10-17th April 2010.
>


I can't commit to it, but am definitely interested. Especially if Carl makes
it. :-)

Michael


>
> I guess the conditions for it to happen are that I am not the only "core
> developer" (cfbolz?), and there are at least a couple of extra people
> with whom to sprint.  The sprint topics are completely open.  The basic
> idea is to do the sprint in Leysin, but I am also open to the idea of
> making a 2nd sprint in Bern with a topic like "let's apply the JIT to
> the Smalltalk interpreter" :-)
>
>
> A bientot,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
http://www.ironpythoninaction.com/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100304/f4c2494b/attachment.htm 

From arigo at tunes.org  Fri Mar  5 11:47:41 2010
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 5 Mar 2010 11:47:41 +0100
Subject: [pypy-dev] Getting anywhere?
Message-ID: <20100305104741.GA17903@code0.codespeak.net>

Hi Maciek,

It's now been two days since you last checked in some text in
svn/user/fijal/website/.  Can you please confirm that you are still
working on it?  I have nothing against general hacking, but just to make
it clear, I plan to not merge most of what we do now inside
'release/1.2.x', so that's not release work.


A bientot,

Armin.

From arigo at tunes.org  Sat Mar  6 22:27:45 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 6 Mar 2010 22:27:45 +0100
Subject: [pypy-dev] speed.pypy.org update
Message-ID: <20100306212745.GA29817@code0.codespeak.net>

Hi all,

Fijal may comment on this more, but note that he tweaked the number of
runs of the benchmarks displayed by speed.pypy.org, which explains the
jumps we see in some of them.  The positive-for-us jumps, I should add
:-)

I think that we should remove the history up to today, as it makes
little sense to compare...


A bientot,

Armin.

From fijall at gmail.com  Sun Mar  7 05:09:35 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 6 Mar 2010 21:09:35 -0700
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <20100306212745.GA29817@code0.codespeak.net>
References: <20100306212745.GA29817@code0.codespeak.net>
Message-ID: <693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>

Hey.

Yes, sorry about not saying that upfront. I've removed --fast from
run, so our numbers looks better (because warmup is more spread over).
It makes sense to remove history IMO.

However, I plan to work a bit more on that, to completely remove
warmup time from some benchmark and to include warmup for each run for
others (like richards, html5lib), so that would be yet another removal
of history.

Right now the "average" is not that meaningful (and stddev even less,
although it's not displayed on speed), since it averages warmup across
all the run. I would like to have warmup either completely included
for each run (for benchmarks that it makes sense) or completely
separated and reported as some other variable.

Cheers,
fijal

On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Fijal may comment on this more, but note that he tweaked the number of
> runs of the benchmarks displayed by speed.pypy.org, which explains the
> jumps we see in some of them. ?The positive-for-us jumps, I should add
> :-)
>
> I think that we should remove the history up to today, as it makes
> little sense to compare...
>
>
> A bientot,
>
> Armin.
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From tobami at googlemail.com  Mon Mar  8 11:29:18 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Mon, 8 Mar 2010 11:29:18 +0100
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>
References: <20100306212745.GA29817@code0.codespeak.net>
	<693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com>
Message-ID: <c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>

Yeah, history is now not very meaningful.

While we are at it, I think we should complete the job and define a
standard set of benchmarks together with the python.org and unladen
guys. So that we can keep the data this time :-)

How were the chats about that developing, fijal?



2010/3/7 Maciej Fijalkowski <fijall at gmail.com>:
> Hey.
>
> Yes, sorry about not saying that upfront. I've removed --fast from
> run, so our numbers looks better (because warmup is more spread over).
> It makes sense to remove history IMO.
>
> However, I plan to work a bit more on that, to completely remove
> warmup time from some benchmark and to include warmup for each run for
> others (like richards, html5lib), so that would be yet another removal
> of history.
>
> Right now the "average" is not that meaningful (and stddev even less,
> although it's not displayed on speed), since it averages warmup across
> all the run. I would like to have warmup either completely included
> for each run (for benchmarks that it makes sense) or completely
> separated and reported as some other variable.
>
> Cheers,
> fijal
>
> On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
>> Hi all,
>>
>> Fijal may comment on this more, but note that he tweaked the number of
>> runs of the benchmarks displayed by speed.pypy.org, which explains the
>> jumps we see in some of them. ?The positive-for-us jumps, I should add
>> :-)
>>
>> I think that we should remove the history up to today, as it makes
>> little sense to compare...
>>
>>
>> A bientot,
>>
>> Armin.
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

From fijall at gmail.com  Mon Mar  8 18:10:16 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 8 Mar 2010 10:10:16 -0700
Subject: [pypy-dev] speed.pypy.org update
In-Reply-To: <c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>
References: <20100306212745.GA29817@code0.codespeak.net>
	<693bc9ab1003062009p6d682ddfg68fd4a94d6b97b9f@mail.gmail.com> 
	<c561346d1003080229r44647a49vbd1eabc9f20d0d0f@mail.gmail.com>
Message-ID: <693bc9ab1003080910u5ab34afeq87d4f8871606fc00@mail.gmail.com>

On Mon, Mar 8, 2010 at 3:29 AM, Miquel Torres <tobami at googlemail.com> wrote:
> Yeah, history is now not very meaningful.
>
> While we are at it, I think we should complete the job and define a
> standard set of benchmarks together with the python.org and unladen
> guys. So that we can keep the data this time :-)
>
> How were the chats about that developing, fijal?
>

Progressing, but slowly. We all have slightly different goals I
believe. For example I think pypy is aiming also at people who are
doing stuff in C and would like to do python instead. US is aiming
exclusively at python programmers. (I think)

Cheers,
fijal

>
>
> 2010/3/7 Maciej Fijalkowski <fijall at gmail.com>:
>> Hey.
>>
>> Yes, sorry about not saying that upfront. I've removed --fast from
>> run, so our numbers looks better (because warmup is more spread over).
>> It makes sense to remove history IMO.
>>
>> However, I plan to work a bit more on that, to completely remove
>> warmup time from some benchmark and to include warmup for each run for
>> others (like richards, html5lib), so that would be yet another removal
>> of history.
>>
>> Right now the "average" is not that meaningful (and stddev even less,
>> although it's not displayed on speed), since it averages warmup across
>> all the run. I would like to have warmup either completely included
>> for each run (for benchmarks that it makes sense) or completely
>> separated and reported as some other variable.
>>
>> Cheers,
>> fijal
>>
>> On Sat, Mar 6, 2010 at 2:27 PM, Armin Rigo <arigo at tunes.org> wrote:
>>> Hi all,
>>>
>>> Fijal may comment on this more, but note that he tweaked the number of
>>> runs of the benchmarks displayed by speed.pypy.org, which explains the
>>> jumps we see in some of them. ?The positive-for-us jumps, I should add
>>> :-)
>>>
>>> I think that we should remove the history up to today, as it makes
>>> little sense to compare...
>>>
>>>
>>> A bientot,
>>>
>>> Armin.
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>

From pedronis at openend.se  Wed Mar 10 20:38:56 2010
From: pedronis at openend.se (Samuele Pedroni)
Date: Wed, 10 Mar 2010 20:38:56 +0100
Subject: [pypy-dev] interesting thread on Lambda The Ultimate on tracing
	trade-offs
Message-ID: <4B97F550.8000707@openend.se>


http://lambda-the-ultimate.org/node/3851

in case people missed it,

Samuele

From tobami at googlemail.com  Wed Mar 10 21:14:36 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Wed, 10 Mar 2010 21:14:36 +0100
Subject: [pypy-dev] speed.pypy.org quick update
Message-ID: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>

Hi!

I wanted to explain a couple of things about the speed website:

- New feature: the Timeline view now defaults to a plot grid, showing
all benchmarks at the same time. It was a feature request made more
than once, so depending on personal tastes, you can bookmark either
/overview/ or /timeline/. Thanks go to nsf for helping with the
implementation.
- The code has now moved to github as Codespeed, a benchmark
visualization framework (http://github.com/tobami/codespeed)
- I have updated speed.pypy.org with version 0.3. Much of the work has
been under the hood to make it feasible for other projects to use
codespeed as a framework.

For those interested in further development you can go to the releases
wiki (still a work in progress):
http://wiki.github.com/tobami/codespeed/releases

Next in the line are some DB changes to be able to save standard
deviation data and the like. Long term goals besides world domination
are integration with buildbot and similarly unrealistic things.
Feedback is always welcome.

Cheers,
Miquel

From bokr at oz.net  Thu Mar 11 00:45:50 2010
From: bokr at oz.net (Bengt Richter)
Date: Wed, 10 Mar 2010 15:45:50 -0800
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
Message-ID: <4B982F2E.4030304@oz.net>

On 03/10/2010 12:14 PM Miquel Torres wrote:
> Hi!
> 
> I wanted to explain a couple of things about the speed website:
> 
> - New feature: the Timeline view now defaults to a plot grid, showing
> all benchmarks at the same time. It was a feature request made more
> than once, so depending on personal tastes, you can bookmark either
> /overview/ or /timeline/. Thanks go to nsf for helping with the
> implementation.
> - The code has now moved to github as Codespeed, a benchmark
> visualization framework (http://github.com/tobami/codespeed)
> - I have updated speed.pypy.org with version 0.3. Much of the work has
> been under the hood to make it feasible for other projects to use
> codespeed as a framework.
> 
> For those interested in further development you can go to the releases
> wiki (still a work in progress):
> http://wiki.github.com/tobami/codespeed/releases
> 
> Next in the line are some DB changes to be able to save standard
> deviation data and the like. Long term goals besides world domination
> are integration with buildbot and similarly unrealistic things.
> Feedback is always welcome.

Nice looking stuff. But a couple comments:

1. IMO standard deviation is too often worse than useless, since it hides
    the true nature of the distribution. I think the assumption of normalcy
    is highly suspect for benchmark timings, and pruning may hide interesting clusters.

    I prefer to look at scattergrams, where things like clustering and correlations
    are easily apparent to the eye, as well as the amount of data (assuming a good
    mapping of density to visuals).

2. IMO benchmark timings are like travel times, comparing different vehicles.
    (pypy with jit being a vehicle capable of dynamic self-modification ;-)
    E.g., which part of travel from Stockholm to Paris would you concentrate
    on improving to improve the overall result? How about travel from Brussels to Paris?
    Or Paris to Sydney? ;-P Different things come into play in different benchmarks/trips.
    A Porsche Turbo and a 2CV will both have to wait for a ferry, if that's part of the trip.

    IOW, it would be nice to see total time broken down somehow, to see what's really
    happening.

    Don't get me wrong, the total times are certainly useful indicators of progress
    (which has been amazing).

3. Speed is ds/dt and you are showing the integral of dt/ds over the trip distance to get time.
    A 25% improvement in total time is not a 25% improvement in speed. I.e., (if you define
    improvement as a percentage change in a desired direction), for e.g. 25%:
    distance/(0.75*time) != 1.25*(distance/time).

    IMO 'speed' (the implication to me in the name speed.pypy.org) would be benchmarks/time
    more appropriately than time/benchmark.

    Both measures are useful, but time percentages are easy to mis{use,construe} ;-)

4. Is there any memory footprint data?

Regards,
Bengt Richter


From fijall at gmail.com  Thu Mar 11 01:32:52 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 10 Mar 2010 17:32:52 -0700
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <4B982F2E.4030304@oz.net>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com> 
	<4B982F2E.4030304@oz.net>
Message-ID: <693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>

Hey.

I'll answer questions that are relevant to benchmarks themselves and
not running.

On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
> On 03/10/2010 12:14 PM Miquel Torres wrote:
>> Hi!
>>
>> I wanted to explain a couple of things about the speed website:
>>
>> - New feature: the Timeline view now defaults to a plot grid, showing
>> all benchmarks at the same time. It was a feature request made more
>> than once, so depending on personal tastes, you can bookmark either
>> /overview/ or /timeline/. Thanks go to nsf for helping with the
>> implementation.
>> - The code has now moved to github as Codespeed, a benchmark
>> visualization framework (http://github.com/tobami/codespeed)
>> - I have updated speed.pypy.org with version 0.3. Much of the work has
>> been under the hood to make it feasible for other projects to use
>> codespeed as a framework.
>>
>> For those interested in further development you can go to the releases
>> wiki (still a work in progress):
>> http://wiki.github.com/tobami/codespeed/releases
>>
>> Next in the line are some DB changes to be able to save standard
>> deviation data and the like. Long term goals besides world domination
>> are integration with buildbot and similarly unrealistic things.
>> Feedback is always welcome.
>
> Nice looking stuff. But a couple comments:
>
> 1. IMO standard deviation is too often worse than useless, since it hides
> ? ?the true nature of the distribution. I think the assumption of normalcy
> ? ?is highly suspect for benchmark timings, and pruning may hide interesting clusters.
>
> ? ?I prefer to look at scattergrams, where things like clustering and correlations
> ? ?are easily apparent to the eye, as well as the amount of data (assuming a good
> ? ?mapping of density to visuals).

That's true. In general a benchmark run over time is a period of
warmup, when JIT compiles assembler followed by thing that can be
described by average and std devation. Personally I would like to have
those 3 measures separated, but didn't implement that yet (it has also
some interesting statistical questions involved). Std deviation is
useful to get whether a difference was meaningful of certain checkin
or just noise.

>
> 2. IMO benchmark timings are like travel times, comparing different vehicles.
> ? ?(pypy with jit being a vehicle capable of dynamic self-modification ;-)
> ? ?E.g., which part of travel from Stockholm to Paris would you concentrate
> ? ?on improving to improve the overall result? How about travel from Brussels to Paris?
> ? ?Or Paris to Sydney? ;-P Different things come into play in different benchmarks/trips.
> ? ?A Porsche Turbo and a 2CV will both have to wait for a ferry, if that's part of the trip.
>
> ? ?IOW, it would be nice to see total time broken down somehow, to see what's really
> ? ?happening.

I can't agree more with that. We already do split time when we perform
benchmarks by hand, but they're not yet integrated into the whole
nightly run. Total time is what users see though, that's why our
public site is focused on that. I want more information available, but
we have only limited amount of manpower and miquel already did quite
amazing job in my opinion :-) We'll probably go into more details.

The part we want to focus on past-release is speeding up certain parts
of tracing as well as limiting it's GC pressure. As you can see, the
split would be very useful for our development.

>
> ? ?Don't get me wrong, the total times are certainly useful indicators of progress
> ? ?(which has been amazing).
>
> 3. Speed is ds/dt and you are showing the integral of dt/ds over the trip distance to get time.
> ? ?A 25% improvement in total time is not a 25% improvement in speed. I.e., (if you define
> ? ?improvement as a percentage change in a desired direction), for e.g. 25%:
> ? ?distance/(0.75*time) != 1.25*(distance/time).
>
> ? ?IMO 'speed' (the implication to me in the name speed.pypy.org) would be benchmarks/time
> ? ?more appropriately than time/benchmark.
>
> ? ?Both measures are useful, but time percentages are easy to mis{use,construe} ;-)

That's correct.

Benchmarks are in general very easy to lie about and they're by
definition flawed. That's why I always include raw data when I publish
stuff on the blog, so people can work on it themselves.

>
> 4. Is there any memory footprint data?
>

No. Memory measurment is hard and it's even less useful without
breaking down. Those particular benchmarks are not very good basis for
memory measurment - in case of pypy you would mostly observe the
default allocated memory (which is roughly 10M for the interpreter +
16M for semispace GC + cache for nursery).

Also our GC is of a kind that it can run faster if you give it more
memory (not that we use this feature, but it's possible).

Cheers,
fijal

From fijall at gmail.com  Thu Mar 11 16:45:37 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 11 Mar 2010 08:45:37 -0700
Subject: [pypy-dev] [pypy-svn] r72100 - pypy/trunk/pypy/tool
In-Reply-To: <20100311131745.993F9282BD7@codespeak.net>
References: <20100311131745.993F9282BD7@codespeak.net>
Message-ID: <693bc9ab1003110745k480dc624kddb988605b9d5bbb@mail.gmail.com>

This comes with no test.

On Thu, Mar 11, 2010 at 6:17 AM,  <arigo at codespeak.net> wrote:
> Author: arigo
> Date: Thu Mar 11 14:17:44 2010
> New Revision: 72100
>
> Modified:
> ? pypy/trunk/pypy/tool/package.py
> Log:
> Accept a third argument: the name to give to the pypy-c.
>
>
> Modified: pypy/trunk/pypy/tool/package.py
> ==============================================================================
> --- pypy/trunk/pypy/tool/package.py ? ? (original)
> +++ pypy/trunk/pypy/tool/package.py ? ? Thu Mar 11 14:17:44 2010
> @@ -2,7 +2,7 @@
> ?""" A sample script that packages PyPy, provided that it's already built.
> ?Usage:
>
> -package.py pypydir [name-of-archive]
> +package.py pypydir [name-of-archive] [name-of-pypy-c]
> ?"""
>
> ?import autopath
> @@ -31,7 +31,7 @@
> ?class PyPyCNotFound(Exception):
> ? ? pass
>
> -def main(basedir, name='pypy-nightly'):
> +def main(basedir, name='pypy-nightly', rename_pypy_c='pypy-c'):
> ? ? basedir = py.path.local(basedir)
> ? ? pypy_c = basedir.join('pypy', 'translator', 'goal', 'pypy-c')
> ? ? if not pypy_c.check():
> @@ -50,11 +50,12 @@
> ? ? for file in ['LICENSE', 'README']:
> ? ? ? ? shutil.copy(str(basedir.join(file)), str(pypydir))
> ? ? pypydir.ensure('bin', dir=True)
> - ? ?shutil.copy(str(pypy_c), str(pypydir.join('bin', 'pypy-c')))
> + ? ?archive_pypy_c = pypydir.join('bin', rename_pypy_c)
> + ? ?shutil.copy(str(pypy_c), str(archive_pypy_c))
> ? ? old_dir = os.getcwd()
> ? ? try:
> ? ? ? ? os.chdir(str(builddir))
> - ? ? ? ?os.system("strip " + str(pypydir.join('bin', 'pypy-c')))
> + ? ? ? ?os.system("strip " + str(archive_pypy_c))
> ? ? ? ? os.system('tar cvjf ' + str(builddir.join(name + '.tar.bz2')) +
> ? ? ? ? ? ? ? ? ? " " + name)
> ? ? finally:
> @@ -62,10 +63,8 @@
> ? ? return builddir # for tests
>
> ?if __name__ == '__main__':
> - ? ?if len(sys.argv) == 1 or len(sys.argv) > 3:
> + ? ?if len(sys.argv) == 1 or len(sys.argv) > 4:
> ? ? ? ? print >>sys.stderr, __doc__
> ? ? ? ? sys.exit(1)
> - ? ?elif len(sys.argv) == 2:
> - ? ? ? ?main(sys.argv[1])
> ? ? else:
> - ? ? ? ?main(sys.argv[1], sys.argv[2])
> + ? ? ? ?main(*sys.argv[1:])
> _______________________________________________
> pypy-svn mailing list
> pypy-svn at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-svn
>

From renesd at gmail.com  Fri Mar 12 11:49:54 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 12 Mar 2010 10:49:54 +0000
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<4B982F2E.4030304@oz.net>
	<693bc9ab1003101632p79f9254cj3a2e1c244314fc9b@mail.gmail.com>
Message-ID: <64ddb72c1003120249pe469a7bv73a10a674b9c5771@mail.gmail.com>

btw, for python memory usage on linux
/proc/PID/status

Here is some code for linux...

wget http://rene.f0o.com/~rene/stuff/memory_usage.py

>>> import memory_usage
>>> bytes_of_resident_memory = memory_usage.resident()


Should be easy enough to add that to benchmarks at the start and end?  Maybe
calling it in the middle would be a little harder... but not too hard.


TODO: Would need to be updated for other platforms, and support measuring
child processes, tests, and code cleanup :)

cu,



On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> Hey.
>
> I'll answer questions that are relevant to benchmarks themselves and
> not running.
>
> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
> > On 03/10/2010 12:14 PM Miquel Torres wrote:
> >> Hi!
> >>
> >> I wanted to explain a couple of things about the speed website:
> >>
> >> - New feature: the Timeline view now defaults to a plot grid, showing
> >> all benchmarks at the same time. It was a feature request made more
> >> than once, so depending on personal tastes, you can bookmark either
> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
> >> implementation.
> >> - The code has now moved to github as Codespeed, a benchmark
> >> visualization framework (http://github.com/tobami/codespeed)
> >> - I have updated speed.pypy.org with version 0.3. Much of the work has
> >> been under the hood to make it feasible for other projects to use
> >> codespeed as a framework.
> >>
> >> For those interested in further development you can go to the releases
> >> wiki (still a work in progress):
> >> http://wiki.github.com/tobami/codespeed/releases
> >>
> >> Next in the line are some DB changes to be able to save standard
> >> deviation data and the like. Long term goals besides world domination
> >> are integration with buildbot and similarly unrealistic things.
> >> Feedback is always welcome.
> >
> > Nice looking stuff. But a couple comments:
> >
> > 1. IMO standard deviation is too often worse than useless, since it hides
> >    the true nature of the distribution. I think the assumption of
> normalcy
> >    is highly suspect for benchmark timings, and pruning may hide
> interesting clusters.
> >
> >    I prefer to look at scattergrams, where things like clustering and
> correlations
> >    are easily apparent to the eye, as well as the amount of data
> (assuming a good
> >    mapping of density to visuals).
>
> That's true. In general a benchmark run over time is a period of
> warmup, when JIT compiles assembler followed by thing that can be
> described by average and std devation. Personally I would like to have
> those 3 measures separated, but didn't implement that yet (it has also
> some interesting statistical questions involved). Std deviation is
> useful to get whether a difference was meaningful of certain checkin
> or just noise.
>
> >
> > 2. IMO benchmark timings are like travel times, comparing different
> vehicles.
> >    (pypy with jit being a vehicle capable of dynamic self-modification
> ;-)
> >    E.g., which part of travel from Stockholm to Paris would you
> concentrate
> >    on improving to improve the overall result? How about travel from
> Brussels to Paris?
> >    Or Paris to Sydney? ;-P Different things come into play in different
> benchmarks/trips.
> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
> that's part of the trip.
> >
> >    IOW, it would be nice to see total time broken down somehow, to see
> what's really
> >    happening.
>
> I can't agree more with that. We already do split time when we perform
> benchmarks by hand, but they're not yet integrated into the whole
> nightly run. Total time is what users see though, that's why our
> public site is focused on that. I want more information available, but
> we have only limited amount of manpower and miquel already did quite
> amazing job in my opinion :-) We'll probably go into more details.
>
> The part we want to focus on past-release is speeding up certain parts
> of tracing as well as limiting it's GC pressure. As you can see, the
> split would be very useful for our development.
>
> >
> >    Don't get me wrong, the total times are certainly useful indicators of
> progress
> >    (which has been amazing).
> >
> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the trip
> distance to get time.
> >    A 25% improvement in total time is not a 25% improvement in speed.
> I.e., (if you define
> >    improvement as a percentage change in a desired direction), for e.g.
> 25%:
> >    distance/(0.75*time) != 1.25*(distance/time).
> >
> >    IMO 'speed' (the implication to me in the name speed.pypy.org) would
> be benchmarks/time
> >    more appropriately than time/benchmark.
> >
> >    Both measures are useful, but time percentages are easy to
> mis{use,construe} ;-)
>
> That's correct.
>
> Benchmarks are in general very easy to lie about and they're by
> definition flawed. That's why I always include raw data when I publish
> stuff on the blog, so people can work on it themselves.
>
> >
> > 4. Is there any memory footprint data?
> >
>
> No. Memory measurment is hard and it's even less useful without
> breaking down. Those particular benchmarks are not very good basis for
> memory measurment - in case of pypy you would mostly observe the
> default allocated memory (which is roughly 10M for the interpreter +
> 16M for semispace GC + cache for nursery).
>
> Also our GC is of a kind that it can run faster if you give it more
> memory (not that we use this feature, but it's possible).
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100312/b5d4f914/attachment.html 

From renesd at gmail.com  Fri Mar 12 14:52:02 2010
From: renesd at gmail.com (=?ISO-8859-1?Q?Ren=E9_Dudfield?=)
Date: Fri, 12 Mar 2010 13:52:02 +0000
Subject: [pypy-dev] memory recording... Re:  speed.pypy.org quick update
Message-ID: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>

Hi again,

trying to do some research on ways to record memory usage in an X-platform
way...

keeping my notes here:

http://renesd.blogspot.com/2010/03/memory-usage-of-processes-from-python.html

So far people have come up with these two useful projects so far:
    http://code.google.com/p/psutil/
    http://code.google.com/p/pympler/

I think psutil will have most info needed to construct a decent memory
recording module for benchmarks.  However, it includes C code, so will
probably have to rip some of the memory parts out, and maybe reimplement
with ctypes.


cu,




On Fri, Mar 12, 2010 at 10:49 AM, Ren? Dudfield <renesd at gmail.com> wrote:

> btw, for python memory usage on linux
> /proc/PID/status
>
> Here is some code for linux...
>
> wget http://rene.f0o.com/~rene/stuff/memory_usage.py<http://rene.f0o.com/%7Erene/stuff/memory_usage.py>
>
> >>> import memory_usage
> >>> bytes_of_resident_memory = memory_usage.resident()
>
>
> Should be easy enough to add that to benchmarks at the start and end?
> Maybe calling it in the middle would be a little harder... but not too hard.
>
>
> TODO: Would need to be updated for other platforms, and support measuring
> child processes, tests, and code cleanup :)
>
> cu,
>
>
>
>
> On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:
>
>> Hey.
>>
>> I'll answer questions that are relevant to benchmarks themselves and
>> not running.
>>
>> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
>> > On 03/10/2010 12:14 PM Miquel Torres wrote:
>> >> Hi!
>> >>
>> >> I wanted to explain a couple of things about the speed website:
>> >>
>> >> - New feature: the Timeline view now defaults to a plot grid, showing
>> >> all benchmarks at the same time. It was a feature request made more
>> >> than once, so depending on personal tastes, you can bookmark either
>> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
>> >> implementation.
>> >> - The code has now moved to github as Codespeed, a benchmark
>> >> visualization framework (http://github.com/tobami/codespeed)
>> >> - I have updated speed.pypy.org with version 0.3. Much of the work has
>> >> been under the hood to make it feasible for other projects to use
>> >> codespeed as a framework.
>> >>
>> >> For those interested in further development you can go to the releases
>> >> wiki (still a work in progress):
>> >> http://wiki.github.com/tobami/codespeed/releases
>> >>
>> >> Next in the line are some DB changes to be able to save standard
>> >> deviation data and the like. Long term goals besides world domination
>> >> are integration with buildbot and similarly unrealistic things.
>> >> Feedback is always welcome.
>> >
>> > Nice looking stuff. But a couple comments:
>> >
>> > 1. IMO standard deviation is too often worse than useless, since it
>> hides
>> >    the true nature of the distribution. I think the assumption of
>> normalcy
>> >    is highly suspect for benchmark timings, and pruning may hide
>> interesting clusters.
>> >
>> >    I prefer to look at scattergrams, where things like clustering and
>> correlations
>> >    are easily apparent to the eye, as well as the amount of data
>> (assuming a good
>> >    mapping of density to visuals).
>>
>> That's true. In general a benchmark run over time is a period of
>> warmup, when JIT compiles assembler followed by thing that can be
>> described by average and std devation. Personally I would like to have
>> those 3 measures separated, but didn't implement that yet (it has also
>> some interesting statistical questions involved). Std deviation is
>> useful to get whether a difference was meaningful of certain checkin
>> or just noise.
>>
>> >
>> > 2. IMO benchmark timings are like travel times, comparing different
>> vehicles.
>> >    (pypy with jit being a vehicle capable of dynamic self-modification
>> ;-)
>> >    E.g., which part of travel from Stockholm to Paris would you
>> concentrate
>> >    on improving to improve the overall result? How about travel from
>> Brussels to Paris?
>> >    Or Paris to Sydney? ;-P Different things come into play in different
>> benchmarks/trips.
>> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
>> that's part of the trip.
>> >
>> >    IOW, it would be nice to see total time broken down somehow, to see
>> what's really
>> >    happening.
>>
>> I can't agree more with that. We already do split time when we perform
>> benchmarks by hand, but they're not yet integrated into the whole
>> nightly run. Total time is what users see though, that's why our
>> public site is focused on that. I want more information available, but
>> we have only limited amount of manpower and miquel already did quite
>> amazing job in my opinion :-) We'll probably go into more details.
>>
>> The part we want to focus on past-release is speeding up certain parts
>> of tracing as well as limiting it's GC pressure. As you can see, the
>> split would be very useful for our development.
>>
>> >
>> >    Don't get me wrong, the total times are certainly useful indicators
>> of progress
>> >    (which has been amazing).
>> >
>> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the
>> trip distance to get time.
>> >    A 25% improvement in total time is not a 25% improvement in speed.
>> I.e., (if you define
>> >    improvement as a percentage change in a desired direction), for e.g.
>> 25%:
>> >    distance/(0.75*time) != 1.25*(distance/time).
>> >
>> >    IMO 'speed' (the implication to me in the name speed.pypy.org) would
>> be benchmarks/time
>> >    more appropriately than time/benchmark.
>> >
>> >    Both measures are useful, but time percentages are easy to
>> mis{use,construe} ;-)
>>
>> That's correct.
>>
>> Benchmarks are in general very easy to lie about and they're by
>> definition flawed. That's why I always include raw data when I publish
>> stuff on the blog, so people can work on it themselves.
>>
>> >
>> > 4. Is there any memory footprint data?
>> >
>>
>> No. Memory measurment is hard and it's even less useful without
>> breaking down. Those particular benchmarks are not very good basis for
>> memory measurment - in case of pypy you would mostly observe the
>> default allocated memory (which is roughly 10M for the interpreter +
>> 16M for semispace GC + cache for nursery).
>>
>> Also our GC is of a kind that it can run faster if you give it more
>> memory (not that we use this feature, but it's possible).
>>
>> Cheers,
>> fijal
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100312/a13891e0/attachment-0001.htm 

From arigo at tunes.org  Fri Mar 12 19:27:51 2010
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 12 Mar 2010 19:27:51 +0100
Subject: [pypy-dev] Released: PyPy 1.2, JIT included
Message-ID: <20100312182751.GA17278@code0.codespeak.net>

==================================
PyPy 1.2: Just-in-Time Compilation
==================================

Welcome to the PyPy 1.2 release.  The highlight of this release is
to be the first that ships with a Just-in-Time compiler that is
known to be faster than CPython (and unladen swallow) on some
real-world applications (or the best benchmarks we could get for
them).  The main theme for the 1.2 release is speed.

Main site:

    http://pypy.org/

The JIT is stable and we don't observe crashes. Nevertheless we
would recommend you to treat it as beta software and as a way to try
out the JIT to see how it works for you.


Highlights of This Release
==========================

* The JIT compiler.

* Various interpreter optimizations that improve performance
  as well as help save memory.

* Introducing a new PyPy website at http://pypy.org/ , made by
  tav and improved by the PyPy team.

* Introducing http://speed.pypy.org/ , a new service that
  monitors our performance nightly, made by Miquel Torres.

* There will be ubuntu packages on "PyPy's PPA" made by
  Bartosz Skowron; however various troubles prevented us from
  having them as of now.


Known JIT problems (or why you should consider this beta software):

* The only supported platform is 32bit x86 for now, we're
  looking for help with other platforms.

* It is still memory-hungry.  There is no limit on the amount
  of RAM that the assembler can consume; it is thus possible
  (although unlikely) that the assembler ends up using
  unreasonable amounts of memory.


If you want to try PyPy, go to the "download page" on our excellent
new site at http://pypy.org/download.html and find the binary for
your platform. If the binary does not work (e.g. on Linux, because
of different versions of external .so dependencies), or if your
platform is not supported, you can try building from the source.


What is PyPy?
=============

Technically, PyPy is both a Python interpreter implementation and an
advanced compiler, or more precisely a framework for implementing
dynamic languages and generating virtual machines for them.  The
focus of this release is the introduction of a new transformation,
the JIT Compiler Generator, and its application to the Python
interpreter.

Socially, PyPy is a collaborative effort of many individuals working
together in a distributed and sprint-driven way since 2003.  PyPy
would not have gotten as far as it has without the coding, feedback
and general support from numerous people.


The PyPy release team,
    Armin Rigo, Maciej Fijalkowski and Amaury Forgeot d'Arc

Together with
    Antonio Cuni, Carl Friedrich Bolz, Holger Krekel and
    Samuele Pedroni

and many others:
    http://codespeak.net/pypy/dist/pypy/doc/contributor.html

From fredrik.johansson at gmail.com  Fri Mar 12 22:01:52 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Fri, 12 Mar 2010 22:01:52 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
Message-ID: <3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>

(Note: Trying a second time, since the mail does not seem to have arrived on
the first try. Sorry if it's a duplicate.)


Dear PyPy developers,

I have found two bugs while trying to run mpmath on pypy-1.2. This is with
the Windows binary:

Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32

My system is an HP 530, Intel Celeron M 520 @ 1.60 GHz, 1024 MB RAM running
Windows XP.

The first bug is an incompatibility with CPython. In CPython, builtin float
functions accept arbitrary objects as long as they have a __float__ method.
Consider the following code:

class x(object):
    def __float__(self): return 1.0

class y(object):
    def __complex__(self): return 1.0j

round(x())
import math; math.log(x())
import cmath; cmath.log(x())
import cmath; cmath.log(y())


CPython 2.6 gives:
1.0
0.0
0j
1.5707963267948966j

CPython 2.5 gives:
1.0
0.0
0j
TypeError: a float is required

PyPy gives:
TypeError: expected float, got x object
TypeError: expected float, got x object
0j
1.5707963267948966j

Presumably PyPy should do the same thing as 2.6 here.


Secondly, there is an error somewhere in the long multiplication code:

C:\pypy12>pypy
Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``no random bugs in random,
after
all''
>>>> import mpmath
>>>> mpmath.runtests()
mpmath imported from C:\pypy12\lib-python\2.5.2\mpmath
mpmath backend: python
mpmath mp class: <mpmath.ctx_mp.MPContext object at 0x0000001a>
mpmath version: 0.15-svn
Python version: 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0]

[...]

test_functions2
    agm                       ok        0.0446171 s
    airy                      ok        0.1957078 s
    appellf1                  ok        1.1814778 s
    bessel                    ok        0.6132425 s
    coulomb                   ok        0.4926749 s
    e1                        ok        0.0109151 s
    ei                        ok        0.0346402 s
    elliptic_integrals        ok        0.2575911 s
    erf                       ok        0.3858091 s
    exp_integrals             ok        0.0715859 s
    RPython traceback:
  File "implement_4.c", line 35008, in
opcode_impl_for_mul__AccessDirect_star_2
  File "implement_9.c", line 47885, in __mm_mul_W_LongObject_W_LongObject
  File "implement_13.c", line 3727, in _k_lopsided_mul
  File "implement_9.c", line 38027, in _k_mul
Fatal RPython error: AssertionError


The test that triggers the failure is "from mpmath import expint;
expint('1.01', '1e-1000')".

For your convenience, I have tracked down the precise numbers. Since they
have several hundred digits, I put them in an attachment.

Thank you for your hard work!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100312/3e7c664f/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mulbug.py
Type: application/octet-stream
Size: 4110 bytes
Desc: not available
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20100312/3e7c664f/attachment.obj 

From alex.gaynor at gmail.com  Fri Mar 12 22:08:10 2010
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 12 Mar 2010 15:08:10 -0600
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
Message-ID: <ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>

So one tiny pony I have is that on the tablular timeline page
(http://speed.pypy.org/timeline/) that when you mouseover a graph it
doesn't show the "coordinates" on graphs of those sizes I don't think
it adds any value, and it's farily distracting.

Thanks for your work,
Alex


-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

From santagada at gmail.com  Fri Mar 12 22:37:49 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Fri, 12 Mar 2010 18:37:49 -0300
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
Message-ID: <16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>


On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:

> So one tiny pony I have is that on the tablular timeline page
> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> doesn't show the "coordinates" on graphs of those sizes I don't think
> it adds any value, and it's farily distracting.

For a start it could be removed (that should be pretty easy) but as a second step it would be interesting to highlight and maybe show the revision or time of the closest point (if revision then highlight all points of that revision).


> Thanks for your work,
> Alex
--
Leonardo Santagada
santagada at gmail.com




From arigo at tunes.org  Sat Mar 13 04:00:16 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 13 Mar 2010 04:00:16 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
Message-ID: <20100313030015.GA24400@code0.codespeak.net>

Hi Fredrik,

On Fri, Mar 12, 2010 at 10:01:52PM +0100, Fredrik Johansson wrote:
> I have found two bugs while trying to run mpmath on pypy-1.2.

Thank you a lot for the precise bug reports :-)  We will fix it.


A bientot,

Armin.

From fijall at gmail.com  Sat Mar 13 07:23:24 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 12 Mar 2010 23:23:24 -0700
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <20100313030015.GA24400@code0.codespeak.net>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com> 
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com> 
	<20100313030015.GA24400@code0.codespeak.net>
Message-ID: <693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>

On Fri, Mar 12, 2010 at 8:00 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Fredrik,
>
> On Fri, Mar 12, 2010 at 10:01:52PM +0100, Fredrik Johansson wrote:
>> I have found two bugs while trying to run mpmath on pypy-1.2.
>
> Thank you a lot for the precise bug reports :-) ?We will fix it.
>

Actually I fixed the one about fatal rpython assertion. There is also
a problem, which seems to be windows only that I can't reproduce.

(you need mpmath for that).

from mpmath import fp
for i in range(1000): fp.zeta(0.5+37235j)

From wkornewald at freenet.de  Sat Mar 13 12:50:32 2010
From: wkornewald at freenet.de (Waldemar Kornewald)
Date: Sat, 13 Mar 2010 12:50:32 +0100
Subject: [pypy-dev] slow benchmark?
Message-ID: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>

Hi,
maybe I'm doing something totally wrong, but the attached trivial
benchmark runs several times slower than on CPython. I've tested it
with the Windows binary from your website. Your other benchmarks run
fast, BTW. Is there something wrong with my code?

Bye,
Waldemar Kornewald

--
http://www.allbuttonspressed.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fak.py
Type: application/octet-stream
Size: 598 bytes
Desc: not available
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20100313/aa76d454/attachment.obj 

From cfbolz at gmx.de  Sat Mar 13 13:15:20 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sat, 13 Mar 2010 13:15:20 +0100
Subject: [pypy-dev] slow benchmark?
In-Reply-To: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>
References: <a8d220611003130350x2b8d6896ldd1c95d0e24809a3@mail.gmail.com>
Message-ID: <348899051003130415s312e584claddf3c8954d073ef@mail.gmail.com>

Hi Waldemar,

2010/3/13 Waldemar Kornewald <wkornewald at freenet.de>:
> Hi,
> maybe I'm doing something totally wrong, but the attached trivial
> benchmark runs several times slower than on CPython. I've tested it
> with the Windows binary from your website. Your other benchmarks run
> fast, BTW. Is there something wrong with my code?

This benchmark is completely dominated by multiplying long integers.
This is nothing that our JIT can speed up, since the multiplication is
implemented in RPython (in CPython it is equivalently implemented in
C). While we use the same algorithm as CPython, we probably have some
overheads that make us slower by a constant factor. This might be
fixable, if we find somebody who is interested in looking at it.

Cheers,

Carl Friedrich

From marius at pov.lt  Sun Mar 14 02:28:08 2010
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 14 Mar 2010 03:28:08 +0200
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
Message-ID: <20100314012808.GA441@fridge.pov.lt>

On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
> > So one tiny pony I have is that on the tablular timeline page
> > (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> > doesn't show the "coordinates" on graphs of those sizes I don't think
> > it adds any value, and it's farily distracting.
> 
> For a start it could be removed (that should be pretty easy)

Actually, if you added units to those numbers, they could answer
important questions like "is a higher line better or is a lower line
better?"

> but as a
> second step it would be interesting to highlight and maybe show the
> revision or time of the closest point (if revision then highlight all
> points of that revision).

Some kind of rounding would be nice, as seeing "0.6 seconds in revision
71807.4" is a bit weird.

Very shiny website, BTW, I love it.

Marius Gedminas
-- 
Q:  How many IBM CPU's does it take to execute a job?
A:  Four; three to hold it down, and one to rip its head off.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20100314/33fb45b4/attachment.pgp 

From fijall at gmail.com  Sun Mar 14 08:04:45 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 14 Mar 2010 00:04:45 -0700
Subject: [pypy-dev] issue tracker
Message-ID: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>

Hello.

I'm considering changing issue tracker as apparently a lot of people
gave feedback after the release. For me the current incarnation is
unbearable - I can't easily track things like "submitted" or "triaged"
from "windows build requires superuser" which always pop at the top or
"polish objspace initialization" which is probably important but I
already decided not to deal with it.

It might be the inherent problem with roundup or just our
implementation. pylib for example moved to bitbucket, probably also
for similar reasons.

Do you have any thoughts or obvious choices?

Cheers,
fijal

From alex.gaynor at gmail.com  Sun Mar 14 08:09:34 2010
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Sun, 14 Mar 2010 01:09:34 -0600
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <ae6584d31003132309u94dce06u482111ea3cb8ed4b@mail.gmail.com>

On Sun, Mar 14, 2010 at 1:04 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?
>
> Cheers,
> fijal
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

I'd say trac is the "obvious" choice since we are on svn.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me

From micahel at gmail.com  Sun Mar 14 08:20:47 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Sun, 14 Mar 2010 20:20:47 +1300
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <aa0dfd861003132320w763e9d6dg2c1326676296215@mail.gmail.com>

On 14 March 2010 20:04, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?

There's always Launchpad; it's not perfect but if it works for ubuntu
it must be at least OK :-)  Also, someone else runs the server, which
probably isn't to be discounted, and can probably be relied on to
improve over time -- there's a lot of development happening on it.  It
does all the usual things reasonably well, with sensible statuses and
tagging and assignment and so on. Possibly a disadvantage is that
because it has a focus on collaboration between projects, it's not
tweakable to meet any particular project's needs.

It should be possible to import the bugs from roundup into lp,
although roundup's 'build your own schema' thing probably means some
hand-holding will be required.

Cheers,
mwh

From santagada at gmail.com  Sun Mar 14 11:11:56 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Sun, 14 Mar 2010 07:11:56 -0300
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <20100314012808.GA441@fridge.pov.lt>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
Message-ID: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>


On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:

> On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>>> So one tiny pony I have is that on the tablular timeline page
>>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>>> doesn't show the "coordinates" on graphs of those sizes I don't think
>>> it adds any value, and it's farily distracting.
>> 
>> For a start it could be removed (that should be pretty easy)
> 
> Actually, if you added units to those numbers, they could answer
> important questions like "is a higher line better or is a lower line
> better?"

Yes but axis should be named in a always visible place, so when people see the graphs they know what they mean.

>> but as a
>> second step it would be interesting to highlight and maybe show the
>> revision or time of the closest point (if revision then highlight all
>> points of that revision).
> 
> Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> 71807.4" is a bit weird.

No rounding but actually showing the data for the closest point and not where the mouse is over.

> Very shiny website, BTW, I love it.

I think this is a clear way to show performance for non developers and is great even for developers, it is a win win website :)

--
Leonardo Santagada
santagada at gmail.com




From tobami at googlemail.com  Sun Mar 14 14:14:25 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 14:14:25 +0100
Subject: [pypy-dev] memory recording... Re: speed.pypy.org quick update
In-Reply-To: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>
References: <64ddb72c1003120552x77fb215rd6d4b36a3ccf7dc9@mail.gmail.com>
Message-ID: <c561346d1003140614w580b1d1cp889c5be0d6704bf5@mail.gmail.com>

Yeah, we really need to sit down and find an acceptable way to measure and
save memory consumption.data.


2010/3/12 Ren? Dudfield <renesd at gmail.com>

> Hi again,
>
> trying to do some research on ways to record memory usage in an X-platform
> way...
>
> keeping my notes here:
>
> http://renesd.blogspot.com/2010/03/memory-usage-of-processes-from-python.html
>
> So far people have come up with these two useful projects so far:
>     http://code.google.com/p/psutil/
>     http://code.google.com/p/pympler/
>
> I think psutil will have most info needed to construct a decent memory
> recording module for benchmarks.  However, it includes C code, so will
> probably have to rip some of the memory parts out, and maybe reimplement
> with ctypes.
>
>
> cu,
>
>
>
>
> On Fri, Mar 12, 2010 at 10:49 AM, Ren? Dudfield <renesd at gmail.com> wrote:
>
>> btw, for python memory usage on linux
>> /proc/PID/status
>>
>> Here is some code for linux...
>>
>> wget http://rene.f0o.com/~rene/stuff/memory_usage.py<http://rene.f0o.com/%7Erene/stuff/memory_usage.py>
>>
>> >>> import memory_usage
>> >>> bytes_of_resident_memory = memory_usage.resident()
>>
>>
>> Should be easy enough to add that to benchmarks at the start and end?
>> Maybe calling it in the middle would be a little harder... but not too hard.
>>
>>
>> TODO: Would need to be updated for other platforms, and support measuring
>> child processes, tests, and code cleanup :)
>>
>> cu,
>>
>>
>>
>>
>> On Thu, Mar 11, 2010 at 12:32 AM, Maciej Fijalkowski <fijall at gmail.com>wrote:
>>
>>> Hey.
>>>
>>> I'll answer questions that are relevant to benchmarks themselves and
>>> not running.
>>>
>>> On Wed, Mar 10, 2010 at 4:45 PM, Bengt Richter <bokr at oz.net> wrote:
>>> > On 03/10/2010 12:14 PM Miquel Torres wrote:
>>> >> Hi!
>>> >>
>>> >> I wanted to explain a couple of things about the speed website:
>>> >>
>>> >> - New feature: the Timeline view now defaults to a plot grid, showing
>>> >> all benchmarks at the same time. It was a feature request made more
>>> >> than once, so depending on personal tastes, you can bookmark either
>>> >> /overview/ or /timeline/. Thanks go to nsf for helping with the
>>> >> implementation.
>>> >> - The code has now moved to github as Codespeed, a benchmark
>>> >> visualization framework (http://github.com/tobami/codespeed)
>>> >> - I have updated speed.pypy.org with version 0.3. Much of the work
>>> has
>>> >> been under the hood to make it feasible for other projects to use
>>> >> codespeed as a framework.
>>> >>
>>> >> For those interested in further development you can go to the releases
>>> >> wiki (still a work in progress):
>>> >> http://wiki.github.com/tobami/codespeed/releases
>>> >>
>>> >> Next in the line are some DB changes to be able to save standard
>>> >> deviation data and the like. Long term goals besides world domination
>>> >> are integration with buildbot and similarly unrealistic things.
>>> >> Feedback is always welcome.
>>> >
>>> > Nice looking stuff. But a couple comments:
>>> >
>>> > 1. IMO standard deviation is too often worse than useless, since it
>>> hides
>>> >    the true nature of the distribution. I think the assumption of
>>> normalcy
>>> >    is highly suspect for benchmark timings, and pruning may hide
>>> interesting clusters.
>>> >
>>> >    I prefer to look at scattergrams, where things like clustering and
>>> correlations
>>> >    are easily apparent to the eye, as well as the amount of data
>>> (assuming a good
>>> >    mapping of density to visuals).
>>>
>>> That's true. In general a benchmark run over time is a period of
>>> warmup, when JIT compiles assembler followed by thing that can be
>>> described by average and std devation. Personally I would like to have
>>> those 3 measures separated, but didn't implement that yet (it has also
>>> some interesting statistical questions involved). Std deviation is
>>> useful to get whether a difference was meaningful of certain checkin
>>> or just noise.
>>>
>>> >
>>> > 2. IMO benchmark timings are like travel times, comparing different
>>> vehicles.
>>> >    (pypy with jit being a vehicle capable of dynamic self-modification
>>> ;-)
>>> >    E.g., which part of travel from Stockholm to Paris would you
>>> concentrate
>>> >    on improving to improve the overall result? How about travel from
>>> Brussels to Paris?
>>> >    Or Paris to Sydney? ;-P Different things come into play in different
>>> benchmarks/trips.
>>> >    A Porsche Turbo and a 2CV will both have to wait for a ferry, if
>>> that's part of the trip.
>>> >
>>> >    IOW, it would be nice to see total time broken down somehow, to see
>>> what's really
>>> >    happening.
>>>
>>> I can't agree more with that. We already do split time when we perform
>>> benchmarks by hand, but they're not yet integrated into the whole
>>> nightly run. Total time is what users see though, that's why our
>>> public site is focused on that. I want more information available, but
>>> we have only limited amount of manpower and miquel already did quite
>>> amazing job in my opinion :-) We'll probably go into more details.
>>>
>>> The part we want to focus on past-release is speeding up certain parts
>>> of tracing as well as limiting it's GC pressure. As you can see, the
>>> split would be very useful for our development.
>>>
>>> >
>>> >    Don't get me wrong, the total times are certainly useful indicators
>>> of progress
>>> >    (which has been amazing).
>>> >
>>> > 3. Speed is ds/dt and you are showing the integral of dt/ds over the
>>> trip distance to get time.
>>> >    A 25% improvement in total time is not a 25% improvement in speed.
>>> I.e., (if you define
>>> >    improvement as a percentage change in a desired direction), for e.g.
>>> 25%:
>>> >    distance/(0.75*time) != 1.25*(distance/time).
>>> >
>>> >    IMO 'speed' (the implication to me in the name speed.pypy.org)
>>> would be benchmarks/time
>>> >    more appropriately than time/benchmark.
>>> >
>>> >    Both measures are useful, but time percentages are easy to
>>> mis{use,construe} ;-)
>>>
>>> That's correct.
>>>
>>> Benchmarks are in general very easy to lie about and they're by
>>> definition flawed. That's why I always include raw data when I publish
>>> stuff on the blog, so people can work on it themselves.
>>>
>>> >
>>> > 4. Is there any memory footprint data?
>>> >
>>>
>>> No. Memory measurment is hard and it's even less useful without
>>> breaking down. Those particular benchmarks are not very good basis for
>>> memory measurment - in case of pypy you would mostly observe the
>>> default allocated memory (which is roughly 10M for the interpreter +
>>> 16M for semispace GC + cache for nursery).
>>>
>>> Also our GC is of a kind that it can run faster if you give it more
>>> memory (not that we use this feature, but it's possible).
>>>
>>> Cheers,
>>> fijal
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>>
>>
>>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100314/167ff28e/attachment-0001.htm 

From benjamin at python.org  Sun Mar 14 15:03:45 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 14 Mar 2010 08:03:45 -0600
Subject: [pypy-dev] issue tracker
In-Reply-To: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
References: <693bc9ab1003132304i7b3d82f5q9b5103835cd385eb@mail.gmail.com>
Message-ID: <1afaf6161003140703p47ae704av88e09d7de00546f3@mail.gmail.com>

2010/3/14 Maciej Fijalkowski <fijall at gmail.com>:
> Hello.
>
> I'm considering changing issue tracker as apparently a lot of people
> gave feedback after the release. For me the current incarnation is
> unbearable - I can't easily track things like "submitted" or "triaged"
> from "windows build requires superuser" which always pop at the top or
> "polish objspace initialization" which is probably important but I
> already decided not to deal with it.
>
> It might be the inherent problem with roundup or just our
> implementation. pylib for example moved to bitbucket, probably also
> for similar reasons.
>
> Do you have any thoughts or obvious choices?

Perhaps we can try and get a python.org roundup tracker. Jython does
this, and the CPython one is quite bearable.



-- 
Regards,
Benjamin

From marius at pov.lt  Sun Mar 14 15:06:26 2010
From: marius at pov.lt (Marius Gedminas)
Date: Sun, 14 Mar 2010 16:06:26 +0200
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
Message-ID: <20100314140625.GA7406@fridge.pov.lt>

On Sun, Mar 14, 2010 at 07:11:56AM -0300, Leonardo Santagada wrote:
> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> > 71807.4" is a bit weird.
> 
> No rounding but actually showing the data for the closest point and
> not where the mouse is over.

Sorry, yes, I meant rounding of coordinates to the nearest data point,
not rounding the interpolated data to appear more truthy.

Marius Gedminas
-- 
UNIX is user friendly. It's just selective about who its friends are.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20100314/19f1dd0d/attachment.pgp 

From tobami at googlemail.com  Sun Mar 14 19:04:22 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 19:04:22 +0100
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
Message-ID: <c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>

Hey, thanks for the feedback.

- In the timeline grid, I agree showing coordinates is not useful. I have
disabled that
- The rounding: yes, it is pretty stupid to show "revision 71807.4". That
goes away once coordinates are disabled though.

>I think this is a clear way to show performance for non developers and is
great even for developers, it is a win
> win website :)

I'm very glad to hear that. That was exactly my intention when starting the
project :-D
I had to scratch my itch of wanting to better follow pypy's performance as a
common python developer, but I also recognized that being such a performance
oriented project, pypy badly needed good performance regression monitoring
and progress tracking.

Cheers!
Miquel


2010/3/14 Leonardo Santagada <santagada at gmail.com>

>
> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>
> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
> >>> So one tiny pony I have is that on the tablular timeline page
> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
> >>> it adds any value, and it's farily distracting.
> >>
> >> For a start it could be removed (that should be pretty easy)
> >
> > Actually, if you added units to those numbers, they could answer
> > important questions like "is a higher line better or is a lower line
> > better?"
>
> Yes but axis should be named in a always visible place, so when people see
> the graphs they know what they mean.
>
> >> but as a
> >> second step it would be interesting to highlight and maybe show the
> >> revision or time of the closest point (if revision then highlight all
> >> points of that revision).
> >
> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
> > 71807.4" is a bit weird.
>
> No rounding but actually showing the data for the closest point and not
> where the mouse is over.
>
> > Very shiny website, BTW, I love it.
>
> I think this is a clear way to show performance for non developers and is
> great even for developers, it is a win win website :)
>
> --
> Leonardo Santagada
> santagada at gmail.com
>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100314/ea7f5a80/attachment.htm 

From tobami at googlemail.com  Sun Mar 14 19:06:42 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Sun, 14 Mar 2010 19:06:42 +0100
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com>
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com>
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt>
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
	<c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com>
Message-ID: <c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>

- About showing axis labels, it can be done, of course, as they are for a
single timeline plot. But please bear in mind that the timeline grid is a
bit of a compromise UI-wise. It currently shows 20 plots (!) at the same
time (probably more in the future if Maciej keeps adding more). Showing a
"seconds" or "lower is better" in all of them would be a big waste of screen
real state. You also have to assume that people who access this site are
knowledgeable enough for being able to interpret plots correctly, specially
if the single plot view does specify units and ALL yaxis are in seconds. It
is not like they change to flops and then to mips and seconds again...



2010/3/14 Miquel Torres <tobami at googlemail.com>

> Hey, thanks for the feedback.
>
> - In the timeline grid, I agree showing coordinates is not useful. I have
> disabled that
> - The rounding: yes, it is pretty stupid to show "revision 71807.4". That
> goes away once coordinates are disabled though.
>
>
> >I think this is a clear way to show performance for non developers and is
> great even for developers, it is a win
> > win website :)
>
> I'm very glad to hear that. That was exactly my intention when starting the
> project :-D
> I had to scratch my itch of wanting to better follow pypy's performance as
> a common python developer, but I also recognized that being such a
> performance oriented project, pypy badly needed good performance regression
> monitoring and progress tracking.
>
> Cheers!
> Miquel
>
>
> 2010/3/14 Leonardo Santagada <santagada at gmail.com>
>
>
>> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>>
>> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>> >>> So one tiny pony I have is that on the tablular timeline page
>> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
>> >>> it adds any value, and it's farily distracting.
>> >>
>> >> For a start it could be removed (that should be pretty easy)
>> >
>> > Actually, if you added units to those numbers, they could answer
>> > important questions like "is a higher line better or is a lower line
>> > better?"
>>
>> Yes but axis should be named in a always visible place, so when people see
>> the graphs they know what they mean.
>>
>> >> but as a
>> >> second step it would be interesting to highlight and maybe show the
>> >> revision or time of the closest point (if revision then highlight all
>> >> points of that revision).
>> >
>> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
>> > 71807.4" is a bit weird.
>>
>> No rounding but actually showing the data for the closest point and not
>> where the mouse is over.
>>
>> > Very shiny website, BTW, I love it.
>>
>> I think this is a clear way to show performance for non developers and is
>> great even for developers, it is a win win website :)
>>
>> --
>> Leonardo Santagada
>> santagada at gmail.com
>>
>>
>>
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100314/70259e7b/attachment.htm 

From fijall at gmail.com  Sun Mar 14 19:18:15 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 14 Mar 2010 11:18:15 -0700
Subject: [pypy-dev] speed.pypy.org quick update
In-Reply-To: <c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>
References: <c561346d1003101214q6dd04f51lb13defb2791d64ec@mail.gmail.com> 
	<ae6584d31003121308s5ff1e3dtdd8348d25fdc1a59@mail.gmail.com> 
	<16870A89-5284-4BBA-934A-DF3CCE12D7C2@gmail.com>
	<20100314012808.GA441@fridge.pov.lt> 
	<2364806D-9111-43DB-95AB-370B35D3696C@gmail.com>
	<c561346d1003141104s1fecc7dey40b87c274e50a113@mail.gmail.com> 
	<c561346d1003141106g66738a36reafeeb22bb27606b@mail.gmail.com>
Message-ID: <693bc9ab1003141118n74267c25g8dffe67d61233140@mail.gmail.com>

Hello.

Please keep in mind that I'm using that on 12' monitor and that makes
screen space even more valuable ;-)

Cheers,
fijal

On Sun, Mar 14, 2010 at 11:06 AM, Miquel Torres <tobami at googlemail.com> wrote:
> - About showing axis labels, it can be done, of course, as they are for a
> single timeline plot. But please bear in mind that the timeline grid is a
> bit of a compromise UI-wise. It currently shows 20 plots (!) at the same
> time (probably more in the future if Maciej keeps adding more). Showing a
> "seconds" or "lower is better" in all of them would be a big waste of screen
> real state. You also have to assume that people who access this site are
> knowledgeable enough for being able to interpret plots correctly, specially
> if the single plot view does specify units and ALL yaxis are in seconds. It
> is not like they change to flops and then to mips and seconds again...
>
>
>
> 2010/3/14 Miquel Torres <tobami at googlemail.com>
>>
>> Hey, thanks for the feedback.
>>
>> - In the timeline grid, I agree showing coordinates is not useful. I have
>> disabled that
>> - The rounding: yes, it is pretty stupid to show "revision 71807.4". That
>> goes away once coordinates are disabled though.
>>
>> >I think this is a clear way to show performance for non developers and is
>> > great even for developers, it is a win
>> > win website :)
>>
>> I'm very glad to hear that. That was exactly my intention when starting
>> the project :-D
>> I had to scratch my itch of wanting to better follow pypy's performance as
>> a common python developer, but I also recognized that being such a
>> performance oriented project, pypy badly needed good performance regression
>> monitoring and progress tracking.
>>
>> Cheers!
>> Miquel
>>
>>
>> 2010/3/14 Leonardo Santagada <santagada at gmail.com>
>>>
>>> On Mar 13, 2010, at 10:28 PM, Marius Gedminas wrote:
>>>
>>> > On Fri, Mar 12, 2010 at 06:37:49PM -0300, Leonardo Santagada wrote:
>>> >> On Mar 12, 2010, at 6:08 PM, Alex Gaynor wrote:
>>> >>> So one tiny pony I have is that on the tablular timeline page
>>> >>> (http://speed.pypy.org/timeline/) that when you mouseover a graph it
>>> >>> doesn't show the "coordinates" on graphs of those sizes I don't think
>>> >>> it adds any value, and it's farily distracting.
>>> >>
>>> >> For a start it could be removed (that should be pretty easy)
>>> >
>>> > Actually, if you added units to those numbers, they could answer
>>> > important questions like "is a higher line better or is a lower line
>>> > better?"
>>>
>>> Yes but axis should be named in a always visible place, so when people
>>> see the graphs they know what they mean.
>>>
>>> >> but as a
>>> >> second step it would be interesting to highlight and maybe show the
>>> >> revision or time of the closest point (if revision then highlight all
>>> >> points of that revision).
>>> >
>>> > Some kind of rounding would be nice, as seeing "0.6 seconds in revision
>>> > 71807.4" is a bit weird.
>>>
>>> No rounding but actually showing the data for the closest point and not
>>> where the mouse is over.
>>>
>>> > Very shiny website, BTW, I love it.
>>>
>>> I think this is a clear way to show performance for non developers and is
>>> great even for developers, it is a win win website :)
>>>
>>> --
>>> Leonardo Santagada
>>> santagada at gmail.com
>>>
>>>
>>>
>>> _______________________________________________
>>> pypy-dev at codespeak.net
>>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>
>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From micahel at gmail.com  Mon Mar 15 08:21:57 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Mon, 15 Mar 2010 20:21:57 +1300
Subject: [pypy-dev] bzr import of pypy from svn
Message-ID: <aa0dfd861003150021l1f197cd3u4e87503dbe15df49@mail.gmail.com>

As I've mentioned a couple of times on #pypy, during the codespeak
outage I imported pypy trunk from Subversion into bzr (from a local
tarball of the repo I had).  Since codespeak came back up again I've
updated it, and Alexander asked for an import of all the branches
rather than just trunk.  This is now done and I've just finishing
pushing them up to Launchpad:

https://code.edge.launchpad.net/~mwhudson/pypy/

I can keep this up to date by hand, but automating it is a bit tricky
given that uploading the end result requires my ssh key.  I'll try to
think of something.

Launchpad can import subversion branches automatically of course, but
this is currently broken by a subversion bug
(http://subversion.tigris.org/issues/show_bug.cgi?id=3480).  A
workaround would be setting up read-only svnserve access or maaaybe
upgrading svn.

Cheers,
mwh

From fijall at gmail.com  Mon Mar 15 20:22:35 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 15 Mar 2010 13:22:35 -0600
Subject: [pypy-dev] someone is wrong on the internet
Message-ID: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>

This is particularly good hate mail I found:

http://www.linuxtoday.com/news_story.php3?ltsn=2010-03-15-013-35-NW-RL-0000

Enjoy :)

From fredrik.johansson at gmail.com  Fri Mar 12 20:44:35 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Fri, 12 Mar 2010 20:44:35 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
Message-ID: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>

Dear PyPy developers,

I have found two bugs while trying to run mpmath on pypy-1.2. This is with
the Windows binary:

Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32

My system is an HP 530, Intel Celeron M 520 @ 1.60 GHz, 1024 MB RAM running
Windows XP.

The first bug is an incompatibility with CPython. In CPython, builtin float
functions accept arbitrary objects as long as they have a __float__ method.
Consider the following code:

class x(object):
    def __float__(self): return 1.0

class y(object):
    def __complex__(self): return 1.0j

round(x())
import math; math.log(x())
import cmath; cmath.log(x())
import cmath; cmath.log(y())


CPython 2.6 gives:
1.0
0.0
0j
1.5707963267948966j

CPython 2.5 gives:
1.0
0.0
0j
TypeError: a float is required

PyPy gives:
TypeError: expected float, got x object
TypeError: expected float, got x object
0j
1.5707963267948966j

Presumably PyPy should do the same thing as 2.6 here.


Secondly, there is an error somewhere in the long multiplication code:

C:\pypy12>pypy
Python 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0] on win32
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``no random bugs in random,
after
all''
>>>> import mpmath
>>>> mpmath.runtests()
mpmath imported from C:\pypy12\lib-python\2.5.2\mpmath
mpmath backend: python
mpmath mp class: <mpmath.ctx_mp.MPContext object at 0x0000001a>
mpmath version: 0.15-svn
Python version: 2.5.2 (72130, Mar 12 2010, 12:25:40)
[PyPy 1.2.0]

[...]

test_functions2
    agm                       ok        0.0446171 s
    airy                      ok        0.1957078 s
    appellf1                  ok        1.1814778 s
    bessel                    ok        0.6132425 s
    coulomb                   ok        0.4926749 s
    e1                        ok        0.0109151 s
    ei                        ok        0.0346402 s
    elliptic_integrals        ok        0.2575911 s
    erf                       ok        0.3858091 s
    exp_integrals             ok        0.0715859 s
    RPython traceback:
  File "implement_4.c", line 35008, in
opcode_impl_for_mul__AccessDirect_star_2
  File "implement_9.c", line 47885, in __mm_mul_W_LongObject_W_LongObject
  File "implement_13.c", line 3727, in _k_lopsided_mul
  File "implement_9.c", line 38027, in _k_mul
Fatal RPython error: AssertionError


The test that triggers the failure is "from mpmath import expint;
expint('1.01', '1e-1000')".

For your convenience, I have tracked down the precise numbers. Since they
have several hundred digits, I put them in an attachment.

Thank you for your hard work!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100312/7f6663fc/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mulbug.py
Type: application/octet-stream
Size: 4110 bytes
Desc: not available
Url : http://codespeak.net/pipermail/pypy-dev/attachments/20100312/7f6663fc/attachment.obj 

From santagada at gmail.com  Mon Mar 15 22:11:56 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 15 Mar 2010 18:11:56 -0300
Subject: [pypy-dev] someone is wrong on the internet
In-Reply-To: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>
References: <693bc9ab1003151222j2ab003c0j8f8efe7f4bfb15c5@mail.gmail.com>
Message-ID: <1439437F-104D-4D23-9FAB-2ED44E8BBE63@gmail.com>

Mega troll win. He is probably in engadget and slashdot trolling all day. Answering to him would only provoke the troll.

On Mar 15, 2010, at 4:22 PM, Maciej Fijalkowski wrote:

> This is particularly good hate mail I found:
> 
> http://www.linuxtoday.com/news_story.php3?ltsn=2010-03-15-013-35-NW-RL-0000
> 
> Enjoy :)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

--
Leonardo Santagada
santagada at gmail.com




From arigo at tunes.org  Tue Mar 16 10:01:47 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 16 Mar 2010 10:01:47 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
	<20100313030015.GA24400@code0.codespeak.net>
	<693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
Message-ID: <20100316090147.GA17016@code0.codespeak.net>

Hi Maciek, hi Fredrik,

On Fri, Mar 12, 2010 at 11:23:24PM -0700, Maciej Fijalkowski wrote:
> Actually I fixed the one about fatal rpython assertion. There is also
> a problem, which seems to be windows only that I can't reproduce.
> 
> from mpmath import fp
> for i in range(1000): fp.zeta(0.5+37235j)

I fixed this in r72263.  The issue was hard to pinpoint because it was
caused by a fast-path malloc: on the slow path (i.e. rarely), it would
call random C code without saving the XMM registers.  That's wrong:
there is -- on Windows only? -- a path through that C code that trashes
the XMM registers.


A bientot,

Armin.

From fredrik.johansson at gmail.com  Tue Mar 16 19:44:54 2010
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Tue, 16 Mar 2010 19:44:54 +0100
Subject: [pypy-dev] Two bugs in PyPy 1.2
In-Reply-To: <20100316090147.GA17016@code0.codespeak.net>
References: <3d0cebfb1003121144i2a00ef6asf497389fe36bdddb@mail.gmail.com>
	<3d0cebfb1003121301w772a8973m6aa9994e217bee98@mail.gmail.com>
	<20100313030015.GA24400@code0.codespeak.net>
	<693bc9ab1003122223u1b88d3f1p3b88afa3e3476214@mail.gmail.com>
	<20100316090147.GA17016@code0.codespeak.net>
Message-ID: <3d0cebfb1003161144k1788f7acraf669db179668f72@mail.gmail.com>

On Tue, Mar 16, 2010 at 10:01 AM, Armin Rigo <arigo at tunes.org> wrote:

> Hi Maciek, hi Fredrik,
>
> On Fri, Mar 12, 2010 at 11:23:24PM -0700, Maciej Fijalkowski wrote:
> > Actually I fixed the one about fatal rpython assertion. There is also
> > a problem, which seems to be windows only that I can't reproduce.
> >
> > from mpmath import fp
> > for i in range(1000): fp.zeta(0.5+37235j)
>
> I fixed this in r72263.  The issue was hard to pinpoint because it was
> caused by a fast-path malloc: on the slow path (i.e. rarely), it would
> call random C code without saving the XMM registers.  That's wrong:
> there is -- on Windows only? -- a path through that C code that trashes
> the XMM registers.
>
>
> A bientot,
>
> Armin.
>

Thank you!

Fredrik
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100316/069f9833/attachment.htm 

From fijall at gmail.com  Wed Mar 17 02:30:34 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 16 Mar 2010 19:30:34 -0600
Subject: [pypy-dev] autonosiness in roundup
Message-ID: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>

Anyone knows how to set up autonosy on roundup? I only get information
about new tickets and then I don't get followups.

From micahel at gmail.com  Wed Mar 17 02:36:17 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Wed, 17 Mar 2010 14:36:17 +1300
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
Message-ID: <aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>

On 17 March 2010 14:30, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Anyone knows how to set up autonosy on roundup? I only get information
> about new tickets and then I don't get followups.

I think it involves editing a particular file on codespeak.  Yay!

Cheers,
mwh

From benjamin at python.org  Wed Mar 17 02:36:23 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 16 Mar 2010 20:36:23 -0500
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
Message-ID: <1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>

2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
> Anyone knows how to set up autonosy on roundup? I only get information
> about new tickets and then I don't get followups.

When that's figured out, I'd also like to be on autonosy.



-- 
Regards,
Benjamin

From micahel at gmail.com  Wed Mar 17 02:38:29 2010
From: micahel at gmail.com (Michael Hudson-Doyle)
Date: Wed, 17 Mar 2010 14:38:29 +1300
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<aa0dfd861003161836q7ba9daafq26a0fd61d18d0b25@mail.gmail.com>
Message-ID: <aa0dfd861003161838y49e1fa55m70b8ca82505ef9b1@mail.gmail.com>

On 17 March 2010 14:36, Michael Hudson-Doyle <micahel at gmail.com> wrote:
> On 17 March 2010 14:30, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> Anyone knows how to set up autonosy on roundup? I only get information
>> about new tickets and then I don't get followups.
>
> I think it involves editing a particular file on codespeak. ?Yay!

In particular, /www/codespeak.net/issue/pypy-dev/detectors/nosyreaction.py

Cheers,
mwh

From anto.cuni at gmail.com  Wed Mar 17 10:09:11 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 17 Mar 2010 10:09:11 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
Message-ID: <4BA09C37.8080400@gmail.com>

Benjamin Peterson wrote:
> 2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
>> Anyone knows how to set up autonosy on roundup? I only get information
>> about new tickets and then I don't get followups.
> 
> When that's figured out, I'd also like to be on autonosy.

me too :-)

From amauryfa at gmail.com  Wed Mar 17 10:19:19 2010
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 17 Mar 2010 10:19:19 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <4BA09C37.8080400@gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
Message-ID: <e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>

2010/3/17 Antonio Cuni <anto.cuni at gmail.com>

> Benjamin Peterson wrote:
> > 2010/3/16 Maciej Fijalkowski <fijall at gmail.com>:
> >> Anyone knows how to set up autonosy on roundup? I only get information
> >> about new tickets and then I don't get followups.
> >
> > When that's figured out, I'd also like to be on autonosy.
>
> me too :-)


What about a mailing list, similar to Python-bugs-list?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100317/c9fd3df0/attachment.htm 

From micktwomey at gmail.com  Fri Mar 19 15:09:27 2010
From: micktwomey at gmail.com (Michael Twomey)
Date: Fri, 19 Mar 2010 14:09:27 +0000
Subject: [pypy-dev] Python Ireland Conference
Message-ID: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>

Hi all,

I'm one of the folks involved in organising the first python Ireland
conference, to be held in Dublin on Saturday 17th of July (with
sprints on Sunday).

We'd love it if someone from PyPy could make it over to give a talk.
The audience is your typical spread of python developers, many of whom
haven't been exposed to PyPy and what it can do. An introduction to
PyPy and the benefits it offers would go down very well.

cheers,
  Michael

From arigo at tunes.org  Tue Mar 23 12:49:31 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 12:49:31 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
Message-ID: <20100323114931.GA15591@code0.codespeak.net>

Hi,

On Wed, Mar 17, 2010 at 10:19:19AM +0100, Amaury Forgeot d'Arc wrote:
> What about a mailing list, similar to Python-bugs-list?

That's a very good idea, but unfortunately I don't know if there are
issues, like what occurs when you reply to an e-mail in that list.  It
probably just works if the mailing list is configured so that replies go
by default to the author of the mail instead of to the list's address.
If that's not an issue then I could make the mailing list (provided
maybe with some reminder of how I'm supposed to do that on
codespeak...).


A bientot,

Armin.

From holger at merlinux.eu  Tue Mar 23 17:28:38 2010
From: holger at merlinux.eu (holger krekel)
Date: Tue, 23 Mar 2010 17:28:38 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323114931.GA15591@code0.codespeak.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
Message-ID: <20100323162838.GU26514@trillke.net>

Hi, 

On Tue, Mar 23, 2010 at 12:49 +0100, Armin Rigo wrote:
> Hi,
> 
> On Wed, Mar 17, 2010 at 10:19:19AM +0100, Amaury Forgeot d'Arc wrote:
> > What about a mailing list, similar to Python-bugs-list?
> 
> That's a very good idea, but unfortunately I don't know if there are
> issues, like what occurs when you reply to an e-mail in that list.  It
> probably just works if the mailing list is configured so that replies go
> by default to the author of the mail instead of to the list's address.
> If that's not an issue then I could make the mailing list (provided
> maybe with some reminder of how I'm supposed to do that on
> codespeak...).

head to http://codespeak.net/mailman/create
and look into /root/.ssh/mmsitepass for password (requires root)
H.

From zejn at kiberpipa.org  Tue Mar 23 18:02:59 2010
From: zejn at kiberpipa.org (Gasper Zejn)
Date: Tue, 23 Mar 2010 18:02:59 +0100
Subject: [pypy-dev] Nightly builds are unavailable
Message-ID: <201003231803.00045.zejn@kiberpipa.org>

The links at the URL http://buildbot.pypy.org/nightly/ are failing with 
"resource unavailable"; misconfiguration?

Just to make sure it's a known problem.

KR,
Gasper

From arigo at tunes.org  Tue Mar 23 18:38:32 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 18:38:32 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323162838.GU26514@trillke.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
	<20100323162838.GU26514@trillke.net>
Message-ID: <20100323173832.GA10910@code0.codespeak.net>

Hi,

On Tue, Mar 23, 2010 at 05:28:38PM +0100, holger krekel wrote:
> > > What about a mailing list, similar to Python-bugs-list?

Done.  The list is there:

    http://codespeak.net/mailman/listinfo/pypy-issue

Note that I did not remove the existing hacks.  Please ask me.  One
issue is that I'm not sure how to make "pypy-issue" autonosy on the
already-existing tracker entries -- so far I fear that it's going to
receive only the new entries (including follow-ups to that).


A bientot,

Armin.

From anto.cuni at gmail.com  Tue Mar 23 19:23:45 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 23 Mar 2010 19:23:45 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <201003231803.00045.zejn@kiberpipa.org>
References: <201003231803.00045.zejn@kiberpipa.org>
Message-ID: <58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>

On Tue, Mar 23, 2010 at 6:02 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
> The links at the URL http://buildbot.pypy.org/nightly/ are failing with
> "resource unavailable"; misconfiguration?
>
> Just to make sure it's a known problem.

It's working for me right now. Maybe it has just been down for a while?

From arigo at tunes.org  Tue Mar 23 19:42:15 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 19:42:15 +0100
Subject: [pypy-dev] autonosiness in roundup
In-Reply-To: <20100323173832.GA10910@code0.codespeak.net>
References: <693bc9ab1003161830x2381726ay7b1189fb77530181@mail.gmail.com>
	<1afaf6161003161836u4b543766xa6c897185ebb22e9@mail.gmail.com>
	<4BA09C37.8080400@gmail.com>
	<e27efe131003170219r5e8d3e85ieed3b4e9b553dfd0@mail.gmail.com>
	<20100323114931.GA15591@code0.codespeak.net>
	<20100323162838.GU26514@trillke.net>
	<20100323173832.GA10910@code0.codespeak.net>
Message-ID: <20100323184215.GA17031@code0.codespeak.net>

Re-hi,

On Tue, Mar 23, 2010 at 06:38:32PM +0100, Armin Rigo wrote:
> Note that I did not remove the existing hacks.  Please ask me.  One
> issue is that I'm not sure how to make "pypy-issue" autonosy on the
> already-existing tracker entries -- so far I fear that it's going to
> receive only the new entries (including follow-ups to that).

That's done too now.  A number of people should be removed from the
"nosy" field as soon as there is a new posting to a existing or new
issue.  Additionally, "pypy-issue" should show up in the "nosy" field.
It goes to the new mailing list, to which I already subscribed precisely
the same list of people.  (This means that you're still getting all
mails from the tracker, but now coming from pypy-issue at codespeak.net, so
your mail filters may need some re-tweaking.)

For everybody else, please feel free to subscribe at
http://codespeak.net/mailman/listinfo/pypy-issue .


A bientot,

Arin.

From zejn at kiberpipa.org  Tue Mar 23 19:43:33 2010
From: zejn at kiberpipa.org (Gasper Zejn)
Date: Tue, 23 Mar 2010 19:43:33 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
Message-ID: <201003231943.33616.zejn@kiberpipa.org>

On Tuesday 23 March 2010 19:23:45 you wrote:
> On Tue, Mar 23, 2010 at 6:02 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
> > The links at the URL http://buildbot.pypy.org/nightly/ are failing with
> > "resource unavailable"; misconfiguration?
> >
> > Just to make sure it's a known problem.
> 
> It's working for me right now. Maybe it has just been down for a while?
> 

Nope, not working. The "directory listing" works, but not the links. Did you 
click on any of the links provided on that url?

KR,
Gasper

From arigo at tunes.org  Tue Mar 23 22:43:55 2010
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 23 Mar 2010 22:43:55 +0100
Subject: [pypy-dev] Branches
Message-ID: <20100323214355.GA30680@code0.codespeak.net>

Hi all,

Can we decide if the following branches have been merged, are useful
enough to be merged, need more work, "maybe one day", are very outdated,
or turned out just to be a plain bad idea?  The numbers are the rev
number of the last change.

Also, what about separating in-development branches that we expect to
merge or kill soon, from the branches that are for "some other idea",
e.g. "effect-analysis", which may not be merged in a long while but
still we may want to keep around?

    * 71164  abort-no-asm (fijal, cfbolz)
    * 70752  bridges-experimental (cfbolz)
    * 55732  build-external (amaury)
    * 63757  classdeco (benjamin)
    * 68723  effect-analysis (verte)
    * 55751  eval-loop-experiments (antocuni)
    * 68703  gc-arena (arigo)
    * 70970  gc-huge-list (fijal, arigo)
    * 70689  jit-profiling (pedronis, fijal)
    * 55472  judy-trees (fijal)
    * 71270  kill-can-inline (cfbolz)
    * 49688  newdotviewer (misto)
    * 64690  pypycpp (igorto)
    * 70786  separate-compilation (amaury)
    * 70024  sepcomp (xoraxax)
    * 66009  type-celldict (cfbolz)
    * 64645  unicode_filename (amaury)
    * 70890  unroll-safe-if-const-arg (fijal, cfbolz, arigo)

Interesting to see the wide range of authors :-)


A bientot,

Armin.

From sl at scrooge.dk  Tue Mar 23 22:46:50 2010
From: sl at scrooge.dk (=?ISO-8859-1?Q?S=F8ren_Laursen?=)
Date: Tue, 23 Mar 2010 22:46:50 +0100
Subject: [pypy-dev] Sandboxing pypy
Message-ID: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>

Hi,


I have been following  the pypy project for over a year. And I have been
playing around with it for some time.



The project I am working with Minimum Intrusion Grid : MiG (
http://sites.google.com/site/minimumintrusiongrid/) are looking into using
pypy.



I would like to use it for sandboxing user code in MiG, more specific allow
the users to develop their own ?scheduler? .



The MiG, it self is written in Python so we might even be able to run it on
pypy.  But right now it is the sandboxing that I am working with.



For me it is a bit unclear in the documentation/website, but for that I read
into ?An attacker that tries to escape the sandbox is stuck within a C
program that contains no external function calls at all except for writing
to stdout and reading from stdin.? means that I have to write functions that
emulates file read/write operations.



I have tried different things using the pypy_interact.py (just love the
?timeout parameter J), looked at code but have not been able to read files
or write files using the pypy_interact.py.



What have I missed?


Regards,



S?ren Laursen
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100323/a022a9bd/attachment-0001.htm 

From benjamin at python.org  Tue Mar 23 22:57:43 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 23 Mar 2010 16:57:43 -0500
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <1afaf6161003231457p7fd6eb0age0904e61f9909bba@mail.gmail.com>

2010/3/23 S?ren Laursen <sl at scrooge.dk>:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read files
> or write files using the pypy_interact.py.

That's probably because pypy_interact.py puts the sandboxed process in a VFS.

Feel free to drop by #pypy IRC anytime.


-- 
Regards,
Benjamin

From ehsanamiri at gmail.com  Wed Mar 24 00:17:07 2010
From: ehsanamiri at gmail.com (Ehsan Amiri)
Date: Tue, 23 Mar 2010 16:17:07 -0700
Subject: [pypy-dev] GSoC projects
Message-ID: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>

Hello all

I am a graduate student interested in participation in GSoC 2010. I found
some PyPy projects closely related to my background. In particular writing a
backend for Intel 64, for JIT compiler, sounds interesting. Also I would
like to know more about projects on stackless features. I would be happy to
hear any comment on these projects and pointers to documents/paper/code that
might be helpful.

Last year I participated in GSoC as well. I developed a prototype of an
architecture independent SIMD library for Python. This library provides an
interface for programmers to write SIMD code in Python and then translates
it to low level architecture specific SIMD code using CorePy. The prototype
that I developed last year supports intel 64 as I did not have enough time
to develop code generators for any other architecture.

Best
Ehsan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100323/d92fe45f/attachment.htm 

From benjamin at python.org  Wed Mar 24 02:31:50 2010
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 23 Mar 2010 20:31:50 -0500
Subject: [pypy-dev] GSoC projects
In-Reply-To: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
Message-ID: <1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>

Hi Ehsan!

2010/3/23 Ehsan Amiri <ehsanamiri at gmail.com>:
> Hello all
> I am a graduate student interested in participation in GSoC 2010. I found
> some PyPy projects closely related to my background. In particular writing a
> backend for Intel 64, for JIT compiler, sounds interesting. Also I would
> like to know more about projects on stackless features. I would be happy to
> hear any comment on these projects and pointers to documents/paper/code that
> might be helpful.

We have somebody who has voiced interest in doing a 64 bit backend,
but there's no reason you couldn't assist them or work on some other
aspect of PyPy.

Feel free to drop by the #pypy channel on Freenode anytime.


-- 
Regards,
Benjamin

From william.leslie.ttg at gmail.com  Wed Mar 24 03:00:29 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Wed, 24 Mar 2010 13:00:29 +1100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <f45df7231003231900l33beeac5q2234cbf1e26fbe19@mail.gmail.com>

On 24 March 2010 08:43, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea? ?The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?

Effect analysis can probably disappear for the moment. I've been
working on it locally, but do not have a regular internet connection.
SVN doesn't have anything useful in it anyway.

-- 
William Leslie

From arigo at tunes.org  Wed Mar 24 10:38:55 2010
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 24 Mar 2010 10:38:55 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <20100324093854.GA18762@code0.codespeak.net>

Hi,

On Tue, Mar 23, 2010 at 10:46:50PM +0100, S?ren Laursen wrote:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read files
> or write files using the pypy_interact.py.

You get indeed a VFS (virtual file system) which is read-only so far.
You can read any file from the virtual path "/tmp/xxx" if you start
pypy_interact.py with the option "--tmp=some/path".  There is no support
yet to allow writes.  It could be easily added by editing vfs.py.


A bientot,

Armin.

From Ronny.Pfannschmidt at gmx.de  Wed Mar 24 10:47:38 2010
From: Ronny.Pfannschmidt at gmx.de (Ronny Pfannschmidt)
Date: Wed, 24 Mar 2010 10:47:38 +0100
Subject: [pypy-dev] dvcs conversation of pypy and maybe all the other
 projects in the repo, too
Message-ID: <1269424058.16631.83.camel@localhost>

Hi,

since pypy's history itself is kinda unsuitable for practically all
converters/importers out there, i started to experiment with analysis of
it in order to eventually convert to a dag based dvcs (most likely hg).

Since that repository carries various other projects as well i wonder if
anyone else is interested in converting to a dvcs.

The needs of pypy's  history alone create the need of a very powerfull
tool anyway, so converting the other projects would be a nice
sideeffect.

-- Ronny



From anto.cuni at gmail.com  Wed Mar 24 10:47:37 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 24 Mar 2010 10:47:37 +0100
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <201003231943.33616.zejn@kiberpipa.org>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com>
	<201003231943.33616.zejn@kiberpipa.org>
Message-ID: <58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>

On Tue, Mar 23, 2010 at 7:43 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:

> Nope, not working. The "directory listing" works, but not the links. Did you
> click on any of the links provided on that url?

Ah no, sorry. My fault, the links indeed don't work.

ciao,
Anto

From anto.cuni at gmail.com  Wed Mar 24 10:51:15 2010
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Wed, 24 Mar 2010 10:51:15 +0100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <58e316a41003240251q6f56d768t4398fcc5d6052d6b@mail.gmail.com>

On Tue, Mar 23, 2010 at 10:43 PM, Armin Rigo <arigo at tunes.org> wrote:

> ? ?* 55751 ?eval-loop-experiments (antocuni)

"maybe one day"

In this branch, I made some changes to the main eval loop and in some
cases I observed speedups up to 20% (which are less relevant now that
we have a jit, but still), but I never managed to reproduce them
consistently.  I'd like to keep it around to investigate if the
changes are worth of or not.

ciao,
Anto

From victor.stinner at haypocalc.com  Wed Mar 24 11:01:56 2010
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 24 Mar 2010 11:01:56 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
Message-ID: <201003241101.56211.victor.stinner@haypocalc.com>

Le mardi 23 mars 2010 22:46:50, S?ren Laursen a ?crit :
> The project I am working with Minimum Intrusion Grid : MiG (
> http://sites.google.com/site/minimumintrusiongrid/) are looking into using
> pypy.
> 
> I would like to use it for sandboxing user code in MiG, more specific allow
> the users to develop their own ?scheduler? .
> 
> The MiG, it self is written in Python so we might even be able to run it on
> pypy.  But right now it is the sandboxing that I am working with.

FYI I wrote a new sandbox project for CPython:

   http://github.com/haypo/pysandbox/

It's currently very specific to CPython: it uses evil tricks to create a read 
only view of the __builtins__ super global dictionary.

It's completly different to the PyPy sandbox: if you escape from the sandbox, 
you get a full access to all Python functions.

A long description: 
http://mail.python.org/pipermail/python-dev/2010-February/097701.html

-- 
Victor Stinner
http://www.haypocalc.com/

From cfbolz at gmx.de  Wed Mar 24 14:40:50 2010
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Wed, 24 Mar 2010 14:40:50 +0100
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <4BAA1662.4090309@gmx.de>

Hi Armin,

On 03/23/2010 10:43 PM, Armin Rigo wrote:
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea?  The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?

Maybe just have a dir parallel to "branch" called "inactive"? or some 
other name.

>      * 71164  abort-no-asm (fijal, cfbolz)

Keep around, I want to reconsider this eventually.

>      * 70752  bridges-experimental (cfbolz)

Same.

>      * 71270  kill-can-inline (cfbolz)

Same.

>      * 66009  type-celldict (cfbolz)

Killed this one.

>      * 70890  unroll-safe-if-const-arg (fijal, cfbolz, arigo)

Would keep this around, even if it ultimately didn't work.

Cheers,

Carl Friedrich

From fijall at gmail.com  Thu Mar 25 04:44:29 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 24 Mar 2010 21:44:29 -0600
Subject: [pypy-dev] Nightly builds are unavailable
In-Reply-To: <58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>
References: <201003231803.00045.zejn@kiberpipa.org>
	<58e316a41003231123x9d6ce03v765c2d2b55caf772@mail.gmail.com> 
	<201003231943.33616.zejn@kiberpipa.org>
	<58e316a41003240247u58ca76bfy19da2dde635b0e6f@mail.gmail.com>
Message-ID: <693bc9ab1003242044w4873c7eavbcab2fe2f1d61c4f@mail.gmail.com>

That's my fault, I'll fix it tomorrow (fighting with twisted.web + holiday)

On Wed, Mar 24, 2010 at 3:47 AM, Antonio Cuni <anto.cuni at gmail.com> wrote:
> On Tue, Mar 23, 2010 at 7:43 PM, Gasper Zejn <zejn at kiberpipa.org> wrote:
>
>> Nope, not working. The "directory listing" works, but not the links. Did you
>> click on any of the links provided on that url?
>
> Ah no, sorry. My fault, the links indeed don't work.
>
> ciao,
> Anto
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From fijall at gmail.com  Thu Mar 25 04:51:38 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 24 Mar 2010 21:51:38 -0600
Subject: [pypy-dev] Branches
In-Reply-To: <20100323214355.GA30680@code0.codespeak.net>
References: <20100323214355.GA30680@code0.codespeak.net>
Message-ID: <693bc9ab1003242051s1f511605y4fc7cb18e90d64ce@mail.gmail.com>

On Tue, Mar 23, 2010 at 3:43 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi all,
>
> Can we decide if the following branches have been merged, are useful
> enough to be merged, need more work, "maybe one day", are very outdated,
> or turned out just to be a plain bad idea? ?The numbers are the rev
> number of the last change.
>
> Also, what about separating in-development branches that we expect to
> merge or kill soon, from the branches that are for "some other idea",
> e.g. "effect-analysis", which may not be merged in a long while but
> still we may want to keep around?
>
> ? ?* 70689 ?jit-profiling (pedronis, fijal)

Killed

> ? ?* 55472 ?judy-trees (fijal)

Killed

> ? ?* 64690 ?pypycpp (igorto)

I suppose that's to-be-killed

> ? ?* 70890 ?unroll-safe-if-const-arg (fijal, cfbolz, arigo)

I would like this to be kept around until jit is powerful enough to handle that.

From kai.aras.010 at googlemail.com  Thu Mar 25 05:00:36 2010
From: kai.aras.010 at googlemail.com (kai.aras)
Date: Thu, 25 Mar 2010 05:00:36 +0100
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
Message-ID: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>

Hi Everyone,

My name is Kai Aras, I'm a Computer-Science student from Germany and  
I'm very interested in doing my Bachelor's Thesis on PyPy and possibly  
also combine that with a GSoC project.

I became aware of the PyPy-Project by the Chaosradio Express (CRE088)  
episode about PyPy from 2008.
Last year, I had the opportunity to take some time to play around with  
PyPy in order to present it at my University, ever since that I became  
more and more attracted to the Python Language and the PyPy-Project  
itself, so writing my Bachelor's Thesis about something PyPy-related  
occured to me quite early.
Now, as the time has come, combining my Thesis with a GSoC Project  
would be a great opportunity.

During the last year, I've worked on a smarthome project using CPython  
on embedded-linux platforms,
as the project grew bigger, more and more problems arised and the need  
for some spezialized Python Runtime grew bigger as well.

I know that targeting Embedded Platforms was an Issue in the original  
EU-Project and there was that Maemo thing last year, so I'm guessing  
this could be a general topic for a GSoC Project as well as a  
Bachelor's Thesis, what do you guys think ? Is this something you  
would like to see as well ?
Personally, I'd love to use PyPy on openWRT, could this might be a  
target of general interest ?

This is just something I thought about from time to time but never  
really had the time to experiment with, so I'm open to all  
suggestions, please let me know if you have any ideas that could be  
suitable for such a project.

Thanks,
Kai

From arigo at tunes.org  Thu Mar 25 12:52:58 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 12:52:58 +0100
Subject: [pypy-dev] GSoC projects
In-Reply-To: <1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
	<1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
Message-ID: <20100325115258.GA10602@code0.codespeak.net>

Hi Benjamin, hi all,

On Tue, Mar 23, 2010 at 08:31:50PM -0500, Benjamin Peterson wrote:
> We have somebody who has voiced interest in doing a 64 bit backend,
> but there's no reason you couldn't assist them or work on some other
> aspect of PyPy.

I should probably mention here what I only explained in private mails so
far.  The 32-bit backend is in pypy/jit/backend/x86/.  However that
directory uses ri386.py and ri386setup.py, which are old and very custom
hacks based on multimethods in order to encode machine code
instructions.  I already started some work on replacing these two files.
It's done in this branch:

    http://codespeak.net/svn/pypy/branch/remove-ri386-multimethod-2

There, we have a single file rx86.py that is simpler, and supports both
32-bit and 64-bit instruction generation.  But the interface is very
different, so that means the whole rest of the backend has to be
adapted.  That would be the main work of a SoC student.

To get started, look at rx86.py and its test files test/test_rx86*.py
and try to write small examples that uses a subclass of
X86_32_CodeBuilder or X86_64_CodeBuilder in order to build and call a
test function.  The 64-bit version is more complicated, because the ABI
on 64-bit is to use some number of registers in order to pass arguments;
see details e.g. in http://www.x86-64.org/documentation/abi.pdf section
"Function Calling Sequence".


A bientot,

Armin.

From arigo at tunes.org  Thu Mar 25 13:04:34 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 13:04:34 +0100
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
Message-ID: <20100325120434.GB10602@code0.codespeak.net>

Hi Kai,

On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
> I know that targeting Embedded Platforms was an Issue in the original  
> EU-Project and there was that Maemo thing last year, so I'm guessing  
> this could be a general topic for a GSoC Project as well as a  
> Bachelor's Thesis, what do you guys think ? Is this something you  
> would like to see as well ?

I am afraid there is a gap (larger than ever) between the small amount
of "core people" and the large amount of directions we could explore.
Thank you for your interest; by no mean I intend to disappoint you, but
there is just no-one working on the general topic of embedding PyPy any
longer.  So it is very unlikely that you can get a SoC project on it.


A bientot,

Armin.

From arigo at tunes.org  Thu Mar 25 13:09:32 2010
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 25 Mar 2010 13:09:32 +0100
Subject: [pypy-dev] Python Ireland Conference
In-Reply-To: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>
References: <50a522ca1003190709k5f26aa67s31f0502c6791dd70@mail.gmail.com>
Message-ID: <20100325120932.GC10602@code0.codespeak.net>

Hi Michael,

On Fri, Mar 19, 2010 at 02:09:27PM +0000, Michael Twomey wrote:
> I'm one of the folks involved in organising the first python Ireland
> conference, to be held in Dublin on Saturday 17th of July (with
> sprints on Sunday).
> 
> We'd love it if someone from PyPy could make it over to give a talk.
> The audience is your typical spread of python developers, many of whom
> haven't been exposed to PyPy and what it can do. An introduction to
> PyPy and the benefits it offers would go down very well.

Thanks for the invitation.  I think that the general "no-answer-coming-
so-far" situation means that we don't really know.  I would be
interested in coming myself, but I cannot give any more precise answer
right now.  Is there a deadline for you to get a more definite yes/no?


A bientot,

Armin.

From fijall at gmail.com  Thu Mar 25 15:34:55 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 08:34:55 -0600
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <20100325120434.GB10602@code0.codespeak.net>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com> 
	<20100325120434.GB10602@code0.codespeak.net>
Message-ID: <693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>

Hello.

On Thu, Mar 25, 2010 at 6:04 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Kai,
>
> On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
>> I know that targeting Embedded Platforms was an Issue in the original
>> EU-Project and there was that Maemo thing last year, so I'm guessing
>> this could be a general topic for a GSoC Project as well as a
>> Bachelor's Thesis, what do you guys think ? Is this something you
>> would like to see as well ?
>
> I am afraid there is a gap (larger than ever) between the small amount
> of "core people" and the large amount of directions we could explore.
> Thank you for your interest; by no mean I intend to disappoint you, but
> there is just no-one working on the general topic of embedding PyPy any
> longer. ?So it is very unlikely that you can get a SoC project on it.
>
>

I think I would like to expand on that. In the past we had a bit of
problems with people doing work for SoC and then abandoning a working
prototype. Since it was out of interest of a core group, we just
killed the code or moved it to some obscure place at some later point.
In a way, we would like to encourage people to do some work which is
more of a core interest (at least for SoC) and if in doubt, we would
choose such people.

That said, personally I would encourage you to investigate this
direction for your bachelor if you feel like it. For example the idea
of having a JIT on embedded devices is for example a very researchy
area, since you need memory for the JIT (which is not free on embedded
devices).

Cheers,
fijal

From tobami at googlemail.com  Thu Mar 25 20:29:17 2010
From: tobami at googlemail.com (Miquel Torres)
Date: Thu, 25 Mar 2010 20:29:17 +0100
Subject: [pypy-dev] Quick "state of performance" analysis
Message-ID: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>

Hi all!

I have implemented a small new feature for the speed center. In the
Overview, you are now able to select to which results you wish to compare
the performance of the selected interpreter. That allows a couple of
interesting investigations to be made (bear with me if I say obvious
things):

1 - http://speed.pypy.org/overview/?interpreter=3)
We already knew that there are 5 benchmarks where pypy-c without the jit is
more than twice times slower than cpython: slowspitfire, twisted_iteration,
twisted_tcp, html5lib and worst of all spambayes.

2 - http://speed.pypy.org/overview/
The jit makes up for some of them, only twisted_tcp, slowspitfire and
spambayes remain as cases where cpython is much faster.

3 - http://speed.pypy.org/overview/?baseline=4
We can now compare pypy-c-jit with cpython with psyco acceleration. The
first thing that stands out, is that pypy is much slower... exactly in the
same cases where pypy-c without the jit is slower than cpython (no
surprise). The second one, is that apart from those 5 cases where the jit is
held back by bad pypy-c baseline performance, pypy-c-jit is already much
better than psyco, about twice as fast!

4 - http://speed.pypy.org/overview/?baseline=3
Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can
confirm that the jit accelerates *everything* that we currenly measure, and
most by a very good factor. The 3 slow benchmarks from comparison 2 are here
last, the are accelerated the least by the jit.

So to me the conclusion is that the jit has good enough performance for a
first version, and that only poor pypy-c performance in some cases is
holding the pypy project from becoming a better option than cpython.
Wouldn't it be a good aim to make it a priority of the next (1.3?) release
to improve on that? ;-)

Apart from it making pypy more "ready", there is a problem if that is not
addressed while further jit development continues. An application is only so
fast as its slowest part, so even if the jit accelerates some cases
100-fold, it won't help many applications.

I plan to add a new view with graphs that will show exactly what the problem
is.

Cheers!
Miquel


PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see
what improvements there have been since then
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100325/56cd8869/attachment.htm 

From santagada at gmail.com  Thu Mar 25 20:55:18 2010
From: santagada at gmail.com (Leonardo Santagada)
Date: Thu, 25 Mar 2010 16:55:18 -0300
Subject: [pypy-dev] Quick "state of performance" analysis
In-Reply-To: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>
References: <c561346d1003251229i7b4c2fabx99c0e4f82588da9d@mail.gmail.com>
Message-ID: <F0D2478E-CE86-4083-8653-70A1603B12CA@gmail.com>


On Mar 25, 2010, at 4:29 PM, Miquel Torres wrote:

> Hi all!
> 
> I have implemented a small new feature for the speed center. In the Overview, you are now able to select to which results you wish to compare the performance of the selected interpreter. That allows a couple of interesting investigations to be made (bear with me if I say obvious things):
> 
> 1 - http://speed.pypy.org/overview/?interpreter=3)
> We already knew that there are 5 benchmarks where pypy-c without the jit is more than twice times slower than cpython: slowspitfire, twisted_iteration, twisted_tcp, html5lib and worst of all spambayes.
> 
> 2 - http://speed.pypy.org/overview/
> The jit makes up for some of them, only twisted_tcp, slowspitfire and spambayes remain as cases where cpython is much faster.
> 
> 3 - http://speed.pypy.org/overview/?baseline=4
> We can now compare pypy-c-jit with cpython with psyco acceleration. The first thing that stands out, is that pypy is much slower... exactly in the same cases where pypy-c without the jit is slower than cpython (no surprise). The second one, is that apart from those 5 cases where the jit is held back by bad pypy-c baseline performance, pypy-c-jit is already much better than psyco, about twice as fast!

Pretty cool, but is python with psyco using psyco.full or just jitting some parts? maybe there is a lot of room for manually tunning psyco (but there is no way to do this for pypy afaik).


> 4 - http://speed.pypy.org/overview/?baseline=3
> Another new comparison is pypy-c-jit to pypy-c (revision 72012). Here we can confirm that the jit accelerates *everything* that we currenly measure, and most by a very good factor. The 3 slow benchmarks from comparison 2 are here last, the are accelerated the least by the jit.
> 
> So to me the conclusion is that the jit has good enough performance for a first version, and that only poor pypy-c performance in some cases is holding the pypy project from becoming a better option than cpython. Wouldn't it be a good aim to make it a priority of the next (1.3?) release to improve on that? ;-)
> 
> Apart from it making pypy more "ready", there is a problem if that is not addressed while further jit development continues. An application is only so fast as its slowest part, so even if the jit accelerates some cases 100-fold, it won't help many applications.
> 
> I plan to add a new view with graphs that will show exactly what the problem is.
> 
> Cheers!
> Miquel
> 
> 
> PD: you can also for example compare pypy-c-jit to pypy-c-jit 72012 and see what improvements there have been since then
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev

--
Leonardo Santagada
santagada at gmail.com



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100325/b43e4d59/attachment.htm 

From william.leslie.ttg at gmail.com  Fri Mar 26 00:57:22 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Fri, 26 Mar 2010 10:57:22 +1100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <201003241101.56211.victor.stinner@haypocalc.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
	<201003241101.56211.victor.stinner@haypocalc.com>
Message-ID: <f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>

On 24 March 2010 21:01, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> FYI I wrote a new sandbox project for CPython:
>
> ? http://github.com/haypo/pysandbox/
>
> It's currently very specific to CPython: it uses evil tricks to create a read
> only view of the __builtins__ super global dictionary.
>
> It's completly different to the PyPy sandbox: if you escape from the sandbox,
> you get a full access to all Python functions.
>
> A long description:
> http://mail.python.org/pipermail/python-dev/2010-February/097701.html

I didn't dive too deeply into the source, but what is to stop one from asking:

[o for o in (1).__class__.__bases__[0].__subclasses__() if o.__name__
== 'file'][0]('/etc/passwd').read()

?

-- 
William Leslie

From william.leslie.ttg at gmail.com  Fri Mar 26 01:06:26 2010
From: william.leslie.ttg at gmail.com (William Leslie)
Date: Fri, 26 Mar 2010 11:06:26 +1100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com>
	<201003241101.56211.victor.stinner@haypocalc.com>
	<f45df7231003251657k75fff75dta3cda9e4cfe6a8dc@mail.gmail.com>
Message-ID: <f45df7231003251706y1e700975h89f9723ce447a896@mail.gmail.com>

On 26 March 2010 10:57, William Leslie <william.leslie.ttg at gmail.com> wrote:
>
> I didn't dive too deeply into the source, but what is to stop one from asking:
>
> [o for o in (1).__class__.__bases__[0].__subclasses__() if o.__name__
> == 'file'][0]('/etc/passwd').read()
>
> ?

Ah right, attributes.py.

>
> --
> William Leslie
>



-- 
William Leslie

From fijall at gmail.com  Fri Mar 26 02:05:21 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 19:05:21 -0600
Subject: [pypy-dev] various benchmark directories
Message-ID: <693bc9ab1003251805t389291e2le159c29ec6dd7f2@mail.gmail.com>

We have a couple of strange benchmark directories scattered a bit all
over the place in PyPy:

pypy/rpython/microbench
pypy/translator/goal/various
pypy/interpreter/callbench
pypy/translator/benchmark
build/benchmem
benchmarks

comes to mind

Do we need all of them? Can we kill some and factor the rest to be in one place?

From fijall at gmail.com  Fri Mar 26 06:44:31 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 25 Mar 2010 23:44:31 -0600
Subject: [pypy-dev] Wikipedia pypy article
Message-ID: <693bc9ab1003252244v2bf6a3er6153199d4b77f977@mail.gmail.com>

http://en.wikipedia.org/wiki/PyPy

It's somewhat out of date, but also extremely confusing with notions
of meta-circular beast. Does someone feel like cleaning it up a bit?
Especially keeping it up to date would be cool.

From sl at scrooge.dk  Fri Mar 26 14:29:19 2010
From: sl at scrooge.dk (=?ISO-8859-1?Q?S=F8ren_Laursen?=)
Date: Fri, 26 Mar 2010 14:29:19 +0100
Subject: [pypy-dev] Sandboxing pypy
In-Reply-To: <20100324093854.GA18762@code0.codespeak.net>
References: <ed932062c4806901e1c4caaccf6abd0f@mail.gmail.com> 
	<20100324093854.GA18762@code0.codespeak.net>
Message-ID: <9e7424864518d7dcc66c8b4ba2cf89da@mail.gmail.com>

Thanks for all the replies.

I have started to look at vfs.py, I will join IRC later today.

Regards,

S?ren

-----Oprindelig meddelelse-----
Fra: Armin Rigo [mailto:arigo at tunes.org]
Sendt: 24. marts 2010 10:39
Til: S?ren Laursen
Cc: pypy-dev at codespeak.net
Emne: Re: [pypy-dev] Sandboxing pypy

Hi,

On Tue, Mar 23, 2010 at 10:46:50PM +0100, S?ren Laursen wrote:
> I have tried different things using the pypy_interact.py (just love the
> ?timeout parameter J), looked at code but have not been able to read
files
> or write files using the pypy_interact.py.

You get indeed a VFS (virtual file system) which is read-only so far.
You can read any file from the virtual path "/tmp/xxx" if you start
pypy_interact.py with the option "--tmp=some/path".  There is no support
yet to allow writes.  It could be easily added by editing vfs.py.


A bientot,

Armin.

From arigo at tunes.org  Sat Mar 27 11:04:57 2010
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 27 Mar 2010 11:04:57 +0100
Subject: [pypy-dev] GSoC projects
In-Reply-To: <20100325115258.GA10602@code0.codespeak.net>
References: <fe13bba81003231617h46f6443dya1f47184fe9b0a33@mail.gmail.com>
	<1afaf6161003231831x70fa6394pacd874434f290627@mail.gmail.com>
	<20100325115258.GA10602@code0.codespeak.net>
Message-ID: <20100327100457.GA27151@code0.codespeak.net>

Re-hi,

On Thu, Mar 25, 2010 at 12:52:58PM +0100, Armin Rigo wrote:
> The 64-bit version is more complicated, because the ABI
> on 64-bit is to use some number of registers in order to pass arguments;
> see details e.g. in http://www.x86-64.org/documentation/abi.pdf section
> "Function Calling Sequence".

I should point out that we only need to pass integers (of at most
64-bit), pointers, and 64-bit floats (that's "double", in C) as
arguments to functions.  Don't go discouraged because of the messy rules
for passing structs, arrays, or 128- or 256-bit stuff.  :-)


A bientot,

Armin.

From kai.aras.010 at googlemail.com  Tue Mar 30 01:11:24 2010
From: kai.aras.010 at googlemail.com (kai.aras)
Date: Tue, 30 Mar 2010 01:11:24 +0200
Subject: [pypy-dev] GSoC Project/Bachelor's Thesis
In-Reply-To: <693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>
References: <3C1318CD-C3E3-421D-8EE0-C724CA98018B@googlemail.com>
	<20100325120434.GB10602@code0.codespeak.net>
	<693bc9ab1003250734s5b628a67h73d78abb69d2d343@mail.gmail.com>
Message-ID: <4FB67D66-EA50-4786-9D18-3FC9507A8163@googlemail.com>

Hi Armin, Hi Maciej,

first of all, thanks for your feedback and sorry for the late reply.

On Mar 25, 2010, at 3:34 PM, Maciej Fijalkowski wrote:

> Hello.
>
> On Thu, Mar 25, 2010 at 6:04 AM, Armin Rigo <arigo at tunes.org> wrote:
>> Hi Kai,
>>
>> On Thu, Mar 25, 2010 at 05:00:36AM +0100, kai.aras wrote:
>>> I know that targeting Embedded Platforms was an Issue in the  
>>> original
>>> EU-Project and there was that Maemo thing last year, so I'm guessing
>>> this could be a general topic for a GSoC Project as well as a
>>> Bachelor's Thesis, what do you guys think ? Is this something you
>>> would like to see as well ?
>>
>> I am afraid there is a gap (larger than ever) between the small  
>> amount
>> of "core people" and the large amount of directions we could explore.
>> Thank you for your interest; by no mean I intend to disappoint you,  
>> but
>> there is just no-one working on the general topic of embedding PyPy  
>> any
>> longer.  So it is very unlikely that you can get a SoC project on it.

I totally get that, no problem. It looks like I wont be able to make  
it in time for a SoC anyway.
However, if you don't think this is a complete dead-end, I'd still  
like to experiment with that for my Bachelor's Thesis,
otherwise please let me know.


> I think I would like to expand on that. In the past we had a bit of
> problems with people doing work for SoC and then abandoning a working
> prototype. Since it was out of interest of a core group, we just
> killed the code or moved it to some obscure place at some later point.
> In a way, we would like to encourage people to do some work which is
> more of a core interest (at least for SoC) and if in doubt, we would
> choose such people.
>
> That said, personally I would encourage you to investigate this
> direction for your bachelor if you feel like it. For example the idea
> of having a JIT on embedded devices is for example a very researchy
> area, since you need memory for the JIT (which is not free on embedded
> devices).
>
> Cheers,
> fijal

Looking into the JIT sounds quite interesting to me too, thanks for  
bringing that up.
I don't have a huge background in this field, so I  wouldn't really  
know how to get started on this,
maybe you could point me in the right direction here, any help would  
be appreciated.

Thanks,
Kai




From jcreigh at gmail.com  Tue Mar 30 05:12:47 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Mon, 29 Mar 2010 21:12:47 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
Message-ID: <4BB16C2F.5050300@gmail.com>

Hi Guys,

I am a student planning on applying to GSoC to create an x86-64 backend 
for the JIT. I have a (very) rough draft of my proposal, and I was 
hoping to get some feedback on it.

Specifically, I'm wondering about operating system support. I've written 
the proposal as if I would support Linux/Mac OS X/Windows.

I would be developing on Linux, so I think we can assume that would be 
fairly well supported, but obviously I would like it to work on Mac OS X 
and Windows as well. (I'm not sure if I would have the time/motivation 
to care about obscure BSDs). OTOH, if there are already a lot of 
outstanding issues on one of those platforms, I don't know that I would 
be able to get it working. So what do you think would be a reasonable 
goal here?

Secondly, my timeline is pretty vague. The PSF proposal template 
recommends a week-by-week timeline, but honestly, I'm not sure how the 
time usage would break down. Any comments on that would be greatly 
appreciated.

Here's the draft:

=== Proposal ===

The PyPy JIT, which has shown substantial performance improvements over
CPython, often several times faster, does not currently support the
x86-64 instruction set, making it impractical to use on 64-bit x86
systems.

My proposal is to extend the existing x86 JIT backend to support x86-64
as well.

=== Deliverables ===

Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
into PyPy trunk.

=== Implementation plan ===

This is not a research proposal. The goal is simply to have a PyPy JIT
that works out of the box on 64-bit CPUs, implemented as conservatively
as possible.

As such, I will attempt to reuse as much of the existing x86 backend
that I can. In fact, the architectural similarities between x86 and
x86-64 are large enough that I hope to implement a unified x86/x86-64
backend with the majority of the code working for either platform.

There is an existing branch that, while very incomplete, has the
beginnings of a unified x86/x86-64 instruction encoding module. I intend
to use that branch as a starting point.

Rough outline:

1. Take the existing "remove-ri386-multimethod-2" branch and use it as a 
basis for instruction encoding.

2. Port the existing 32-bit backend to use the new instruction encoding 
scheme.

3. Add 64-bit support to the backend,
         A) Modify register allocator to use new general purpose and
         floating point registers.
         B) Port "ResOperation" operations to 64-bit
         C) Port guard failure handling to 64-bit

4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.

=== About Me ===

I am a first-year Computer Science student at Flathead Valley Community
College planning to transfer to Montana State University.

I have several years of professional development experience. I am
comfortable programming in Python, C and x86 assembly.

Starting May 17th, I will be able to commit 40 hours/week to the project
until the end of August. I may travel for a few weeks at some point in
the summer, but I will have a laptop with me with the expectation of
continuing full-time work.

=== Contact information ===

Email: jcreigh at gmail.com
IRC:   "jcreigh" on Freenode
Phone: (will be given on request, but not preferred)

From fijall at gmail.com  Tue Mar 30 17:14:53 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 30 Mar 2010 09:14:53 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <4BB16C2F.5050300@gmail.com>
References: <4BB16C2F.5050300@gmail.com>
Message-ID: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>

On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh at gmail.com> wrote:
> Hi Guys,
>
> I am a student planning on applying to GSoC to create an x86-64 backend
> for the JIT. I have a (very) rough draft of my proposal, and I was
> hoping to get some feedback on it.
>
> Specifically, I'm wondering about operating system support. I've written
> the proposal as if I would support Linux/Mac OS X/Windows.

That would be great to have, but I would be happy enough to have it on
linux. Let's say that if pypy continues to work on os x and windows
you'll fix remaining 64bit JIT bugs.

>
> I would be developing on Linux, so I think we can assume that would be
> fairly well supported, but obviously I would like it to work on Mac OS X
> and Windows as well. (I'm not sure if I would have the time/motivation
> to care about obscure BSDs). OTOH, if there are already a lot of
> outstanding issues on one of those platforms, I don't know that I would
> be able to get it working. So what do you think would be a reasonable
> goal here?

It's not like pypy works on anything else than linux os x and windows.

>
> Secondly, my timeline is pretty vague. The PSF proposal template
> recommends a week-by-week timeline, but honestly, I'm not sure how the
> time usage would break down. Any comments on that would be greatly
> appreciated.

I think week based planning is ridiculous. That may be done for some
simpler stuff, but not for this. I would say have some milestones, but
avoid precise timing.

>
> Here's the draft:
>
> === Proposal ===
>
> The PyPy JIT, which has shown substantial performance improvements over
> CPython, often several times faster, does not currently support the
> x86-64 instruction set, making it impractical to use on 64-bit x86
> systems.

and over unladen swallow.

>
> My proposal is to extend the existing x86 JIT backend to support x86-64
> as well.
>
> === Deliverables ===
>
> Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> into PyPy trunk.
>
> === Implementation plan ===
>
> This is not a research proposal. The goal is simply to have a PyPy JIT
> that works out of the box on 64-bit CPUs, implemented as conservatively
> as possible.
>
> As such, I will attempt to reuse as much of the existing x86 backend
> that I can. In fact, the architectural similarities between x86 and
> x86-64 are large enough that I hope to implement a unified x86/x86-64
> backend with the majority of the code working for either platform.
>
> There is an existing branch that, while very incomplete, has the
> beginnings of a unified x86/x86-64 instruction encoding module. I intend
> to use that branch as a starting point.
>
> Rough outline:
>
> 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> basis for instruction encoding.
>
> 2. Port the existing 32-bit backend to use the new instruction encoding
> scheme.
>
> 3. Add 64-bit support to the backend,
> ? ? ? ? A) Modify register allocator to use new general purpose and
> ? ? ? ? floating point registers.
> ? ? ? ? B) Port "ResOperation" operations to 64-bit
> ? ? ? ? C) Port guard failure handling to 64-bit
>
> 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.

There is also part about asmgcc.

>
> === About Me ===
>
> I am a first-year Computer Science student at Flathead Valley Community
> College planning to transfer to Montana State University.
>
> I have several years of professional development experience. I am
> comfortable programming in Python, C and x86 assembly.
>
> Starting May 17th, I will be able to commit 40 hours/week to the project
> until the end of August. I may travel for a few weeks at some point in
> the summer, but I will have a laptop with me with the expectation of
> continuing full-time work.
>
> === Contact information ===
>
> Email: jcreigh at gmail.com
> IRC: ? "jcreigh" on Freenode
> Phone: (will be given on request, but not preferred)
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

Proposal is rather short, but I don't see how to expand it. You might
want to add that in case you finish this stuff earlier you would do
this and that (or just general JIT work or not do anything and drink
beer).

From glavoie at gmail.com  Tue Mar 30 17:40:28 2010
From: glavoie at gmail.com (Gabriel Lavoie)
Date: Tue, 30 Mar 2010 11:40:28 -0400
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
Message-ID: <d05df8621003300840t69137b72v980eb0f8ee216ece@mail.gmail.com>

Starting from May 1 I leave my current job and I'm returning full time at
university to work on my MSc project with PyPy. A FreeBSD 7.3 x86_64
buildbot from me is already available for the PyPy project. I also work
under MacOS X (Snow Leopard) for my MSc project. I could probably help you
for the particularities of these two platforms.

Keep me updated!

Gabriel Lavoie

2010/3/30 Maciej Fijalkowski <fijall at gmail.com>

> On Mon, Mar 29, 2010 at 9:12 PM, Jason Creighton <jcreigh at gmail.com>
> wrote:
> > Hi Guys,
> >
> > I am a student planning on applying to GSoC to create an x86-64 backend
> > for the JIT. I have a (very) rough draft of my proposal, and I was
> > hoping to get some feedback on it.
> >
> > Specifically, I'm wondering about operating system support. I've written
> > the proposal as if I would support Linux/Mac OS X/Windows.
>
> That would be great to have, but I would be happy enough to have it on
> linux. Let's say that if pypy continues to work on os x and windows
> you'll fix remaining 64bit JIT bugs.
>
> >
> > I would be developing on Linux, so I think we can assume that would be
> > fairly well supported, but obviously I would like it to work on Mac OS X
> > and Windows as well. (I'm not sure if I would have the time/motivation
> > to care about obscure BSDs). OTOH, if there are already a lot of
> > outstanding issues on one of those platforms, I don't know that I would
> > be able to get it working. So what do you think would be a reasonable
> > goal here?
>
> It's not like pypy works on anything else than linux os x and windows.
>
> >
> > Secondly, my timeline is pretty vague. The PSF proposal template
> > recommends a week-by-week timeline, but honestly, I'm not sure how the
> > time usage would break down. Any comments on that would be greatly
> > appreciated.
>
> I think week based planning is ridiculous. That may be done for some
> simpler stuff, but not for this. I would say have some milestones, but
> avoid precise timing.
>
> >
> > Here's the draft:
> >
> > === Proposal ===
> >
> > The PyPy JIT, which has shown substantial performance improvements over
> > CPython, often several times faster, does not currently support the
> > x86-64 instruction set, making it impractical to use on 64-bit x86
> > systems.
>
> and over unladen swallow.
>
> >
> > My proposal is to extend the existing x86 JIT backend to support x86-64
> > as well.
> >
> > === Deliverables ===
> >
> > Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> > into PyPy trunk.
> >
> > === Implementation plan ===
> >
> > This is not a research proposal. The goal is simply to have a PyPy JIT
> > that works out of the box on 64-bit CPUs, implemented as conservatively
> > as possible.
> >
> > As such, I will attempt to reuse as much of the existing x86 backend
> > that I can. In fact, the architectural similarities between x86 and
> > x86-64 are large enough that I hope to implement a unified x86/x86-64
> > backend with the majority of the code working for either platform.
> >
> > There is an existing branch that, while very incomplete, has the
> > beginnings of a unified x86/x86-64 instruction encoding module. I intend
> > to use that branch as a starting point.
> >
> > Rough outline:
> >
> > 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> > basis for instruction encoding.
> >
> > 2. Port the existing 32-bit backend to use the new instruction encoding
> > scheme.
> >
> > 3. Add 64-bit support to the backend,
> >         A) Modify register allocator to use new general purpose and
> >         floating point registers.
> >         B) Port "ResOperation" operations to 64-bit
> >         C) Port guard failure handling to 64-bit
> >
> > 4. Test 64-bit on Mac OS X and Windows and fix inevitable issues.
>
> There is also part about asmgcc.
>
> >
> > === About Me ===
> >
> > I am a first-year Computer Science student at Flathead Valley Community
> > College planning to transfer to Montana State University.
> >
> > I have several years of professional development experience. I am
> > comfortable programming in Python, C and x86 assembly.
> >
> > Starting May 17th, I will be able to commit 40 hours/week to the project
> > until the end of August. I may travel for a few weeks at some point in
> > the summer, but I will have a laptop with me with the expectation of
> > continuing full-time work.
> >
> > === Contact information ===
> >
> > Email: jcreigh at gmail.com
> > IRC:   "jcreigh" on Freenode
> > Phone: (will be given on request, but not preferred)
> > _______________________________________________
> > pypy-dev at codespeak.net
> > http://codespeak.net/mailman/listinfo/pypy-dev
> >
>
> Proposal is rather short, but I don't see how to expand it. You might
> want to add that in case you finish this stuff earlier you would do
> this and that (or just general JIT work or not do anything and drink
> beer).
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>



-- 
Gabriel Lavoie
glavoie at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20100330/6305efe4/attachment-0001.htm 

From jcreigh at gmail.com  Wed Mar 31 00:18:03 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Tue, 30 Mar 2010 16:18:03 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com>
Message-ID: <4BB2789B.70406@gmail.com>

I agree with fijal that it seems short, but I don't know what else to 
put in.

Updated draft:

=== Proposal ===

The PyPy JIT, which has shown substantial performance improvements over
other implementations of Python, including CPython and Unladen Swallow,
does not currently support the x86-64 instruction set, making it
impractical to use on 64-bit x86 systems.

My proposal is to extend the existing x86 JIT backend to support x86-64
as well.

=== Deliverables ===

Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
into PyPy trunk.

=== Implementation plan ===

This is not a research proposal. The goal is simply to have a PyPy JIT
that works out of the box on 64-bit CPUs, implemented as conservatively
as possible.

As such, I will attempt to reuse as much of the existing x86 backend
that I can. In fact, the architectural similarities between x86 and
x86-64 are large enough that I hope to implement a unified x86/x86-64
backend with the majority of the code working for either platform.

There is an existing branch that, while very incomplete, has the
beginnings of a unified x86/x86-64 instruction encoding module, which
can encode instructions for either x86 or x86-64 in a fairly seamless
manner. I intend to use that branch as a starting point.

Milestones:

1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
basis for instruction encoding. Modify the existing 32-bit backend to
use the new instruction encoding scheme.

2. Add 64-bit support to the backend,
         A) Add tests to ensure that 64-bit instructions are being
         generated correctly.
         B) Modify register allocator to use new general purpose and
         floating point registers.
         C) Port 32-bit specific portions of the JIT (for example, guard
         failure handling) to 64-bit

3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues.

If the 64-bit JIT is completed before the end of the summer, I will
spend the remainder of the time working on other 64-bit or JIT-related
aspects of PyPy, for example adapting the "asmgcc" garbage collection
strategy to 64-bit.

=== Development Workflow ===

The PyPy project makes extensive use of Subversion branches for
development, so I will follow that convention and develop the 64-bit JIT
in one or more branches.

For unit testing, I will use the py.test framework (already used
throughout PyPy), aiming for 100% test coverage.

=== About Me ===

I am a first-year Computer Science student at Flathead Valley Community
College planning to transfer to Montana State University.

I have several years of professional development experience. I am
comfortable programming in Python, C and x86 assembly.

Starting May 17th, I will be able to commit 40 hours/week to the project
until the end of August. I may travel for a few weeks at some point in
the summer, but I will have a laptop with me with the expectation of
continuing full-time work.

=== Contact information ===

Email: jcreigh at gmail.com
IRC:   "jcreigh" on Freenode
Blog:  http://jcreigh.blogspot.com
Phone: <will be included in actual proposal, but pypy-dev is publicly 
archived...>

From fijall at gmail.com  Wed Mar 31 04:03:58 2010
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 30 Mar 2010 20:03:58 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <4BB2789B.70406@gmail.com>
References: <4BB16C2F.5050300@gmail.com>
	<693bc9ab1003300814q6df14a0bo9a2e812e9aa78c26@mail.gmail.com> 
	<4BB2789B.70406@gmail.com>
Message-ID: <y2j693bc9ab1003301903gfbdc2014ld1af2113bfc87a36@mail.gmail.com>

Just a meta-note, how about keeping it in svn? email is rarely a good
version control system (that's why you have your user dir on
codespeak, or you can create one in /svn/user)

On Tue, Mar 30, 2010 at 4:18 PM, Jason Creighton <jcreigh at gmail.com> wrote:
> I agree with fijal that it seems short, but I don't know what else to
> put in.
>
> Updated draft:
>
> === Proposal ===
>
> The PyPy JIT, which has shown substantial performance improvements over
> other implementations of Python, including CPython and Unladen Swallow,
> does not currently support the x86-64 instruction set, making it
> impractical to use on 64-bit x86 systems.
>
> My proposal is to extend the existing x86 JIT backend to support x86-64
> as well.
>
> === Deliverables ===
>
> Stable, tested 64-bit JIT for PyPy on Linux, Mac OS X and Windows merged
> into PyPy trunk.
>
> === Implementation plan ===
>
> This is not a research proposal. The goal is simply to have a PyPy JIT
> that works out of the box on 64-bit CPUs, implemented as conservatively
> as possible.
>
> As such, I will attempt to reuse as much of the existing x86 backend
> that I can. In fact, the architectural similarities between x86 and
> x86-64 are large enough that I hope to implement a unified x86/x86-64
> backend with the majority of the code working for either platform.
>
> There is an existing branch that, while very incomplete, has the
> beginnings of a unified x86/x86-64 instruction encoding module, which
> can encode instructions for either x86 or x86-64 in a fairly seamless
> manner. I intend to use that branch as a starting point.
>
> Milestones:
>
> 1. Take the existing "remove-ri386-multimethod-2" branch and use it as a
> basis for instruction encoding. Modify the existing 32-bit backend to
> use the new instruction encoding scheme.
>
> 2. Add 64-bit support to the backend,
> ? ? ? ? A) Add tests to ensure that 64-bit instructions are being
> ? ? ? ? generated correctly.
> ? ? ? ? B) Modify register allocator to use new general purpose and
> ? ? ? ? floating point registers.
> ? ? ? ? C) Port 32-bit specific portions of the JIT (for example, guard
> ? ? ? ? failure handling) to 64-bit
>
> 3. Test 64-bit on Mac OS X and Windows, and fix inevitable issues.
>
> If the 64-bit JIT is completed before the end of the summer, I will
> spend the remainder of the time working on other 64-bit or JIT-related
> aspects of PyPy, for example adapting the "asmgcc" garbage collection
> strategy to 64-bit.
>
> === Development Workflow ===
>
> The PyPy project makes extensive use of Subversion branches for
> development, so I will follow that convention and develop the 64-bit JIT
> in one or more branches.
>
> For unit testing, I will use the py.test framework (already used
> throughout PyPy), aiming for 100% test coverage.
>
> === About Me ===
>
> I am a first-year Computer Science student at Flathead Valley Community
> College planning to transfer to Montana State University.
>
> I have several years of professional development experience. I am
> comfortable programming in Python, C and x86 assembly.
>
> Starting May 17th, I will be able to commit 40 hours/week to the project
> until the end of August. I may travel for a few weeks at some point in
> the summer, but I will have a laptop with me with the expectation of
> continuing full-time work.
>
> === Contact information ===
>
> Email: jcreigh at gmail.com
> IRC: ? "jcreigh" on Freenode
> Blog: ?http://jcreigh.blogspot.com
> Phone: <will be included in actual proposal, but pypy-dev is publicly
> archived...>
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From jcreigh at gmail.com  Wed Mar 31 19:24:07 2010
From: jcreigh at gmail.com (Jason Creighton)
Date: Wed, 31 Mar 2010 11:24:07 -0600
Subject: [pypy-dev] Rough draft of x86-64 JIT backend GSoC proposal
In-Reply-To: <20100330221477.SM416116@[207.67.72.231]>
References: <20100330221477.SM416116@[207.67.72.231]>
Message-ID: <4BB38537.1060005@gmail.com>

michaelschneider wrote:
> Jason,
> 
> My name is Mike Schneider (bigdog on the irc).
> 
> I think what you have is very good.  Take these as suggestions.

I found them extremely helpful, thank you very much.

I've tried to implement them in my latest draft, and I've also taken 
fijal's suggestion of storing it in svn. Here it is, if anyone has any 
further feedback/suggestions:

http://codespeak.net/svn/user/jcreigh/documents/pypy_amd64_gsoc_proposal.txt

Cheers,

Jason