From fijal at genesilico.pl  Tue Apr  1 06:24:16 2008
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Tue, 01 Apr 2008 06:24:16 +0200
Subject: [pypy-dev] PyOpenGL
Message-ID: <47F1B8F0.5000803@genesilico.pl>

It seems that sentence on a web page saying "The binding is created 
using the standard (in Python 2.5) ctypes library" is simply a lie :( 
There is a SWIG binding laying there. There is roughly 20k lines of swig 
interface code. Not sure how easy would be to wrap automatically, but 
this binding is definitely not written using ctypes, hence I think we're 
not going to support this any time soon.



:.


From amauryfa at gmail.com  Tue Apr  1 09:30:19 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 1 Apr 2008 09:30:19 +0200
Subject: [pypy-dev]  PyOpenGL
In-Reply-To: <e27efe130804010029s494b36e0o6cc6963ac0a539c6@mail.gmail.com>
References: <47F1B8F0.5000803@genesilico.pl>
	<e27efe130804010029s494b36e0o6cc6963ac0a539c6@mail.gmail.com>
Message-ID: <e27efe130804010030t580ad785y8f7dd0886be7feb8@mail.gmail.com>

Hello,

On Tue, Apr 1, 2008 at 6:24 AM, Maciek Fijalkowski <fijal at genesilico.pl> wrote:
> It seems that sentence on a web page saying "The binding is created
>  using the standard (in Python 2.5) ctypes library" is simply a lie :(
>  There is a SWIG binding laying there. There is roughly 20k lines of swig
>  interface code. Not sure how easy would be to wrap automatically, but
>  this binding is definitely not written using ctypes, hence I think we're
>  not going to support this any time soon.

It seems that Version 3 of PyOpenGL migrated from SWIG to ctypes:
http://pyopengl.sourceforge.net/ctypes/development.html

The source code contains tons of blocks like:

# /usr/include/GL/freeglut_std.h 385
glutInitWindowPosition = platform.createBaseFunction(
       'glutInitWindowPosition', dll=platform.GLUT, resultType=None,
       argTypes=[c_int,c_int],
       doc='glutInitWindowPosition( c_int(x), c_int(y) ) -> None',
       argNames=('x', 'y'),
)

which is more pypy-friendly.

--
Amaury Forgeot d'Arc

From fijall at gmail.com  Tue Apr  1 17:48:17 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 1 Apr 2008 17:48:17 +0200
Subject: [pypy-dev] PyOpenGL
In-Reply-To: <e27efe130804010030t580ad785y8f7dd0886be7feb8@mail.gmail.com>
References: <47F1B8F0.5000803@genesilico.pl>
	<e27efe130804010029s494b36e0o6cc6963ac0a539c6@mail.gmail.com>
	<e27efe130804010030t580ad785y8f7dd0886be7feb8@mail.gmail.com>
Message-ID: <693bc9ab0804010848j4f8b9e19v4bf41cfc0e02edfc@mail.gmail.com>

Hum, good news :) I'll try.

On Tue, Apr 1, 2008 at 9:30 AM, Amaury Forgeot d'Arc <amauryfa at gmail.com> wrote:
> Hello,
>
>
>  On Tue, Apr 1, 2008 at 6:24 AM, Maciek Fijalkowski <fijal at genesilico.pl> wrote:
>  > It seems that sentence on a web page saying "The binding is created
>  >  using the standard (in Python 2.5) ctypes library" is simply a lie :(
>  >  There is a SWIG binding laying there. There is roughly 20k lines of swig
>  >  interface code. Not sure how easy would be to wrap automatically, but
>  >  this binding is definitely not written using ctypes, hence I think we're
>  >  not going to support this any time soon.
>
>  It seems that Version 3 of PyOpenGL migrated from SWIG to ctypes:
>  http://pyopengl.sourceforge.net/ctypes/development.html
>
>  The source code contains tons of blocks like:
>
>  # /usr/include/GL/freeglut_std.h 385
>  glutInitWindowPosition = platform.createBaseFunction(
>        'glutInitWindowPosition', dll=platform.GLUT, resultType=None,
>        argTypes=[c_int,c_int],
>        doc='glutInitWindowPosition( c_int(x), c_int(y) ) -> None',
>        argNames=('x', 'y'),
>  )
>
>  which is more pypy-friendly.
>
>  --
>  Amaury Forgeot d'Arc
>
>
> _______________________________________________
>  pypy-dev at codespeak.net
>  http://codespeak.net/mailman/listinfo/pypy-dev
>

From bea at changemaker.nu  Fri Apr  4 14:25:25 2008
From: bea at changemaker.nu (Beatrice During)
Date: Fri, 04 Apr 2008 14:25:25 +0200
Subject: [pypy-dev] Next sprints
In-Reply-To: <20080313083931.GA8571@code0.codespeak.net>
References: <20080313083931.GA8571@code0.codespeak.net>
Message-ID: <47F61E35.700@changemaker.nu>

Hi there

Just checking to see how things went with the sprint
plans?

Cheers

Bea

Armin Rigo wrote:
> Hi all,
>
> This e-mail is about starting the discussion for ideas and plans about
> the next sprints.
>
> What we have planned at the moment:
>
> * The next PyPy sprint is going to be a small one at PyCon
>   (http://us.pycon.org/2008/sprints/projects/#pypy).
>
> * In Leysin, beginning of April, will probably be held a "Paper writing"
>   sprint.  Confirmation and more details should come soon (from Michael
>   Leuschel and Carl Friedrich).
>
> Another idea I now about is a follow-up Smalltalk-in-RPython (SPy)
> sprint (nothing concrete though).
>
> It seems that we could starting planning for a regular PyPy development
> sprint soon.  I'm never tired of inviting people to Leysin but I guess
> we could pick another location now (additionally, there are many places
> where Spring is a nice season, but Leysin is not one of them).
>
>
> A bientot,
>
> Armin
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>
>   


-- 
Bea D?ring
Change Maker

J?rntorget 3
412 55 G?teborg
phone: +46 31 7750940
mobile: +46 734 22 89 06
bea at changemaker.nu
www.changemaker.nu


From fijal at genesilico.pl  Fri Apr  4 18:41:49 2008
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 04 Apr 2008 18:41:49 +0200
Subject: [pypy-dev] Next sprints
In-Reply-To: <47F61E35.700@changemaker.nu>
References: <20080313083931.GA8571@code0.codespeak.net>
	<47F61E35.700@changemaker.nu>
Message-ID: <47F65A4D.4000701@genesilico.pl>

Beatrice During wrote:
> Hi there
>
> Just checking to see how things went with the sprint
> plans?
>
> Cheers
>
> Bea
>   

The didn't

:.


From brett at python.org  Wed Apr  9 00:51:58 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 8 Apr 2008 15:51:58 -0700
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
Message-ID: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>

A discussion of what parts of the sys module various VMs should be
expected to implement has come up on python-dev. I am hoping someone
here can let me know what PyPy currently implements.

 - argv
 - byteorder
 - path, path_hooks, meta_path, path_importer_cache, and modules
 - copyright, hexversion, version, and version_info
 - displayhook, __displayhook__
 - excepthook, __excepthook__, exc_info, and exc_clear
 - exec_prefix and prefix
 - executable
 - exit
 - flags, py3kwarning, dont_write_bytecode, and warn_options
 - getfilesystemencoding
 - get/setprofile
 - get/settrace, call_tracing
 - getwindowsversion
 - maxint and maxunicode
 - platform
 - ps1 and ps2
 - stdin, stderr, stdout, __stdin__, __stderr__, __stdout__
 - tracebacklimit
 - get/setrecursionlimit
 - get/setcheckinterval
 - _getframe and _current_frame
 - getrefcount
 - get/setdlopenflags
 - settscdumps
 - api_version
 - winver
 - dllhandle
 - float_info
 - _compact_freelists
 - _clear_type_cache
 - subversion
 - builtin_module_names
 - callstats
 - intern

-Brett

From brunogola at gmail.com  Wed Apr  9 01:15:29 2008
From: brunogola at gmail.com (Bruno Gola)
Date: Tue, 8 Apr 2008 20:15:29 -0300
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
In-Reply-To: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
References: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
Message-ID: <b27f1a340804081615x7d3f7b90pf83e51db1b651981@mail.gmail.com>

Hi,

On Tue, Apr 8, 2008 at 7:51 PM, Brett Cannon <brett at python.org> wrote:
> A discussion of what parts of the sys module various VMs should be
>  expected to implement has come up on python-dev. I am hoping someone
>  here can let me know what PyPy currently implements.

I've made a comparison between python2.4 sys module and the one in
pypy, it's really simple and I'm not sure that all the others features
are implemented and working as expected, but I hope it helps.

This are the features from 2.4 sys that are not present in PyPy's sys:
- exc_type
- getdlopenflags
- getrefcount
- pydebug
- setdlopenflags
-- 
Bruno Fialho Marques Gola <brunogola at gmail.com>
http://www.brunogola.com.br
Cel: (11) 9294-5883

From arigo at tunes.org  Wed Apr  9 16:40:19 2008
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 9 Apr 2008 16:40:19 +0200
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
In-Reply-To: <b27f1a340804081615x7d3f7b90pf83e51db1b651981@mail.gmail.com>
References: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
	<b27f1a340804081615x7d3f7b90pf83e51db1b651981@mail.gmail.com>
Message-ID: <20080409144019.GA17589@code0.codespeak.net>

Hi Bruno,

On Tue, Apr 08, 2008 at 08:15:29PM -0300, Bruno Gola wrote:
> This are the features from 2.4 sys that are not present in PyPy's sys:
> - exc_type

Actually, exc_type, exc_value and exc_traceback are present in PyPy's
sys module but they are not in sys.__dict__.  There is a __getattr__
handler on the sys module that provides these attributes in a
thread-safe way.

> - pydebug

Where does this come from?  It is not in my Python 2.4 either.


A bientot,

Armin.

From arigo at tunes.org  Wed Apr  9 16:49:54 2008
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 9 Apr 2008 16:49:54 +0200
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
In-Reply-To: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
References: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
Message-ID: <20080409144954.GA18427@code0.codespeak.net>

Hi Brett,

On Tue, Apr 08, 2008 at 03:51:58PM -0700, Brett Cannon wrote:
> A discussion of what parts of the sys module various VMs should be
> expected to implement has come up on python-dev. I am hoping someone
> here can let me know what PyPy currently implements.

PyPy implements the attributes that existed in 2.4 already.  We haven't
decided anything about the newer attributes but I guess they will be
added as we upgrade.  Here is the justification for the few 2.4
attributes that are missing (mentioned by Bruno) and the post-2.4
attributes that maybe we will not implement when we upgrade.

>  - exec_prefix and prefix

These ones are stubs at the moment.  We haven't thought much about what
they should contain and how to install PyPy's standard library in a way
that applications like distutils will find it.

>  - getfilesystemencoding

No support for file system encoding yet.

>  - getwindowsversion
>  - winver
>  - dllhandle

Not much support for Windows-specific features.  Should probably be
improved over time.

>  - getrefcount

Makes only sense in CPython.

>  - get/setdlopenflags
>  - api_version

No support for loading separately-compiled extension modules in PyPy so
far.

>  - settscdumps
>  - callstats

I think these are ways to access internal performance counters of
ceval.c.

>  - _compact_freelists
>  - _clear_type_cache

Looks very much like internal details of CPython.


A bientot,

Armin.

From brett at python.org  Wed Apr  9 19:24:03 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 9 Apr 2008 19:24:03 +0200
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
In-Reply-To: <b27f1a340804081615x7d3f7b90pf83e51db1b651981@mail.gmail.com>
References: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
	<b27f1a340804081615x7d3f7b90pf83e51db1b651981@mail.gmail.com>
Message-ID: <bbaeab100804091024k311a11e2x1a4e022d8757b616@mail.gmail.com>

On Tue, Apr 8, 2008 at 4:15 PM, Bruno Gola <brunogola at gmail.com> wrote:
> Hi,
>
>
>  On Tue, Apr 8, 2008 at 7:51 PM, Brett Cannon <brett at python.org> wrote:
>  > A discussion of what parts of the sys module various VMs should be
>  >  expected to implement has come up on python-dev. I am hoping someone
>  >  here can let me know what PyPy currently implements.
>
>  I've made a comparison between python2.4 sys module and the one in
>  pypy, it's really simple and I'm not sure that all the others features
>  are implemented and working as expected, but I hope it helps.
>
>  This are the features from 2.4 sys that are not present in PyPy's sys:
>  - exc_type
>  - getdlopenflags
>  - getrefcount
>  - pydebug

What attribute is that? The 2.6 trunk doesn't have it.

>  - setdlopenflags
>  --

Thanks for the info, Bruno!

-Brett

From brett at python.org  Wed Apr  9 19:27:18 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 9 Apr 2008 19:27:18 +0200
Subject: [pypy-dev] What parts of the sys modules does PyPy implement?
In-Reply-To: <20080409144954.GA18427@code0.codespeak.net>
References: <bbaeab100804081551m1dd6fff8u3245aef16840ebf6@mail.gmail.com>
	<20080409144954.GA18427@code0.codespeak.net>
Message-ID: <bbaeab100804091027p355d7b51v4b0d39dd7182a79f@mail.gmail.com>

On Wed, Apr 9, 2008 at 7:49 AM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Brett,
>
>
>  On Tue, Apr 08, 2008 at 03:51:58PM -0700, Brett Cannon wrote:
>  > A discussion of what parts of the sys module various VMs should be
>  > expected to implement has come up on python-dev. I am hoping someone
>  > here can let me know what PyPy currently implements.
>
>  PyPy implements the attributes that existed in 2.4 already.  We haven't
>  decided anything about the newer attributes but I guess they will be
>  added as we upgrade.  Here is the justification for the few 2.4
>  attributes that are missing (mentioned by Bruno) and the post-2.4
>  attributes that maybe we will not implement when we upgrade.
>

Thanks, Armin.

-Brett

From cfbolz at gmx.de  Mon Apr 14 23:44:19 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Mon, 14 Apr 2008 23:44:19 +0200
Subject: [pypy-dev] Workshop on Self-sustaining Systems (S3) 2008 | Call for
	Participation
Message-ID: <348899050804141444t607fa2ceqf5cde3b100483b9f@mail.gmail.com>

Call for Participation

*** Workshop on Self-sustaining Systems (S3) 2008 ***

May 15-16, 2008
Hasso-Plattner-Institut
Potsdam, Germany

http://www.swa.hpi.uni-potsdam.de/s3/

----------------------------------------------------------------------

[ Important dates ]

 * Early registration: April 20, 2008
 * S3 workshop: May 15-16, 2008

[ Program ]

 * 3 invited talks
 * 6 technical papers
 * http://www.swa.hpi.uni-potsdam.de/s3/program/

[ Invited speakers ]

 * Ian Piumarta (Viewpoints)
 * Dan Ingalls (Sun Labs)
 * Richard P. Gabriel (IBM Research)

[ Registration ]

 * http://www.swa.hpi.uni-potsdam.de/s3/registration/

We hope to see you in Potsdam,
Kim Rose and Robert Hirschfeld

From lac at openend.se  Tue Apr 15 02:39:57 2008
From: lac at openend.se (Laura Creighton)
Date: Tue, 15 Apr 2008 02:39:57 +0200
Subject: [pypy-dev] If we are getting somewhere with a port of pygame ...
Message-ID: <200804150039.m3F0dvjS007933@theraft.openend.se>

now would be a great time to let the pygame list know.

Laura, going back to vacation, and not going to be the person who
tells them if there is any telling that needs doing.

------- Forwarded Message

Return-Path: owner-pygame-users at seul.org
Delivery-Date: Mon Apr 14 02:04:47 2008
Date: Mon, 14 Apr 2008 10:04:39 +1000
From: "=?ISO-8859-1?Q?Ren=E9_Dudfield?=" <renesd at gmail.com>
To: pygame-users at seul.org
Subject: Re: [pygame] SDL ctypes ImportError
In-Reply-To: <a0efdb8a0804131650q7ffc58xeb7fe9300f96e4d7 at mail.gmail.com>


hello,

pygame-ctypes has been abandoned.

The authors email is in the docs if you'd like to email him.  You
might get a good response on the ctypes list to your issue.


cheers,



On Mon, Apr 14, 2008 at 9:50 AM, PyMike <pymike93 at gmail.com> wrote:
> Hi,
>
> I've been interested in making something similar to pygame, but completely
> in python. I came across http://www.pygame.org/ctypes/ and after I installed
> the SDL lib I get this error when I run __init__.py...
>
> >>>
> Traceback (most recent call last):
>    File "C:\Python25\Lib\site-packages\SDL\dll.py", line 221, in <module>
>     _dll = SDL_DLL('SDL', 'SDL_Linked_Version')
>    File "C:\Python25\Lib\site-packages\SDL\dll.py", line 54, in __init__
>     _platform_library_name(library_name)
>  ImportError: Dynamic library "SDL" was not found
> >>>
>
> ...and I have all the SDL DLLs that are included pygame in the same dir as
> the SDL lib. I want to be able to include SDL with the pkg and not have
> users have to install it into their system folder for the python lib to
> work.
>
> Please help!
> --
> - pymike (pymike93.googlepages.com)

------- End of Forwarded Message


From fijal at genesilico.pl  Tue Apr 15 07:34:01 2008
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Tue, 15 Apr 2008 07:34:01 +0200
Subject: [pypy-dev] If we are getting somewhere with a port of pygame ...
In-Reply-To: <200804150039.m3F0dvjS007933@theraft.openend.se>
References: <200804150039.m3F0dvjS007933@theraft.openend.se>
Message-ID: <48043E49.4000902@genesilico.pl>

Laura Creighton wrote:
> now would be a great time to let the pygame list know.
>
> Laura, going back to vacation, and not going to be the person who
> tells them if there is any telling that needs doing.
>   
>   
No, we're not. We're basically using pyglet for graphics.


:.


From Martin.vonLoewis at hpi.uni-potsdam.de  Wed Apr 16 09:21:03 2008
From: Martin.vonLoewis at hpi.uni-potsdam.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 16 Apr 2008 09:21:03 +0200
Subject: [pypy-dev] S3 Call for Participation
Message-ID: <4805A8DF.8030504@hpi.uni-potsdam.de>

----------------------------------------------------------------------

Call for Participation

*** Workshop on Self-sustaining Systems (S3) 2008 ***

May 15-16, 2008
Hasso-Plattner-Institut
Potsdam, Germany

http://www.swa.hpi.uni-potsdam.de/s3/

----------------------------------------------------------------------

[ Important dates ]

  * Early registration: April 20, 2008
  * S3 workshop: May 15-16, 2008

[ Program ]

  * 3 invited talks
  * 6 technical papers
  * http://www.swa.hpi.uni-potsdam.de/s3/program/

[ Invited speakers ]

  * Ian Piumarta (Viewpoints)
  * Dan Ingalls (Sun Labs)
  * Richard P. Gabriel (IBM Research)

[ Registration ]

  * http://www.swa.hpi.uni-potsdam.de/s3/registration/

We hope to see you in Potsdam,
Kim Rose and Robert Hirschfeld

----------------------------------------------------------------------

From arigo at tunes.org  Tue Apr 22 12:01:05 2008
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 22 Apr 2008 12:01:05 +0200
Subject: [pypy-dev] io-improvements branch
Message-ID: <20080422100104.GA24646@code0.codespeak.net>

Hi,

I reviewed the io-improvements branch that speeds up operations like
os.read() by avoiding many copies of the data.  First note that the
branch also kills coalloc, which proved to be not useful (cfbolz:
objections?).

There is a failure in test_lloperation.py that shows that the llinterp
doesn't implement the new operations malloc_nonmovable(_varsize).  But
first, we need to add a test in rpython/memory/test/test_gc.py that uses
these operations (as well as gc_can_move, which I think is also untested
in the llinterp).

When that's done I think we can run all tests and if they pass, merge
the branch.


A bientot,

Armin

From anto.cuni at gmail.com  Tue Apr 22 22:56:55 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Tue, 22 Apr 2008 22:56:55 +0200
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
Message-ID: <480E5117.2040306@gmail.com>

Hi Niko,
hi all,

there is an interesting thread going on on the jvm-languages mailing 
list; among the other things, I discovered that the JVM can handle the 
exception much faster if you override the fillInStack method to do 
nothing instead of building the traceback.

I think that since we don't rely on jvm tracebacks for exceptions, 
overriding such a method in all our exception classes might lead to some 
speedup.

Note that hotspot is smart enough to optimize well the case in which you 
raise a prebuilt exception, but in all cases in which you have to 
dynamically construct a new exception (e.g., OperationError) it can't.

ciao,
Anto

-------- Original Message --------
Subject: [jvm-l] Re: Fan Programing Language
Date: Tue, 22 Apr 2008 15:05:04 -0400
From: John Cowan <johnwcowan at gmail.com>
Reply-To: jvm-languages at googlegroups.com
To: jvm-languages at googlegroups.com
References: 
<bfb6e3b0-8e81-4e38-95ea-2ce1187558fc at u69g2000hse.googlegroups.com> 
<200804221911.40276.jon at ffconsultancy.com> <480E31B8.5090504 at gmail.com> 
<200804221941.11608.jon at ffconsultancy.com>


On Tue, Apr 22, 2008 at 2:41 PM, Jon Harrop <jon at ffconsultancy.com> wrote:

>  > 2) you are allocating a new exception every time; the optimization
>  > described here [1] works only if the exception is pre-allocated.
>  > [1] http://blogs.sun.com/jrose/entry/longjumps_considered_inexpensive
>
>  I think that is not thread safe. Specifically, when the branch conveys
>  information (passed as arguments using a tail call, or embedded in the
>  exception) then you must use a locally allocated exception, right?

Yes, you must.   However, what makes allocating an exception expensive
is the fillInStack method, which has to walk the JVM stack.  If you
override that in your exception class with a do-nothing method, then
locally allocating exceptions is very cheap.

-- 
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures

--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google 
Groups "JVM Languages" group.
To post to this group, send email to jvm-languages at googlegroups.com
To unsubscribe from this group, send email to 
jvm-languages-unsubscribe at googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/jvm-languages?hl=en
-~----------~----~----~----~------~----~------~--~---



From paul.degrandis at gmail.com  Tue Apr 22 23:25:38 2008
From: paul.degrandis at gmail.com (Paul deGrandis)
Date: Tue, 22 Apr 2008 17:25:38 -0400
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <480E5117.2040306@gmail.com>
References: <480E5117.2040306@gmail.com>
Message-ID: <9c0bb8a00804221425u2e69f00dgbf59f77eb1bd818d@mail.gmail.com>

Anto,

Niko had previously discovered this and we dscussed coming up with a
way to do it.  This is a good motivator to stepping up and getting it
done.  Thanks for hitting the list with this.

Paul


On 4/22/08, Antonio Cuni <anto.cuni at gmail.com> wrote:
> Hi Niko,
>  hi all,
>
>  there is an interesting thread going on on the jvm-languages mailing
>  list; among the other things, I discovered that the JVM can handle the
>  exception much faster if you override the fillInStack method to do
>  nothing instead of building the traceback.
>
>  I think that since we don't rely on jvm tracebacks for exceptions,
>  overriding such a method in all our exception classes might lead to some
>  speedup.
>
>  Note that hotspot is smart enough to optimize well the case in which you
>  raise a prebuilt exception, but in all cases in which you have to
>  dynamically construct a new exception (e.g., OperationError) it can't.
>
>  ciao,
>  Anto
>
>  -------- Original Message --------
>  Subject: [jvm-l] Re: Fan Programing Language
>  Date: Tue, 22 Apr 2008 15:05:04 -0400
>  From: John Cowan <johnwcowan at gmail.com>
>  Reply-To: jvm-languages at googlegroups.com
>  To: jvm-languages at googlegroups.com
>  References:
>  <bfb6e3b0-8e81-4e38-95ea-2ce1187558fc at u69g2000hse.googlegroups.com>
>  <200804221911.40276.jon at ffconsultancy.com> <480E31B8.5090504 at gmail.com>
>  <200804221941.11608.jon at ffconsultancy.com>
>
>
>  On Tue, Apr 22, 2008 at 2:41 PM, Jon Harrop <jon at ffconsultancy.com> wrote:
>
>  >  > 2) you are allocating a new exception every time; the optimization
>  >  > described here [1] works only if the exception is pre-allocated.
>  >  > [1] http://blogs.sun.com/jrose/entry/longjumps_considered_inexpensive
>  >
>  >  I think that is not thread safe. Specifically, when the branch conveys
>  >  information (passed as arguments using a tail call, or embedded in the
>  >  exception) then you must use a locally allocated exception, right?
>
>  Yes, you must.   However, what makes allocating an exception expensive
>  is the fillInStack method, which has to walk the JVM stack.  If you
>  override that in your exception class with a do-nothing method, then
>  locally allocating exceptions is very cheap.
>
>  --
>  GMail doesn't have rotating .sigs, but you can see mine at
>  http://www.ccil.org/~cowan/signatures
>
>  --~--~---------~--~----~------------~-------~--~----~
>  You received this message because you are subscribed to the Google
>  Groups "JVM Languages" group.
>  To post to this group, send email to jvm-languages at googlegroups.com
>  To unsubscribe from this group, send email to
>  jvm-languages-unsubscribe at googlegroups.com
>  For more options, visit this group at
>  http://groups.google.com/group/jvm-languages?hl=en
>  -~----------~----~----~----~------~----~------~--~---
>
>
>  _______________________________________________
>  pypy-dev at codespeak.net
>  http://codespeak.net/mailman/listinfo/pypy-dev
>

From niko at alum.mit.edu  Wed Apr 23 06:22:42 2008
From: niko at alum.mit.edu (Niko Matsakis)
Date: Wed, 23 Apr 2008 06:22:42 +0200
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <480E5117.2040306@gmail.com>
References: <480E5117.2040306@gmail.com>
Message-ID: <2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>

> there is an interesting thread going on on the jvm-languages mailing  
> list; among the other things, I discovered that the JVM can handle  
> the exception much faster if you override the fillInStack method to  
> do nothing instead of building the traceback.

I didn't know about overriding the fillInStack method, that should be  
fairly easy to do.


Niko


From charles.nutter at sun.com  Fri Apr 25 06:05:41 2008
From: charles.nutter at sun.com (Charles Oliver Nutter)
Date: Thu, 24 Apr 2008 23:05:41 -0500
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>
References: <480E5117.2040306@gmail.com>
	<2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>
Message-ID: <48115895.4050606@sun.com>

Niko Matsakis wrote:
>> there is an interesting thread going on on the jvm-languages mailing  
>> list; among the other things, I discovered that the JVM can handle  
>> the exception much faster if you override the fillInStack method to  
>> do nothing instead of building the traceback.
> 
> I didn't know about overriding the fillInStack method, that should be  
> fairly easy to do.

Trivial even.

And if you are just using exceptions as a message-passing or 
stack-unrolling mechanism, just instantiate it once per thread.

- Charlie

From anto.cuni at gmail.com  Fri Apr 25 11:25:17 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 25 Apr 2008 11:25:17 +0200
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <48115895.4050606@sun.com>
References: <480E5117.2040306@gmail.com>
	<2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>
	<48115895.4050606@sun.com>
Message-ID: <4811A37D.4030304@gmail.com>

Charles Oliver Nutter wrote:
> Niko Matsakis wrote:
>>> there is an interesting thread going on on the jvm-languages mailing  
>>> list; among the other things, I discovered that the JVM can handle  
>>> the exception much faster if you override the fillInStack method to  
>>> do nothing instead of building the traceback.
>>
>> I didn't know about overriding the fillInStack method, that should be  
>> fairly easy to do.
> 
> Trivial even.

I tried to manually override fillInStackTrace by editing the generated 
.j file; the resulting pypy-jvm runs about 8% faster on richards and 14% 
on pystone.

> And if you are just using exceptions as a message-passing or 
> stack-unrolling mechanism, just instantiate it once per thread.

I think we already do that; all the standard exceptions are prebuilt 
objects, stored as public static fields.

I don't know if there are other exceptions that are used inside the 
interpreter to unroll the stack but which are not prebuilt.

ciao,
anto

From arigo at tunes.org  Fri Apr 25 13:56:33 2008
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 25 Apr 2008 13:56:33 +0200
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <4811A37D.4030304@gmail.com>
References: <480E5117.2040306@gmail.com>
	<2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>
	<48115895.4050606@sun.com> <4811A37D.4030304@gmail.com>
Message-ID: <20080425115633.GA1322@code0.codespeak.net>

Hi Antonio,

On Fri, Apr 25, 2008 at 11:25:17AM +0200, Antonio Cuni wrote:
> I don't know if there are other exceptions that are used inside the 
> interpreter to unroll the stack but which are not prebuilt.

We tried to avoid all non-prebuilt exceptions for control flow purposes.
I think that in any performance-critical place there is only
OperationError that could possibly be instantiated at run-time.


A bientot,

Armin.

From anto.cuni at gmail.com  Sat Apr 26 12:52:33 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 26 Apr 2008 12:52:33 +0200
Subject: [pypy-dev] [Fwd: Re: Fan Programing Language]
In-Reply-To: <20080425115633.GA1322@code0.codespeak.net>
References: <480E5117.2040306@gmail.com>
	<2F5B0247-7980-430C-8C10-B9CA598276B5@alum.mit.edu>
	<48115895.4050606@sun.com> <4811A37D.4030304@gmail.com>
	<20080425115633.GA1322@code0.codespeak.net>
Message-ID: <48130971.8030600@gmail.com>

Armin Rigo wrote:

> We tried to avoid all non-prebuilt exceptions for control flow purposes.
> I think that in any performance-critical place there is only
> OperationError that could possibly be instantiated at run-time.

I supposed this, but I was not sure.

Good news :-).

ciao,
Anto

From fijall at gmail.com  Mon Apr 28 12:38:52 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 28 Apr 2008 12:38:52 +0200
Subject: [pypy-dev] microbench & pybench
Message-ID: <693bc9ab0804280338h77c82c45u6acc5288333bd8f9@mail.gmail.com>

First: benchmarks (especially those which are micro) sucks. It seems
that these days we're vastly faster on microbenchs than cpython 2.5
and way slower on some tests from pybench (like small tuple
operations). Here are some results (it's the other way around than on
tuatara, higher is better, more than 1 means faster than cpython).
This only showcases how much microbenchmarks are dependand on tight
loop speed in my opinion :-( Pybench is slightly smarter in this
regard.

 0.22x slower on test_unicode.test_bad_case_python2_5()
 0.26x slower on test_formatting.test_format_unicode()
 0.26x slower on test_formatting.test_format_long()
 0.78x slower on test_exception.test_reraise()
 0.83x slower on test_bltn.test_call_fabs()
 0.84x slower on test_exception.test_except_specific_builtin_exception()
 0.87x slower on test_exception.test_except_multiple_builtin_exception()
 0.87x slower on test_bltn.test_call_sin()
 0.91x slower on test_exception.test_except_specific_user_exception()
 0.94x slower on test_count1.test_loop_unrolled()
 0.94x slower on test_count1.test_loop()
 0.97x slower on test_dict.test_dict_creation_mode3()
 1.01x slower on test_exception.test_except_multiple_user_exception()
 1.10x slower on test_bltn.test_isinstance2()
 1.12x slower on test_count1.test_call_function_with_arguments_in_cellvars()
 1.15x slower on test_count1.test_count_in_attr()
 1.15x slower on test_dispatch.test_dispatch_nop()
 1.18x slower on test_unicode.test_find_pattern1()
 1.19x slower on test_bltn.test_isinstance1()
 1.19x slower on test_count1.test_loop_other_count()
 1.23x slower on test_exception.test_raise_builtin_exception()
 1.23x slower on test_count1.test_call_function_without_arguments_in_cellvars()
 1.24x slower on test_count1.test_call_method_of_old_style_class()
 1.28x slower on test_count1.test_count_in_global()
 1.29x slower on test_exception.test_raise_user_exception()
 1.29x slower on test_count1.test_call_method_of_new_style_class()
 1.29x slower on test_bltn.test_isinstance3()
 1.30x slower on test_count1.test_count_in_global2()
 1.30x slower on test_formatting.test_number_formatting()
 1.32x slower on test_create1.test_simple_loop_with_new_style_class_new()
 1.35x slower on test_exception.test_try_except_finally()
 1.36x slower on test_count1.test_count_in_dict()
 1.37x slower on test_exception.test_try_except_else()
 1.38x slower on test_count1.test_call_nested_function_many_args()
 1.39x slower on test_dict.test_dict_class_dict_getmethod()
 1.39x slower on test_exception.test_try_except()
 1.40x slower on test_count1.test_call_nested_function()
 1.40x slower on test_create1.test_simple_loop()
 1.42x slower on test_dict.test_dict_instance_setattr_instance_dict()
 1.44x slower on test_dict.test_dict_instance_setnewattr_instance_dict()
 1.45x slower on test_count1.test_call_nested_function_other_count()
 1.49x slower on test_list.test_list_setitem()
 1.55x slower on test_list.test_list_extend()
 1.59x slower on test_count1.test_call_function()
 1.63x slower on test_dict.test_dict_getitem()
 1.64x slower on test_count1.test_count_in_slot()
 1.65x slower on test_formatting.test_dict_formatting()
 1.68x slower on test_dict.test_dict_creation_mode1()
 1.72x slower on test_unicode.test_find_pattern2()
 1.75x slower on test_dict.test_dict_setitem2()
 1.75x slower on test_dict.test_dict_raw_range()
 1.75x slower on test_dict.test_dict_creation_mode2()
 1.76x slower on test_dict.test_dict_instance_getattr_instance_dict()
 1.80x slower on test_list.test_list_append()
 1.92x slower on test_create1.test_simple_loop_with_old_style_class_creation()
 2.01x slower on test_formatting.test_simple_formatting()
 2.12x slower on test_formatting.test_evil_formatting()
 2.27x slower on test_create1.test_simple_loop_with_new_style_class_creation()
 2.37x slower on test_list.test_list_getitem()
 2.55x slower on test_dict.test_dict_setitem1()
 2.59x slower on test_unicode.test_find_worstcase()
 2.67x slower on test_unicode.test_find_pattern4()
 2.95x slower on test_list.test_list_slice()
 3.24x slower on test_unicode.test_count_worstcase()
 3.27x slower on test_exception.test_instantiate_user_exception()
 3.64x slower on test_exception.test_instantiate_builtin_exception()
 3.73x slower on test_formatting.test_repr_formatting()
 4.22x slower on test_unicode.test_find_pattern8()
 6.32x slower on test_unicode.test_find_pattern16()

PyPy-c compiled using --gc=generation --gcrootfinder=asmgcc --faassen
cpython 2.5 from release

On pybench we're 70% slower in general than cpython, 55% slower with
--objspace-std-withstrslice, comparison table of withstrslice
(positive numbers means slowdown compared to cpython, negative means
speedup, in %):

Tests:                              per run    per oper.    diff *)
------------------------------------------------------------------------
          BuiltinFunctionCalls:      37.70 ms    0.30 us    -7.82%
           BuiltinMethodLookup:     133.65 ms    0.25 us   +47.27%
                 CompareFloats:      71.55 ms    0.16 us    +5.53%
         CompareFloatsIntegers:     376.70 ms    0.84 us  +339.04%
               CompareIntegers:      69.20 ms    0.08 us   -19.25%
        CompareInternedStrings:      91.60 ms    0.18 us   +51.66%
                  CompareLongs:      71.55 ms    0.16 us   +19.65%
                CompareStrings:     104.65 ms    0.21 us   +17.12%
                CompareUnicode:      67.95 ms    0.18 us   -25.78%
                 ConcatStrings:      28.30 ms    0.19 us   -70.40%
                 ConcatUnicode:     442.10 ms    2.95 us  +225.67%
               CreateInstances:      81.60 ms    1.94 us   +20.18%
       CreateStringsWithConcat:      46.40 ms    0.23 us    +8.54%
       CreateUnicodeWithConcat:     107.45 ms    0.54 us    +7.56%
                  DictCreation:     104.65 ms    0.70 us   +77.67%
             DictWithFloatKeys:     311.75 ms    0.52 us  +135.46%
           DictWithIntegerKeys:     171.65 ms    0.29 us  +113.89%
            DictWithStringKeys:     105.70 ms    0.18 us   +33.04%
                      ForLoops:     147.00 ms   14.70 us  +145.00%
                    IfThenElse:      79.15 ms    0.12 us    +4.77%
                   ListSlicing:      52.00 ms   14.86 us   +63.27%
                NestedForLoops:      93.00 ms    0.27 us  +126.83%
          NormalClassAttribute:     185.40 ms    0.31 us  +114.09%
       NormalInstanceAttribute:     113.25 ms    0.19 us   +40.25%
           PythonFunctionCalls:      99.55 ms    0.60 us   +21.92%
             PythonMethodCalls:      82.75 ms    1.10 us   +23.69%
                     Recursion:      86.65 ms    6.93 us   +46.99%
                  SecondImport:      51.90 ms    2.08 us   +44.17%
           SecondPackageImport:      34.95 ms    1.40 us    -7.66%
         SecondSubmoduleImport:      77.75 ms    3.11 us   +52.45%
       SimpleComplexArithmetic:      67.75 ms    0.31 us   +47.60%
        SimpleDictManipulation:     105.70 ms    0.35 us  +104.05%
         SimpleFloatArithmetic:      80.20 ms    0.15 us    -4.18%
      SimpleIntFloatArithmetic:      87.15 ms    0.13 us   +21.72%
       SimpleIntegerArithmetic:      89.20 ms    0.14 us   +22.95%
        SimpleListManipulation:      54.50 ms    0.20 us   +21.92%
          SimpleLongArithmetic:      63.80 ms    0.39 us   +64.01%
                    SmallLists:      66.30 ms    0.26 us    -6.09%
                   SmallTuples:     250.55 ms    1.04 us  +270.36%
         SpecialClassAttribute:     197.45 ms    0.33 us  +127.61%
      SpecialInstanceAttribute:     114.35 ms    0.19 us   -14.41%
                StringMappings:     104.10 ms    0.83 us   +30.86%
              StringPredicates:      68.25 ms    0.24 us   -10.02%
                 StringSlicing:      64.30 ms    0.37 us   +17.87%
                     TryExcept:     105.80 ms    0.07 us   +15.82%
                TryRaiseExcept:      37.55 ms    2.50 us   +14.31%
                  TupleSlicing:      91.65 ms    0.87 us   +43.54%
               UnicodeMappings:     132.60 ms    7.37 us   +67.11%
             UnicodePredicates:      56.95 ms    0.25 us   -15.75%
                UnicodeSlicing:     169.15 ms    0.97 us  +123.60%
------------------------------------------------------------------------
            Average round time:    6238.00 ms              +55.21%

Cheers,
fijal

From santagada at gmail.com  Mon Apr 28 13:20:49 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Mon, 28 Apr 2008 12:20:49 +0100
Subject: [pypy-dev] microbench & pybench
In-Reply-To: <693bc9ab0804280338h77c82c45u6acc5288333bd8f9@mail.gmail.com>
References: <693bc9ab0804280338h77c82c45u6acc5288333bd8f9@mail.gmail.com>
Message-ID: <FED477CE-2F52-4FA6-B849-08A2B5751541@gmail.com>


On 28/04/2008, at 11:38, Maciej Fijalkowski wrote:

> First: benchmarks (especially those which are micro) sucks. It seems
> that these days we're vastly faster on microbenchs than cpython 2.5
> and way slower on some tests from pybench (like small tuple
> operations). Here are some results (it's the other way around than on
> tuatara, higher is better, more than 1 means faster than cpython).
> This only showcases how much microbenchmarks are dependand on tight
> loop speed in my opinion :-( Pybench is slightly smarter in this
> regard.

One important question is, can we profile some real python programs  
and see the same behavior in slowdowns? Because right now what you are  
doing is comparing two synthetic benchmarks, which I think is not as  
usefull for pypython.

The python interpreter needs better benchmarks... that is a given.


From lac at openend.se  Fri May  2 08:04:13 2008
From: lac at openend.se (Laura Creighton)
Date: Fri, 02 May 2008 08:04:13 +0200
Subject: [pypy-dev] Nicolas Dudfield's GSoC project
Message-ID: <200805020604.m4264DFm018662@theraft.openend.se>


This showed up on the pygame mailing list.
Nicolas cc'd here.

------- Forwarded Message

Greetings all,

I am a participant in the Google Summer of Code. My application for
writing tests for pygame was accepted. The goal, over and above having
tests for their own sake, is to make pygame easier to port to new and
different platforms and versions of Python.

Py3k is due soon, and people have all sorts of ideas for pygame, such
as deploying on flash VMs using pypy. Really cool ideas. All of this
will be made much easier with more extensive testing in place. A rough
estimate for current unittest coverage is about 20%. Ideally, at least
95% would be covered.

A brief outline
===============

1.  Write unittests for pygame
2.  Develop a speed regression framework
3.  Develop an interactive testing framework.

1)  Write unittests for pygame
==============================

Being a student, the fact is that I don't have that much experience
writing tests. I haven't spent 10 years as a QA testing manager "in
the real world" for example. My planned approach, that seems to have
won endorsement from the PSF, is to do "breadth first"
testing. Obviously you want bang for your buck, but where to start? By
first getting a big picture view of pygame and becoming aware of the
interconnections I will be better prepared to go into the next
phase. That being that as time allows and with consultation from the
more experienced pygamers in the community I will dive into more
thorough pinpointed testing. Like all plans you can only really set
your sails and adjust accordingly to changing conditions.

2)  Develop a speed regression framework
========================================

"Ok, the change I made passes all the tests, great. Next....? "

As part of the tests I will also profile select areas so it's easy to
judge when performance regressions are introduced.

3)  Develop an interactive testing framework
============================================

I like to think of testing as ESP; extra-sensory programming. You use
the computer as an extra sense in the perception of "does it work like
it should". Instead of trying all the different permutations yourself
(haphazardly and slowly), you instruct your servant to do them
signalling the result in a condensed form. "Does all that work?" Your
computer signals "yes" (hopefully). The two of you zip through the
tasks.

Some times it is hard to instruct your faithful servant
though. "Jiles, Did you hear that mp3? Did it play twice?". Jiles can
only "hear" the other world. You have to "tell" Jiles to help you as
best he can; running through a batch of tests in a logical order,
asking you if they passed, and carefully recording your response.

I will develop a framework for interactive testing. At this stage I am
thinking it would be good to use pygame as the gui component but if
something else in the python opensource world allows the job to be
done faster I will probably use that. I have a little experience with
PyQt4.

Conclusion
==========

There is lot's of work ahead and I will appreciate any help and advice.

Thanks,

akalias, aka Nicholas Dudfield

ps. My blog is at blog.akalias.net and my irc handle: akalias.
------- End of Forwarded Message

My thoughts -- this is too ambitious a project for a summer.  Just getting
the unit tests into pygame may take the whole time.  But if you want a
speed regression framework, well, we sort of have a way to measure
the speed of pypy and graph the results.  See.
http://morepypy.blogspot.com/2007/12/various-performance-improvements.html

Whatever you want to do, Nicholas, you probably don't want to write a
whole new testing framework from scratch.  (We did that.  We know how
much work that is. ) but instead write  something so that people can
use their existing frameworks to measure performance.

Laura


From holger at merlinux.de  Fri May  2 17:37:49 2008
From: holger at merlinux.de (holger krekel)
Date: Fri, 2 May 2008 17:37:49 +0200
Subject: [pypy-dev] rlcompleter2 on pypy
Message-ID: <20080502153749.GJ24157@trillke.net>

Hi Armin, all,

don't know if you noticed: i think that your readline/curses
hack is good enough that my good old completion module mostly
works out of the box on pypy-c. (http://codespeak.net/rlcompleter2/).
It produces slightly different output and there probably also 
are some corner cases that pypy doesn't cover yet. 

nice!
holger

From fijal at genesilico.pl  Fri May  2 19:05:42 2008
From: fijal at genesilico.pl (Maciek Fijalkowski)
Date: Fri, 02 May 2008 19:05:42 +0200
Subject: [pypy-dev] rlcompleter2 on pypy
In-Reply-To: <20080502153749.GJ24157@trillke.net>
References: <20080502153749.GJ24157@trillke.net>
Message-ID: <481B49E6.50102@genesilico.pl>

holger krekel wrote:
> Hi Armin, all,
>
> don't know if you noticed: i think that your readline/curses
> hack is good enough that my good old completion module mostly
> works out of the box on pypy-c. (http://codespeak.net/rlcompleter2/).
> It produces slightly different output and there probably also 
> are some corner cases that pypy doesn't cover yet. 
>
> nice!
> holger

I think i'm using it :)

:.


From cfbolz at gmx.de  Sun May  4 17:19:21 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Sun, 04 May 2008 17:19:21 +0200
Subject: [pypy-dev] PyPy Berlin Sprint, 17th - 22nd May 2008
Message-ID: <481DD3F9.7070205@gmx.de>

=====================================================================
         PyPy Berlin Sprint (17-22nd May 2008)
=====================================================================

The next PyPy sprint will be in the crashed `c-base space station`_,
Berlin, Germany, Earth, Solar System.  This is a fully public sprint:
newcomers (from all planets) and topics other than those proposed below
are welcome.

.. _`c-base space station`: http://www.c-base.org/


------------------------------
Goals and topics of the sprint
------------------------------

  - work on PyPy's JIT generator: we are refactoring parts of the
    compiling logic, in ways that may also allow generating better
    machine code for loops (people with knowledge on compilers and SSA,
    welcome)

  - work on the SPy VM, PyPy's Squeak implementation, particularly the
    graphics capabilities

  - work on PyPy's GameBoy emulator, which also needs graphics support

  - trying some large pure-Python applications or libraries on PyPy and
    fixing the resulting bugs. Possibilities are Zope 3, Django and
    others.

* We are open to all sorts of other tasks during the sprint, just
  propose something.

-----------------------
Location & Accomodation
-----------------------

The sprint will take place in the c-base in Berlin. The address is::

    c-base e.V.
    Rungestrasse 20
    10179 Berlin

To get there, take the U-Bahn or S-Bahn to the station
"Jannowitzbr?cke". See here_ for a map to get to c-base from there.

.. _here:
http://maps.google.com/maps?f=q&hl=en&geocode=&q=c-base+berlin&ie=UTF8&ll=52.515464,13.408985&spn=0.020449,0.057335&z=15&iwloc=A

If you want to come, please register via by svn:

  http://codespeak.net/svn/pypy/extradoc/sprintinfo/berlin-2008/people.txt

or write a mail to the pypy-sprint mailing list if you do not yet have
check-in rights:

  http://codespeak.net/mailman/listinfo/pypy-sprint

Of course, especially given the location, it's ok to show up even if you
didn't register.  (The c-base has probably got the plans of Guido's
famous time machine anyway, so you can register last week.)


-----------
Exact times
-----------

The sprint will be from 17th to 22nd of May 2008. We will start
sprinting every day at 10. An introduction will be given on the first
day, if there is interest.

From brunogola at gmail.com  Sun May 11 04:02:30 2008
From: brunogola at gmail.com (Bruno Gola)
Date: Sat, 10 May 2008 23:02:30 -0300
Subject: [pypy-dev] 2.5 features, introducing myself
Message-ID: <b27f1a340805101902g5573efecnc2487398cb6b9aaa@mail.gmail.com>

Hi,

Maybe I should have sent this earlier. I was finishing my work within
my old job and now I can start coding for PyPy full-time! :-D

My name is Bruno Gola, I'm an undergraduating in Computer Science, I
live in Brazil and I'll work on supporting (C)Python 2.5 features in
PyPy this winter :-)

I want to thank Carl for mentoring me, he'd already gave me access to
the repository and set up a branch for me at
http://codespeak.net/svn/pypy/branch/2.5-features/.

I'm sure I will have a lot of fun and that I will learn a lot with
this project (which is part of my graduation project). My plans after
GSoC are to continue supporting new features from CPython.

If you are interested in testing and watching what I'm doing you can
visit my blog (http://blog.brunogola.com.br/ , category PyPy) and the
2.5-features branch.

Actually I've already started coding some small and easy changes
(__hash__() returning a long and defaultdict).

I'll try to be available always on IRC, my nick is bgola, feel free to
talk to me :)

Thanks,
-- 
Bruno Fialho Marques Gola <brunogola at gmail.com>
http://www.brunogola.com.br
Cel: (11) 9294-5883

From scott+pypy-dev at scottdial.com  Mon May 12 06:25:51 2008
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Mon, 12 May 2008 00:25:51 -0400
Subject: [pypy-dev] Windows Testing
Message-ID: <4827C6CF.4010307@scottdial.com>

Greetings Everyone,

I was indulging in a bit of googling of my name and ran across this blog 
post from months back:

http://morepypy.blogspot.com/2007/11/sprint-discussions-releases-testing.html

"""
As you can see, we are lacking in the Windows testing area, which is an 
even worse problem because none of the currently active developers has 
Windows as his primary OS. We should improve this by finding a Windows 
machine where the tests are run nightly and where we can log in to try 
bug-fixes quickly. The latter bit is important, we had a nightly windows 
test run before (thanks to Scott Dial) but it didn't help, because even 
if you tried to fix a bug you would have to wait until the next night to 
see whether it worked.
"""

Oddly enough, I came to this conclusion as well, but at the time I 
withdrew my box because some serious problems developed with PyPy on 
Win32 that made it impractical to continue to run the test suite. Early 
this February I took some time to try and run the test suite again (same 
problems exist as they did almost a year ago).

However, this time, I set it up on a VMWare instance on a Linux server. 
If someone would be interested in having access to that VMWare instance, 
then I would be willing to provide the login information. Alternatively, 
I might be able to invest some time in solving the bugs, but I'm not 
sure. In the past, I got the impression that the development team was 
indifferent to Win32 support and was not interested in being an advocate.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From gbowyer at fastmail.co.uk  Tue May 13 11:02:51 2008
From: gbowyer at fastmail.co.uk (Greg Bowyer)
Date: Tue, 13 May 2008 10:02:51 +0100
Subject: [pypy-dev] Windows Testing
In-Reply-To: <4827C6CF.4010307@scottdial.com>
References: <4827C6CF.4010307@scottdial.com>
Message-ID: <g0bl4b$eac$1@ger.gmane.org>

Scott Dial wrote:
> Greetings Everyone,
> 
> I was indulging in a bit of googling of my name and ran across this blog 
> post from months back:
> 
> http://morepypy.blogspot.com/2007/11/sprint-discussions-releases-testing.html
> 
> """
> As you can see, we are lacking in the Windows testing area, which is an 
> even worse problem because none of the currently active developers has 
> Windows as his primary OS. We should improve this by finding a Windows 
> machine where the tests are run nightly and where we can log in to try 
> bug-fixes quickly. The latter bit is important, we had a nightly windows 
> test run before (thanks to Scott Dial) but it didn't help, because even 
> if you tried to fix a bug you would have to wait until the next night to 
> see whether it worked.
> """
> 
> Oddly enough, I came to this conclusion as well, but at the time I 
> withdrew my box because some serious problems developed with PyPy on 
> Win32 that made it impractical to continue to run the test suite. Early 
> this February I took some time to try and run the test suite again (same 
> problems exist as they did almost a year ago).
> 
> However, this time, I set it up on a VMWare instance on a Linux server. 
> If someone would be interested in having access to that VMWare instance, 
> then I would be willing to provide the login information. Alternatively, 
> I might be able to invest some time in solving the bugs, but I'm not 
> sure. In the past, I got the impression that the development team was 
> indifferent to Win32 support and was not interested in being an advocate.
> 
> -Scott
> 

Out of interest what is broken on the windows testing front ?


From michaelschneider at current.net  Tue May 13 16:29:13 2008
From: michaelschneider at current.net (Michael Schneider)
Date: Tue, 13 May 2008 10:29:13 -0400
Subject: [pypy-dev] Windows Testing
In-Reply-To: <4827C6CF.4010307@scottdial.com>
References: <4827C6CF.4010307@scottdial.com>
Message-ID: <4829A5B9.7050803@current.net>

Scott,

I am replying to your email with good news.  Over the last couple of 
days, the pypy devs have worked
to address the build issues that was triggering failures that occurred 
during  the --allworkingmdoules windows translation.

If you perform an update from svn, and follow the instructions below, 
you will have a running pypy exe.

We welcome you back with open arms, and have several opportunities for 
you to improve pypy on windows.

I put together some notes that should help you get started again.

Please don't hesitate to ask any questions,
Welcome back,
Mike



------------------- Individual Development Setup Environment 
-------------------------

To setup your windows dev env:

1) Install or activate (vcvars32.bat) visual studio 2003
2) follow instructions on 
http://codespeak.net/pypy/dist/pypy/doc/windows.html
3) install zlib, bz2 and openssl dev environments
     - install dll's into  Windows/system32
    - install  headers into vc/include/..
    - install lib files into vc/lib

4) download pypy from svn
5) cd pypy\translator\goal\
6) translate.py --text --run targetpypystandalone.py - -allworkingmodules

You should now have a running interpreter.

------------------- Buildbot support ------------------------------------
JP has setup pypy buildbots for windows, linux 64, and linux 32

Here is a link to the buildbot page:  
http://office.divmod.com:10408/waterfall

Buildbot kicks off a nightly build at 7:00 EST, and can be triggered on 
demand.


------------------ Proposed  Access to Windows VM on Scott's linux 
box------------

This would be very valuable for debugging purposes.  Please go ahead and 
set this up.
I would be happy to help you address any issues that arise.


---------------- pypy modules disabled on windows -------------------------

the file   pypy\config\pypypotion.py  defines which modules will be 
include in the build.

on windows, we remove the following libraries from the translator built 
with --allworkingmodules

A good project would be to finish some of the modules on this list.

if sys.platform == "win32":
    # unix only modules
    del working_modules["crypt"]
    del working_modules["fcntl"]
    del working_modules["termios"]
    del working_modules["_minimal_curses"]
    # modules currently missing explicit windows support
    del working_modules["signal"]
    del working_modules["_rawffi"]
    # modules with broken windows support
    del working_modules["mmap"]    # MLS - Added 5/11/08 - broken
    del working_modules["_socket"] # MLS - Added 5/11/08 - broken
    del working_modules["select"] # MLS - Added 5/11/08 - broken


------------------------------------------------------------------------------


Scott Dial wrote:
> Greetings Everyone,
>
> I was indulging in a bit of googling of my name and ran across this blog 
> post from months back:
>
> http://morepypy.blogspot.com/2007/11/sprint-discussions-releases-testing.html
>
> """
> As you can see, we are lacking in the Windows testing area, which is an 
> even worse problem because none of the currently active developers has 
> Windows as his primary OS. We should improve this by finding a Windows 
> machine where the tests are run nightly and where we can log in to try 
> bug-fixes quickly. The latter bit is important, we had a nightly windows 
> test run before (thanks to Scott Dial) but it didn't help, because even 
> if you tried to fix a bug you would have to wait until the next night to 
> see whether it worked.
> """
>
> Oddly enough, I came to this conclusion as well, but at the time I 
> withdrew my box because some serious problems developed with PyPy on 
> Win32 that made it impractical to continue to run the test suite. Early 
> this February I took some time to try and run the test suite again (same 
> problems exist as they did almost a year ago).
>
> However, this time, I set it up on a VMWare instance on a Linux server. 
> If someone would be interested in having access to that VMWare instance, 
> then I would be willing to provide the login information. Alternatively, 
> I might be able to invest some time in solving the bugs, but I'm not 
> sure. In the past, I got the impression that the development team was 
> indifferent to Win32 support and was not interested in being an advocate.
>
> -Scott
>
>   


From amauryfa at gmail.com  Tue May 13 17:17:44 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Tue, 13 May 2008 17:17:44 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <4829A5B9.7050803@current.net>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
Message-ID: <e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>

Hello,

Michael Schneider wrote:
> A good project would be to finish some of the modules on this list.

Many months ago, I did some changes in my workspace to address windows support.
Since there seems to be an new interest for pypy-on-win32, I tried to
revive them.
For example, all tests pass in test_rffi and test_ll2ctypes.

The problem is that my machine at home only has 512Mb of memory, and
cannot do the whole pypy translation.
But if there are windows buildbots, it is easy for me to check the
results there, while I work on smaller parts.

I will try to carefully commit my changes one by one.
I am currently working on the _socket and select modules, with good
progress so far.

Cheers,

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Wed May 14 14:49:08 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 May 2008 14:49:08 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
Message-ID: <693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>

Hi Amaury!

Thanks for doing good job bringing more support for pypy on windows. I
think next direction would be to bring more modules to windows, from
which I guess _rawffi is the most important (this will bring ctypes to
windows platform). I can help you bits with that if you would like to
go in that direction.

Cheers,
fijal

From amauryfa at gmail.com  Wed May 14 15:44:41 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 14 May 2008 15:44:41 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
Message-ID: <e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>

Hello,

Maciej Fijalkowski wrote:
> Hi Amaury!
>
> Thanks for doing good job bringing more support for pypy on windows. I
> think next direction would be to bring more modules to windows, from
> which I guess _rawffi is the most important (this will bring ctypes to
> windows platform). I can help you bits with that if you would like to
> go in that direction.

Well, I am not a ffi expert, but at least I know how to run the unit tests.
The first step I see is to get the equivalent of libffi for win32 and
microsoft compilers.
My first thought was to steal it from the
cpython/Modules/_ctypes/libffi_msvc directory,
do you have other ideas?

-- 
Amaury Forgeot d'Arc

From fijall at gmail.com  Wed May 14 15:58:06 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 14 May 2008 15:58:06 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
	<e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
Message-ID: <693bc9ab0805140658m201896e7o266bd6afd208e2e1@mail.gmail.com>

On Wed, May 14, 2008 at 3:44 PM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
> Hello,
>
>
>
>  Maciej Fijalkowski wrote:
>  > Hi Amaury!
>  >
>  > Thanks for doing good job bringing more support for pypy on windows. I
>  > think next direction would be to bring more modules to windows, from
>  > which I guess _rawffi is the most important (this will bring ctypes to
>  > windows platform). I can help you bits with that if you would like to
>  > go in that direction.
>
>  Well, I am not a ffi expert, but at least I know how to run the unit tests.
>  The first step I see is to get the equivalent of libffi for win32 and
>  microsoft compilers.
>  My first thought was to steal it from the
>  cpython/Modules/_ctypes/libffi_msvc directory,
>  do you have other ideas?
>

Honestly, no :) You don't need to be an ffi expert, all logic is
there, it's just missing few places where windows support is needed.
We tried including libffi into pypy, but failed for some obscure build
process reasons.

Thanks for tackling this!

Cheers,
fijal

From amauryfa at gmail.com  Wed May 14 19:34:11 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 14 May 2008 19:34:11 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <693bc9ab0805140658m201896e7o266bd6afd208e2e1@mail.gmail.com>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
	<e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
	<693bc9ab0805140658m201896e7o266bd6afd208e2e1@mail.gmail.com>
Message-ID: <e27efe130805141034p39ed8c26o7ac54f0a73d59099@mail.gmail.com>

Maciej Fijalkowski wrote:
>>  > Hi Amaury!
>>  >
>>  > Thanks for doing good job bringing more support for pypy on windows. I
>>  > think next direction would be to bring more modules to windows, from
>>  > which I guess _rawffi is the most important (this will bring ctypes to
>>  > windows platform). I can help you bits with that if you would like to
>>  > go in that direction.
>>
>>  Well, I am not a ffi expert, but at least I know how to run the unit tests.
>>  The first step I see is to get the equivalent of libffi for win32 and
>>  microsoft compilers.
>>  My first thought was to steal it from the
>>  cpython/Modules/_ctypes/libffi_msvc directory,
>>  do you have other ideas?
>>
>
> Honestly, no :) You don't need to be an ffi expert, all logic is
> there, it's just missing few places where windows support is needed.
> We tried including libffi into pypy, but failed for some obscure build
> process reasons.
>
> Thanks for tackling this!

OK, It seems that I have a working version of rlib\libffi for win32,
enough for test_libffi to pass.

The problem I have now is that I used the files from
cpython/Modules/_ctypes/libffi_msvc verbatim.
The ExternalCompilationInfo looks like this:

    c_source = py.code.Source("""
        /* 40 lines of external definitions, needed to compile the library */
        """)
    eci = ExternalCompilationInfo(
        includes = ['ffi.h', 'windows.h'],
        libraries = ['kernel32'],
        include_dirs = ['c:/afa/python/trunk/Modules/_ctypes/libffi_msvc'],
        separate_module_files =
['c:/afa/python/trunk/Modules/_ctypes/libffi_msvc/ffi.c',

'c:/afa/python/trunk/Modules/_ctypes/libffi_msvc/prep_cif.c',

'c:/afa/python/trunk/Modules/_ctypes/libffi_msvc/win32.c',
                                 ],
        separate_module_sources = [c_source],
        export_symbols = ['ffi_call', 'ffi_prep_cif', 'ffi_prep_closure'],
        )

What shall I do with these external files? Copy them into a
subdirectory of translator/c/src?

-- 
Amaury Forgeot d'Arc

From 2008a at usenet.alexanderweb.de  Wed May 14 21:12:44 2008
From: 2008a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Wed, 14 May 2008 21:12:44 +0200
Subject: [pypy-dev] Windows Testing
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
	<e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
	<693bc9ab0805140658m201896e7o266bd6afd208e2e1@mail.gmail.com>
	<e27efe130805141034p39ed8c26o7ac54f0a73d59099@mail.gmail.com>
Message-ID: <msnrf5-d0b.ln1@tuxpaddy.alexanderweb.homeip.net>

Hi Amaury,

Amaury Forgeot d'Arc wrote:

> What shall I do with these external files? Copy them into a
> subdirectory of translator/c/src?

You can use SVN externals to refer to the CPython repository. Not sure
whether c/src is the right location because they are not backend specific.
Maybe somewhere below rlib? But on the other hand, c/src already has other
files that are not C backend specific.

Kind regards,
Alexander



From michaelschneider at current.net  Thu May 15 05:35:40 2008
From: michaelschneider at current.net (Michael Schneider)
Date: Wed, 14 May 2008 23:35:40 -0400
Subject: [pypy-dev] Windows Testing
In-Reply-To: <e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
References: <4827C6CF.4010307@scottdial.com>
	<4829A5B9.7050803@current.net>	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
	<e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
Message-ID: <482BAF8C.8060806@current.net>

Amaury


I think the python version of libffi is a good starting point for windows.  It seems to be one of the few versions with windows support.

When you are successful, let me know, and I can copy .lib .dll and headers up to the windows build slave.

Good luck, and keep us posted,

Thanks again,
Mike

Amaury Forgeot d'Arc wrote:
> Hello,
>
> Maciej Fijalkowski wrote:
>   
>> Hi Amaury!
>>
>> Thanks for doing good job bringing more support for pypy on windows. I
>> think next direction would be to bring more modules to windows, from
>> which I guess _rawffi is the most important (this will bring ctypes to
>> windows platform). I can help you bits with that if you would like to
>> go in that direction.
>>     
>
> Well, I am not a ffi expert, but at least I know how to run the unit tests.
> The first step I see is to get the equivalent of libffi for win32 and
> microsoft compilers.
> My first thought was to steal it from the
> cpython/Modules/_ctypes/libffi_msvc directory,
> do you have other ideas?
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20080514/899afcac/attachment.htm 

From amauryfa at gmail.com  Thu May 15 10:36:06 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 15 May 2008 10:36:06 +0200
Subject: [pypy-dev] Windows Testing
In-Reply-To: <482BAF8C.8060806@current.net>
References: <4827C6CF.4010307@scottdial.com> <4829A5B9.7050803@current.net>
	<e27efe130805130817k68937dc8u5f24dc503bb47f57@mail.gmail.com>
	<693bc9ab0805140549y6fc1a79bud992306f23fbbecb@mail.gmail.com>
	<e27efe130805140644x64602c1dv791847f0a6fc09c7@mail.gmail.com>
	<482BAF8C.8060806@current.net>
Message-ID: <e27efe130805150136l7a9f6329w4ba50422f422f809@mail.gmail.com>

Hello,

Michael Schneider wrote:
> Amaury
>
>
> I think the python version of libffi is a good starting point for windows.
> It seems to be one of the few versions with windows support.
>
> When you are successful, let me know, and I can copy .lib .dll and headers
> up to the windows build slave.

The "cpython version of libffi" is not a library: it's a collection of
7 sources, that
do not form a library: some important symbols are missing (they are
defined in ctypes.c)
and there is even a reference to Py_FatalError()...
I had to copy the missing externals in another file.

And there is the problem of the versions of the msvc compiler.
Debugging libffi is fun, especially when run on top of ctypes...
But I prefer that everything uses the same compiler and options, so
building from source is better.
It's small: 1540 lines of code.

If nobody objects, I will add (a slightly modified version of) the
needed files in
translator/c/src/libffi_msvc.

-- 
Amaury Forgeot d'Arc

From amauryfa at gmail.com  Thu May 15 14:46:30 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Thu, 15 May 2008 14:46:30 +0200
Subject: [pypy-dev] [pypy-svn] r54748 - in
	pypy/dist/pypy/translator/tool: . test
Message-ID: <e27efe130805150546k6039c159i48abfed0c8d85b35@mail.gmail.com>

Hello Antonio,

> revert r54745, as it breaks translation on linux

You were right to revert the change, but I am still surprised.
What kind of error did you get?

-- 
Amaury Forgeot d'Arc

From anto.cuni at gmail.com  Thu May 15 15:08:08 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 15 May 2008 15:08:08 +0200
Subject: [pypy-dev] [pypy-svn] r54748 - in
	pypy/dist/pypy/translator/tool: . test
In-Reply-To: <e27efe130805150546k6039c159i48abfed0c8d85b35@mail.gmail.com>
References: <e27efe130805150546k6039c159i48abfed0c8d85b35@mail.gmail.com>
Message-ID: <482C35B8.4080201@gmail.com>

Amaury Forgeot d'Arc wrote:
> Hello Antonio,
> 
>> revert r54745, as it breaks translation on linux
> 
> You were right to revert the change, but I am still surprised.
> What kind of error did you get?

Hi Amaury,

what happens is that one of the module_*.c contains this line:

#include <.>

and of course gcc complains about it :-).
I tried a bit to investigate, but didn't go anywhere.

It's not necessary to translate pypy to see the error, 
targetnopstandalone is enough.

If you want to check but don't have any other unix account, I think 
codespeak should be enough (but you need to select another gc since 
boehm is not installed there)
./translate.py --batch --gc=generation targetnopstandalone.py

ciao,
Anto

From fijall at gmail.com  Thu May 15 18:00:41 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 15 May 2008 18:00:41 +0200
Subject: [pypy-dev] Bad news about pysqlite-ctypes
Message-ID: <693bc9ab0805150900j6d85f0a5kb6c9768b7c46c982@mail.gmail.com>

I'm not sure we should use this. It's seems to be broken. Tests of
divmod's axiom (ORM) on top of cpython + sqlite-ctypes show:
FAILED (skips=5, expectedFailures=1, errors=400, successes=47)
JP Calderone also provided some simple code snippets that explode, ie:

>>> from pysqlite2.dbapi2 import connect
>>> db = connect(':memory:')
>>> db.execute('BEGIN IMMEDIATE TRANSACTION')
<pysqlite2.dbapi2.Cursor object at 0xb7dc4f6c>
>>> db.execute('COMMIT')
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "/home/exarkun/Scratch/Source/pysqlite3/pysqlite2/dbapi2.py",
line 315, in execute
    return cur.execute(*args)
  File "/home/exarkun/Scratch/Source/pysqlite3/pysqlite2/dbapi2.py",
line 483, in execute
    raise self.connection._get_exception()
pysqlite2.dbapi2.OperationalError: SQL logic error or missing database
>>>

>>> from pysqlite2.dbapi2 import connect
>>> db = connect(':memory:')
>>> db.execute('create table foo ( x int )')
<pysqlite2.dbapi2.Cursor object at 0xb7d63f6c>
>>> list(_)
[infinite rec]

Not sure where to go from now....

Cheers,
fijal

From holger at merlinux.de  Fri May 16 16:09:05 2008
From: holger at merlinux.de (holger krekel)
Date: Fri, 16 May 2008 16:09:05 +0200
Subject: [pypy-dev] 3rd party modules / fixing
Message-ID: <20080516140905.GB759@trillke.net>

Hi all,

after some discussion with maciej and elsewhere  
i'd suggest to setup a 

    pypy/dist/lib-python/3rdparty 
                                                                                                                                           
directory where we temporarily put our own versions of
packages that we aim to support and from which we can build
upstream patches.  If we evolve into maintaining packages or
if the packages get too big we could shift them to a
different place so that they don't come along with a standard
checkout).  In any case, let's note down exact source
information including urls, revision etc.  in the commit log
messages so that things remain easily trackable.  I guess 
for convenience we could also add that directory above to 
PyPy's app level module lookup paths. 
                                                                                                                                           
best,                                                                                                                                      
                                                                                                                                           
holger                       

-- 
Holger Krekel - freelance manager and programmer 
pylib py.test/greenlets/svn APIs: http://pylib.org 
PyPy  Python/Compiler tool chain: http://codespeak.net/pypy 
merlinux collaborative contracting: http://merlinux.de 

From anto.cuni at gmail.com  Fri May 16 22:07:08 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Fri, 16 May 2008 22:07:08 +0200
Subject: [pypy-dev] Running pysqlite on pypy
Message-ID: <482DE96C.8010504@gmail.com>

Hi Gerhard, hi all,

in the last days, we have been trying to run django on pypy, using the 
ctypes based implementation of pysqlite.

In doing this, we encountered a problem; we have exchanged a bit of 
private mails, so I try to sum up here:

This snippet explodes when run with pysqlite-ctypes, either on cpython 
or pypy-c:

 >>>> from pysqlite2.dbapi2 import connect
 >>>> db = connect(':memory:')
 >>>> db.execute('BEGIN IMMEDIATE TRANSACTION')
<pysqlite2.dbapi2.Cursor object at 0xb7dc4f6c>
 >>>> db.execute('COMMIT')

Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "/home/exarkun/Scratch/Source/pysqlite3/pysqlite2/dbapi2.py",
  line 315, in execute
      return cur.execute(*args)
    File "/home/exarkun/Scratch/Source/pysqlite3/pysqlite2/dbapi2.py",
  line 483, in execute
      raise self.connection._get_exception()
  pysqlite2.dbapi2.OperationalError: SQL logic error or missing database


The very same thing happens on python 2.4 + pysqlite2 (non-ctypes version):

 >>> from pysqlite2.dbapi2 import connect
 >>> db = connect(':memory:')
 >>> db.execute('BEGIN IMMEDIATE TRANSACTION')
<pysqlite2.dbapi2.Cursor object at 0xb7cb0860>
 >>> db.execute('COMMIT')
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
pysqlite2.dbapi2.OperationalError: cannot commit - no transaction is active


However, it works perfectly on cpython 2.5 + sqlite3:

 >>> from sqlite3.dbapi2 import connect
 >>> db = connect(':memory:')
 >>> db.execute('BEGIN IMMEDIATE')
<sqlite3.Cursor object at 0xf7cff050>
 >>> db.execute('COMMIT')
<sqlite3.Cursor object at 0xf7cff020>


Samuele pointed out that maybe it's just a difference between pysqlite2 
and pysqlite3; after more digging, he changed pysqlite-ctypes to print 
every SQL statement before sending it to sqlite; what he got is this:

Python 2.5.1 (r251:54863, Jan 17 2008, 19:35:17)
[GCC 4.0.1 (Apple Inc. build 5465)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> from pysqlite2 import dbapi2
 >>> db = dbapi2.connect(':memory:')
 >>> db.execute('BEGIN IMMEDIATE TRANSACTION')
BEGIN IMMEDIATE TRANSACTION
<pysqlite2.dbapi2.Cursor object at 0x55e90>
 >>> db.execute('COMMIT')
COMMIT
COMMIT
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pysqlite2/dbapi2.py", line 318, in execute
    return cur.execute(*args)
  File "pysqlite2/dbapi2.py", line 489, in execute
    raise self.connection._get_exception()
pysqlite2.dbapi2.OperationalError: SQL logic error or missing database

The double COMMIT is probably causing the problem; not sure if it's a 
bug in pysqlite-ctypes or an expected behavior.


Gerhard, what do you think about all of this? What's the best way to solve?

ciao,
Anto

From fijall at gmail.com  Fri May 16 22:23:33 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 16 May 2008 22:23:33 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482DE96C.8010504@gmail.com>
References: <482DE96C.8010504@gmail.com>
Message-ID: <693bc9ab0805161323ucf95ce0o3374d87af97d68ca@mail.gmail.com>

> Samuele pointed out that maybe it's just a difference between pysqlite2
> and pysqlite3; after more digging, he changed pysqlite-ctypes to print
> every SQL statement before sending it to sqlite; what he got is this:

Please note that for my pysqlite2 it works just fine (on both
cpython2.4 and cpython2.5)

From gh at ghaering.de  Sat May 17 03:27:17 2008
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Sat, 17 May 2008 03:27:17 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482DE96C.8010504@gmail.com>
References: <482DE96C.8010504@gmail.com>
Message-ID: <482E3475.9040102@ghaering.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antonio Cuni wrote:
> Hi Gerhard, hi all,
> 
> in the last days, we have been trying to run django on pypy, using the
> ctypes based implementation of pysqlite.
> 
> In doing this, we encountered a problem; we have exchanged a bit of
> private mails, so I try to sum up here:
> 
> This snippet explodes when run with pysqlite-ctypes, either on cpython
> or pypy-c:
> 
>>>>> from pysqlite2.dbapi2 import connect
>>>>> db = connect(':memory:')
>>>>> db.execute('BEGIN IMMEDIATE TRANSACTION')

That's illegal usage of pysqlite. In the very unlikely case that what
you really want to do is *not* use DB-API implicit transactions. You
have to switch to autocommit mode first:

set isolation_level=None; either the property on the connection object,
or directly in the connect() call. That was the root of the problem -
pysqlite's ctypes branch apparently will need to do a few more sanity
checks.

> [...] The double COMMIT is probably causing the problem; not sure if it's a
> bug in pysqlite-ctypes or an expected behavior.

Both, kind of ;-)

> Gerhard, what do you think about all of this? What's the best way to solve?

I just pushed an ad-hoc fix for this particular problem to the hg repo:

changeset:   328:c42db28b5031
branch:      ctypes
tag:         tip
parent:      317:89e6da6ea1cb
user:        Gerhard Haering <gh at ghaering.de>
date:        Sat May 17 03:19:37 2008 +0200
summary:     Quick fix: only use implicit transactions when autocommit
is off.

HTH & good night; just shout if there's anything else that needs fixing

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFILjR1dIO4ozGCH14RAsMLAJwM2M7SgW5pQaV+4M/cezRb+whkDwCfbP7e
/Dh7VRnCyqfCNPDKk4BGTqE=
=9ByT
-----END PGP SIGNATURE-----

From anto.cuni at gmail.com  Sat May 17 09:31:29 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 17 May 2008 09:31:29 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <693bc9ab0805161323ucf95ce0o3374d87af97d68ca@mail.gmail.com>
References: <482DE96C.8010504@gmail.com>
	<693bc9ab0805161323ucf95ce0o3374d87af97d68ca@mail.gmail.com>
Message-ID: <482E89D1.7010406@gmail.com>

Maciej Fijalkowski wrote:
>> Samuele pointed out that maybe it's just a difference between pysqlite2
>> and pysqlite3; after more digging, he changed pysqlite-ctypes to print
>> every SQL statement before sending it to sqlite; what he got is this:
> 
> Please note that for my pysqlite2 it works just fine (on both
> cpython2.4 and cpython2.5)

which pysqlite2? The standard version or the ctypes-based version?

From anto.cuni at gmail.com  Sat May 17 10:02:13 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Sat, 17 May 2008 10:02:13 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482E3475.9040102@ghaering.de>
References: <482DE96C.8010504@gmail.com> <482E3475.9040102@ghaering.de>
Message-ID: <482E9105.50504@gmail.com>

Gerhard H?ring wrote:

> I just pushed an ad-hoc fix for this particular problem to the hg repo:
> 
> changeset:   328:c42db28b5031
> branch:      ctypes
> tag:         tip
> parent:      317:89e6da6ea1cb
> user:        Gerhard Haering <gh at ghaering.de>
> date:        Sat May 17 03:19:37 2008 +0200
> summary:     Quick fix: only use implicit transactions when autocommit
> is off.
> 
> HTH & good night; just shout if there's anything else that needs fixing


Hi Gerhard,
thank you very much for the fix; indeed, your fix, together with a small 
patches by me, solved the problem:

--- a/pysqlite2/dbapi2.py       Tue Jan 15 16:31:23 2008 +0100
+++ b/pysqlite2/dbapi2.py       Sat May 17 09:57:52 2008 +0200
@@ -236,7 +236,7 @@ def connect(database, **kwargs):
      return factory(database, **kwargs)

  class Connection(object):
-    def __init__(self, database, isolation_level="", detect_types=0, 
*args, **kwargs):
+    def __init__(self, database, isolation_level=None, detect_types=0, 
*args, **kwargs):
          self.db = c_void_p()
          ret  = sqlite.sqlite3_open(database, byref(self.db))


Now the django test that was previously failing passes; I've not run the 
others yet, but I expect them to pass as well, or to be broken for other 
reasons :-)


One more questions: I have two versions of pysqlite-ctypes around:

1) the one which was originally at http://hg.ghaering.de/pysqlite3/; 
that link is now broken but you can still download it from 
http://codespeak.net/~cfbolz/pysqlite3.tar.gz

2) the one in the official pysqlite repo; I've got it by doing hg clone 
http://oss.itsystementwicklung.de/hg/pysqlite/ && hg up -C ctypes pysqlite

I found that while (2) is supposed to be newer, it misses some features 
that (1) has, in particular Connection.create_function (which is needed 
by django).
I'm not an expert of mercurial, but it seems that some changesets went 
lost when moving from the old url to the new one.

thank you again for your efforts!
ciao,
Anto

From gh at ghaering.de  Sat May 17 12:32:45 2008
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Sat, 17 May 2008 12:32:45 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482E9105.50504@gmail.com>
References: <482DE96C.8010504@gmail.com> <482E3475.9040102@ghaering.de>
	<482E9105.50504@gmail.com>
Message-ID: <482EB44D.7000303@ghaering.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Antonio Cuni wrote:
> [...] One more questions: I have two versions of pysqlite-ctypes around:
> 
> 1) the one which was originally at http://hg.ghaering.de/pysqlite3/;
> that link is now broken but you can still download it from
> http://codespeak.net/~cfbolz/pysqlite3.tar.gz

That was more a proof-of-concept (*). Without deep ctypes wizardry I
couldn't make anything with callbacks from SQLite work reliably. That's
why create_function, create_aggregate and friends were removed. If it's
possible to make that work at all, it's beyond my understanding of ctypes.

FWIW I'd be very surprised if you'd get callbacks from C to Python
working with PyPy, using the same ctypes API. Basically anything that
involes a py_object smells like a problem for me.

> 2) the one in the official pysqlite repo; I've got it by doing hg clone
> http://oss.itsystementwicklung.de/hg/pysqlite/ && hg up -C ctypes pysqlite
> 
> I found that while (2) is supposed to be newer, it misses some features
> that (1) has, in particular Connection.create_function (which is needed
> by django).

create_function wasn't really fully implemented there, either. Also,
everything that involved callbacks led to lost references and/or
segfaults. That's why I dumped all that.

> I'm not an expert of mercurial, but it seems that some changesets went
> lost when moving from the old url to the new one.

OTOH all pysqlite branches are now conveniently (for me, at least) in a
single repository.

- -- Gerhard


(*) The current pysqlite-ctypes at least passes all tests and doesn't
leak references running the tests.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFILrRNdIO4ozGCH14RAj/BAKCJC/D7ypxQQz3PfcxdZtu+boFUxACeP7bC
iKEo4ojZq5m3z6nPHak2kxg=
=ZOqI
-----END PGP SIGNATURE-----

From fijall at gmail.com  Sat May 17 15:12:36 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 17 May 2008 15:12:36 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482EB44D.7000303@ghaering.de>
References: <482DE96C.8010504@gmail.com> <482E3475.9040102@ghaering.de>
	<482E9105.50504@gmail.com> <482EB44D.7000303@ghaering.de>
Message-ID: <693bc9ab0805170612k4a86713cq54b8920bbafa3ea2@mail.gmail.com>

> That was more a proof-of-concept (*). Without deep ctypes wizardry I
> couldn't make anything with callbacks from SQLite work reliably. That's
> why create_function, create_aggregate and friends were removed. If it's
> possible to make that work at all, it's beyond my understanding of ctypes.
>

Hi Gerhard.

We tackled the issue by creating closures instead of py_object. This
way it's saner and does not segfault for example on django test suite.
We temporarily commited it to svn branch on codespeak, here:
http://codespeak.net/svn/user/fijal/pysqlite2/, but we would rather
prefer to keep it in your mercurial branch. Do you think our changes
makes any sense? If so, can we have a commit permissions on mercurial
branch? Otherwise, we'll at least try to keep it in sync.

Cheers,
fijal

From gh at ghaering.de  Sat May 17 16:41:56 2008
From: gh at ghaering.de (=?UTF-8?B?R2VyaGFyZCBIw6RyaW5n?=)
Date: Sat, 17 May 2008 16:41:56 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <693bc9ab0805170612k4a86713cq54b8920bbafa3ea2@mail.gmail.com>
References: <482DE96C.8010504@gmail.com> <482E3475.9040102@ghaering.de>	
	<482E9105.50504@gmail.com> <482EB44D.7000303@ghaering.de>
	<693bc9ab0805170612k4a86713cq54b8920bbafa3ea2@mail.gmail.com>
Message-ID: <482EEEB4.5060506@ghaering.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Maciej Fijalkowski wrote:
>> That was more a proof-of-concept (*). Without deep ctypes wizardry I
>> couldn't make anything with callbacks from SQLite work reliably. That's
>> why create_function, create_aggregate and friends were removed. If it's
>> possible to make that work at all, it's beyond my understanding of ctypes.
>>
> 
> Hi Gerhard.
> 
> We tackled the issue by creating closures instead of py_object. This
> way it's saner and does not segfault for example on django test suite.
> We temporarily commited it to svn branch on codespeak, here:
> http://codespeak.net/svn/user/fijal/pysqlite2/, but we would rather
> prefer to keep it in your mercurial branch. Do you think our changes
> makes any sense? If so, can we have a commit permissions on mercurial
> branch? Otherwise, we'll at least try to keep it in sync.

I'll need to send the username / password in encrypted form. Do you have
a PGP key?

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFILu60dIO4ozGCH14RAjiAAKCO/M0vmNXoaCr1q0Ms/WC+h0zrRQCdHQoD
970UVPxRhm5sWGBgurcyIiU=
=K+3U
-----END PGP SIGNATURE-----

From fijall at gmail.com  Sun May 18 23:21:15 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sun, 18 May 2008 23:21:15 +0200
Subject: [pypy-dev] [pypy-svn] r54903 - in pypy/branch/win32port/pypy:
	config rlib rlib/test
In-Reply-To: <20080518174812.3D0D22A0180@codespeak.net>
References: <20080518174812.3D0D22A0180@codespeak.net>
Message-ID: <693bc9ab0805181421s4ea08ddree0b326ffb2c6be6@mail.gmail.com>

> +def test_translate():
> +    from pypy.translator.interactive import Translation
> +
> +    def func():
> +        poll({})
> +
> +    t = Translation(func)
> +    fc = t.compile_c([])

wuaa. Please use from pypy.translator.c.test.test_genc import compile
and use this one.

Cheers,
fijal

From gh at ghaering.de  Mon May 19 07:55:05 2008
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Mon, 19 May 2008 07:55:05 +0200
Subject: [pypy-dev] Running pysqlite on pypy
In-Reply-To: <482EEEB4.5060506@ghaering.de>
References: <482DE96C.8010504@gmail.com>
	<482E3475.9040102@ghaering.de>		<482E9105.50504@gmail.com>
	<482EB44D.7000303@ghaering.de>	<693bc9ab0805170612k4a86713cq54b8920bbafa3ea2@mail.gmail.com>
	<482EEEB4.5060506@ghaering.de>
Message-ID: <48311639.7070001@ghaering.de>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Gerhard H?ring wrote:
> Maciej Fijalkowski wrote:
>>> That was more a proof-of-concept (*). Without deep ctypes wizardry I
>>> couldn't make anything with callbacks from SQLite work reliably. That's
>>> why create_function, create_aggregate and friends were removed. If it's
>>> possible to make that work at all, it's beyond my understanding of ctypes.
>>>
>> Hi Gerhard.
> 
>> We tackled the issue by creating closures instead of py_object. This
>> way it's saner and does not segfault for example on django test suite.
>> We temporarily commited it to svn branch on codespeak, here:
>> http://codespeak.net/svn/user/fijal/pysqlite2/, but we would rather
>> prefer to keep it in your mercurial branch. Do you think our changes
>> makes any sense? If so, can we have a commit permissions on mercurial
>> branch? Otherwise, we'll at least try to keep it in sync.
> 
> I'll need to send the username / password in encrypted form. Do you have
> a PGP key?

I'll happily set up authentication on my public pysqlite repository so
you guys can have commit permissions. But I won't send sensitive
information in cleartext, so if you're still interested, do you have a
PGP key or anything?

- -- Gerhard
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFIMRY5dIO4ozGCH14RAqbzAJ46HF3VA4Nflq1UaGN5l4iLJwg+tQCgrHa7
yq3VYKKciAD4V3CdVljtiOA=
=bjaf
-----END PGP SIGNATURE-----

From bea at changemaker.nu  Tue May 20 13:47:26 2008
From: bea at changemaker.nu (Beatrice During)
Date: Tue, 20 May 2008 13:47:26 +0200
Subject: [pypy-dev] Europython 2008 - talks?
Message-ID: <4832BA4E.7030309@changemaker.nu>

Hi there

Thursday 22nd of May is the submission deadline for
EuroPython 2008 (http://www.europython.org/FrontPage, 7-12th of July).

Question 1: Are any pypy people going to submit talks?
Question 2: Is there a sprint interest? (3 days after the conference)

Just checking....

Cheers

Bea  

From fijall at gmail.com  Tue May 20 16:03:48 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 20 May 2008 16:03:48 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <4832BA4E.7030309@changemaker.nu>
References: <4832BA4E.7030309@changemaker.nu>
Message-ID: <693bc9ab0805200703w6c2fac08gef946b04b06583f@mail.gmail.com>

On Tue, May 20, 2008 at 1:47 PM, Beatrice During <bea at changemaker.nu> wrote:
> Hi there
>
> Thursday 22nd of May is the submission deadline for
> EuroPython 2008 (http://www.europython.org/FrontPage, 7-12th of July).

Wuaaa!!!

>
> Question 1: Are any pypy people going to submit talks?

Yes :)

> Question 2: Is there a sprint interest? (3 days after the conference)

Yes.

From jacob at openend.se  Tue May 20 16:36:47 2008
From: jacob at openend.se (Jacob =?iso-8859-1?q?Hall=E9n?=)
Date: Tue, 20 May 2008 16:36:47 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <4832BA4E.7030309@changemaker.nu>
References: <4832BA4E.7030309@changemaker.nu>
Message-ID: <200805201636.55610.jacob@openend.se>

On tisdagen den 20 maj 2008, Beatrice During wrote:
> Hi there
>
> Thursday 22nd of May is the submission deadline for
> EuroPython 2008 (http://www.europython.org/FrontPage, 7-12th of July).
>
> Question 1: Are any pypy people going to submit talks?

I have submitted a high level talk about PyPy - essentially the architecture 
overview I presented on the US tour, plus information about where we stand 
today.

Jacob

From theller at ctypes.org  Wed May 21 11:21:11 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 21 May 2008 11:21:11 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
 lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <20080520141729.3216716853A@codespeak.net>
References: <20080520141729.3216716853A@codespeak.net>
Message-ID: <4833E987.5070909@ctypes.org>

afa at codespeak.net schrieb:
> > Author: afa
> > Date: Tue May 20 16:17:22 2008
> > New Revision: 55005
> > 
> > Modified:
> >    pypy/branch/win32port/pypy/lib/_ctypes/builtin.py
> >    pypy/branch/win32port/pypy/lib/msvcrt.py
> >    pypy/branch/win32port/pypy/module/_rawffi/__init__.py
> >    pypy/branch/win32port/pypy/module/_rawffi/interp_rawffi.py
> >    pypy/branch/win32port/pypy/module/_rawffi/test/test__rawffi.py
> >    pypy/branch/win32port/pypy/rlib/libffi.py
> >    pypy/branch/win32port/pypy/rlib/rwin32.py
> >    pypy/branch/win32port/pypy/rlib/test/test_libffi.py
> > Log:
> > Maciej was right: it's better to dynamically get the name of the msvcrt library:
> > the same generated sources may be recompiled, e.g. in debug mode.
> > 
> > I hope I got it right: in pypy-c.exe, we compile a function that 
> > retrieves the dll containing the "fopen" pointer address.

> > +        #include <stdio.h>
> > +
> > +        /* Get the module where the "fopen" function resides in */
> > +        HANDLE get_libc_handle() {
> > +            MEMORY_BASIC_INFORMATION  mi;
> > +            memset(&mi, 0, sizeof(mi));
> > +
> > +            if( !VirtualQueryEx(GetCurrentProcess(), &fopen, &mi, sizeof(mi)) )
> > +                return 0;
> > +
> > +            return (HMODULE)mi.AllocationBase;
> > +        }
> > +        '''],

Ha, that's the best approach I have seen so far to get the name if the
msvcrt runtime library.  Hope you don't mind if I steal it for 'official'
ctypes (I will check if it works in win64 also before) ?

-- Thanks, Thomas

From fijall at gmail.com  Wed May 21 12:29:33 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 12:29:33 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
	lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <4833E987.5070909@ctypes.org>
References: <20080520141729.3216716853A@codespeak.net>
	<4833E987.5070909@ctypes.org>
Message-ID: <693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>

> Ha, that's the best approach I have seen so far to get the name if the
> msvcrt runtime library.  Hope you don't mind if I steal it for 'official'
> ctypes (I will check if it works in win64 also before) ?
>
> -- Thanks, Thomas

Cool :)

Besides, can you take a look at small script here:
http://codespeak.net/svn/pypy/dist/pypy/lib/ctypes_support.py? It
tries to address two issues - "how to find c library" and "how to get
errno", what do you think?

Cheers,
fijal

From theller at ctypes.org  Wed May 21 13:22:22 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 21 May 2008 13:22:22 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
 lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>
References: <20080520141729.3216716853A@codespeak.net>	
	<4833E987.5070909@ctypes.org>
	<693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>
Message-ID: <483405EE.7000908@ctypes.org>

Maciej Fijalkowski schrieb:
>> Ha, that's the best approach I have seen so far to get the name if the
>> msvcrt runtime library.  Hope you don't mind if I steal it for 'official'
>> ctypes (I will check if it works in win64 also before) ?
>>
>> -- Thanks, Thomas
> 
> Cool :)
> 
> Besides, can you take a look at small script here:
> http://codespeak.net/svn/pypy/dist/pypy/lib/ctypes_support.py? It
> tries to address two issues - "how to find c library" and "how to get
> errno", what do you think?
> 
> Cheers,
> fijal

Even cooler that I can read from day to day how you reimplemented
ctypes for pypy :-)

"how to find c library" - is only a problem on windows (which I hope can
now be solved better).  On linux (not sure about *nix) simply call
LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions
from all loaded shared libs from that handle.

"how to get errno" - the problem with this is that errno is only valid
(in the calling thread) until another stdlib library function has been
called in this thread.

So, since arbitrary code can run (in CPython's ctypes, at least) between
the time a function in a shared lib has been called and the time you access
errno there is no guarantee that it is still valid.
There has been some discussion on the ctypes-users list some months ago
about that, and the only reliable way to get errno for the function call
seems to be to get errno immediately after the function call.
The question is how to make this value available to the calling Python code.
Two ways were discussed: To attach the errno value in thread local storage
to the function object itself, or to pass it to the '.errcheck' function.
Nothing of this has yet been implemented in ctypes.

-- 
Thanks,
Thomas

From fijall at gmail.com  Wed May 21 15:38:00 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 15:38:00 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
	lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <483405EE.7000908@ctypes.org>
References: <20080520141729.3216716853A@codespeak.net>
	<4833E987.5070909@ctypes.org>
	<693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>
	<483405EE.7000908@ctypes.org>
Message-ID: <693bc9ab0805210638x37a82aa4oe3250dea7dce777f@mail.gmail.com>

> Even cooler that I can read from day to day how you reimplemented
> ctypes for pypy :-)
>
> "how to find c library" - is only a problem on windows (which I hope can
> now be solved better).  On linux (not sure about *nix) simply call
> LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions
> from all loaded shared libs from that handle.

No. That does not sound to me like a robust solution. (We should
probably move this discussion to ctypes-users though). Because you can
have strange stuff in there and I really want to access libc, not
something else. I think that helper in ctypes that gives you *exactly*
libc, not only on windows is a good thing to have.

>
> "how to get errno" - the problem with this is that errno is only valid
> (in the calling thread) until another stdlib library function has been
> called in this thread.
>
> So, since arbitrary code can run (in CPython's ctypes, at least) between
> the time a function in a shared lib has been called and the time you access
> errno there is no guarantee that it is still valid.
> There has been some discussion on the ctypes-users list some months ago
> about that, and the only reliable way to get errno for the function call
> seems to be to get errno immediately after the function call.
> The question is how to make this value available to the calling Python code.
> Two ways were discussed: To attach the errno value in thread local storage
> to the function object itself, or to pass it to the '.errcheck' function.
> Nothing of this has yet been implemented in ctypes.
>

Ah! Good :) I think I should start reading ctypes mailing list. We
have support for such stuff already, as we need to get errno when
testing posix stuff (I can explain in details if you like :)

It sounds like vastly better solution that ours, please implement it,
we'll follow :)

Cheers,
fijal

From fijall at gmail.com  Wed May 21 17:06:21 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 17:06:21 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <200805201636.55610.jacob@openend.se>
References: <4832BA4E.7030309@changemaker.nu>
	<200805201636.55610.jacob@openend.se>
Message-ID: <693bc9ab0805210806t65a67d57h678dfe6b2459b72e@mail.gmail.com>

From a brief discussion with people, I suggest the following:

1. Do an official pypy status talk, which will describe what cool
stuff you can do with pypy, where we are, where we're going and what
are our goals. (45 minutes incl questions)

2. Afterwards, do an architecture talk (I suggest 30 min including
questions), for those who want to dig deeper

3. Do some obscure talk. Rough idea is to give some details about
garbage collection strategies and what does it mean for various
operations (also 30 min).

We can do the last as BoF or Open Session, but a talk would be nicer I suppose.

I can do first and third, probably with someone.

Cheers,
fijal

On Tue, May 20, 2008 at 4:36 PM, Jacob Hall?n <jacob at openend.se> wrote:
> On tisdagen den 20 maj 2008, Beatrice During wrote:
>> Hi there
>>
>> Thursday 22nd of May is the submission deadline for
>> EuroPython 2008 (http://www.europython.org/FrontPage, 7-12th of July).
>>
>> Question 1: Are any pypy people going to submit talks?
>
> I have submitted a high level talk about PyPy - essentially the architecture
> overview I presented on the US tour, plus information about where we stand
> today.
>
> Jacob
> _______________________________________________
> pypy-dev at codespeak.net
> http://codespeak.net/mailman/listinfo/pypy-dev
>

From arigo at tunes.org  Wed May 21 17:36:26 2008
From: arigo at tunes.org (Armin Rigo)
Date: Wed, 21 May 2008 17:36:26 +0200
Subject: [pypy-dev] Performance Plots moved
Message-ID: <20080521153626.GA20888@code0.codespeak.net>

Hi all,

I moved the entry point url of the performance plots of tuatara
on tuatara itself, which makes some sense (they were on wyvern, and
wyvern is down currently (but should come back on-line one of these days
with a fresh new hard drive)).

The url is: http://tuatara.cs.uni-duesseldorf.de/plots.html


A bientot,

Armin.

From theller at ctypes.org  Wed May 21 18:48:33 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 21 May 2008 18:48:33 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
 lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <693bc9ab0805210638x37a82aa4oe3250dea7dce777f@mail.gmail.com>
References: <20080520141729.3216716853A@codespeak.net>	
	<4833E987.5070909@ctypes.org>	
	<693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>	
	<483405EE.7000908@ctypes.org>
	<693bc9ab0805210638x37a82aa4oe3250dea7dce777f@mail.gmail.com>
Message-ID: <48345261.2010202@ctypes.org>

Maciej Fijalkowski schrieb:
>> Even cooler that I can read from day to day how you reimplemented
>> ctypes for pypy :-)
>>
>> "how to find c library" - is only a problem on windows (which I hope can
>> now be solved better).  On linux (not sure about *nix) simply call
>> LoadLibrary(None) [=> dlopen(NULL) in C], and you can get all functions
>> from all loaded shared libs from that handle.
> 
> No. That does not sound to me like a robust solution. (We should
> probably move this discussion to ctypes-users though).

I've cc'd to ctypes.users.

> Because you can
> have strange stuff in there and I really want to access libc, not
> something else. I think that helper in ctypes that gives you *exactly*
> libc, not only on windows is a good thing to have.

That may be, but AFAICT it doesn't work this way on linux.  Once you have
a loaded shared libraries (in the current process), you can access *all*
exported functions from this process from it:

theller at tubu610:~$ python
Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from ctypes import *
>>> from ctypes.util import find_library
>>> m = CDLL(find_library("m"))
>>> m
<CDLL 'libm.so.6', handle b7f652b0 at b7d867ac>
>>> c = CDLL(find_library("c"))
>>> c
<CDLL 'libc.so.6', handle b7f65558 at b7d8606c>
>>> c.printf
<_FuncPtr object at 0xb7da41cc>
>>> m.printf
<_FuncPtr object at 0xb7da4234>
>>> c.printf("hello, world\n")
hello, world
13
>>> m.printf("hello, world\n")
hello, world
13
>>> find_library("c")
'libc.so.6'
>>> find_library("m")
'libm.so.6'
>>>

It may be possible to get different behaviour when the libraries are loaded
using different RTLD_ flags.

>>
>> "how to get errno" - the problem with this is that errno is only valid
>> (in the calling thread) until another stdlib library function has been
>> called in this thread.
>>
>> So, since arbitrary code can run (in CPython's ctypes, at least) between
>> the time a function in a shared lib has been called and the time you access
>> errno there is no guarantee that it is still valid.
>> There has been some discussion on the ctypes-users list some months ago
>> about that, and the only reliable way to get errno for the function call
>> seems to be to get errno immediately after the function call.
>> The question is how to make this value available to the calling Python code.
>> Two ways were discussed: To attach the errno value in thread local storage
>> to the function object itself, or to pass it to the '.errcheck' function.
>> Nothing of this has yet been implemented in ctypes.
>>
> 
> Ah! Good :) I think I should start reading ctypes mailing list. We
> have support for such stuff already, as we need to get errno when
> testing posix stuff (I can explain in details if you like :)
> 
> It sounds like vastly better solution that ours, please implement it,
> we'll follow :)

There's an entry in the python bug tracker at
http://bugs.python.org/issue1798
where you could add comments if you have any.

> Cheers,
> fijal


-- 
Thanks,
Thomas

From theller at ctypes.org  Wed May 21 18:52:25 2008
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 21 May 2008 18:52:25 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
 lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <20080520141729.3216716853A@codespeak.net>
References: <20080520141729.3216716853A@codespeak.net>
Message-ID: <g11k05$i5p$1@ger.gmane.org>

afa at codespeak.net schrieb:
> Author: afa
> Date: Tue May 20 16:17:22 2008
> New Revision: 55005
> 
> Modified:
>    pypy/branch/win32port/pypy/lib/_ctypes/builtin.py
>    pypy/branch/win32port/pypy/lib/msvcrt.py
>    pypy/branch/win32port/pypy/module/_rawffi/__init__.py
>    pypy/branch/win32port/pypy/module/_rawffi/interp_rawffi.py
>    pypy/branch/win32port/pypy/module/_rawffi/test/test__rawffi.py
>    pypy/branch/win32port/pypy/rlib/libffi.py
>    pypy/branch/win32port/pypy/rlib/rwin32.py
>    pypy/branch/win32port/pypy/rlib/test/test_libffi.py
> Log:
> Maciej was right: it's better to dynamically get the name of the msvcrt library:
> the same generated sources may be recompiled, e.g. in debug mode.
> 
> I hope I got it right: in pypy-c.exe, we compile a function that 
> retrieves the dll containing the "fopen" pointer address.
> 
> Hardcoded "libc.so.6" on posix platforms.
> There must be a better way.

May be that dl_iterate_phdr(3) can help?

Thomas


From fijall at gmail.com  Wed May 21 21:27:24 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 21:27:24 +0200
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
	lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <48345261.2010202@ctypes.org>
References: <20080520141729.3216716853A@codespeak.net>
	<4833E987.5070909@ctypes.org>
	<693bc9ab0805210329m43b9c4a9hdd554e33d2797e64@mail.gmail.com>
	<483405EE.7000908@ctypes.org>
	<693bc9ab0805210638x37a82aa4oe3250dea7dce777f@mail.gmail.com>
	<48345261.2010202@ctypes.org>
Message-ID: <693bc9ab0805211227n64155278k8d6dc1c9737a1e35@mail.gmail.com>

> I've cc'd to ctypes.users.

Thank you, subscribed

> theller at tubu610:~$ python
> Python 2.5.2 (r252:60911, Apr 21 2008, 11:12:42)
> [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> from ctypes import *
>>>> from ctypes.util import find_library
>>>> m = CDLL(find_library("m"))
>>>> m
> <CDLL 'libm.so.6', handle b7f652b0 at b7d867ac>
>>>> c = CDLL(find_library("c"))
>>>> c
> <CDLL 'libc.so.6', handle b7f65558 at b7d8606c>
>>>> c.printf
> <_FuncPtr object at 0xb7da41cc>
>>>> m.printf
> <_FuncPtr object at 0xb7da4234>
>>>> c.printf("hello, world\n")
> hello, world
> 13
>>>> m.printf("hello, world\n")
> hello, world
> 13
>>>> find_library("c")
> 'libc.so.6'
>>>> find_library("m")
> 'libm.so.6'
>>>>
>
> It may be possible to get different behaviour when the libraries are loaded
> using different RTLD_ flags.

yes, it depends on RTLD_GLOBAL vs RTLD_LOCAL. Note, that unless you
specify RTLD_GLOBAL, you don't get this behavior (RTLD_LOCAL is the
default for C and accidentally for pypy as well).

For me details on top of cpython are slightly different and it's very
platform dependent. (ie libm has no qsort, but libc has sqrt,
obscure).

I would rather like to keep it clean and not populate namespace if not
necessary, hence libc helper finder would be cool.

Thomas: thanks for feedback

Cheers,
fijal

From fijall at gmail.com  Wed May 21 21:36:00 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 21:36:00 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <693bc9ab0805210806t65a67d57h678dfe6b2459b72e@mail.gmail.com>
References: <4832BA4E.7030309@changemaker.nu>
	<200805201636.55610.jacob@openend.se>
	<693bc9ab0805210806t65a67d57h678dfe6b2459b72e@mail.gmail.com>
Message-ID: <693bc9ab0805211236t2b7fcbffw33e7b923473f797d@mail.gmail.com>

Due to short time, I'll just submit this structure. We can discuss and
change it probably further (but 3 slots sounds lik

On Wed, May 21, 2008 at 5:06 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> From a brief discussion with people, I suggest the following:
>
> 1. Do an official pypy status talk, which will describe what cool
> stuff you can do with pypy, where we are, where we're going and what
> are our goals. (45 minutes incl questions)
>
> 2. Afterwards, do an architecture talk (I suggest 30 min including
> questions), for those who want to dig deeper
>
> 3. Do some obscure talk. Rough idea is to give some details about
> garbage collection strategies and what does it mean for various
> operations (also 30 min).
>
> We can do the last as BoF or Open Session, but a talk would be nicer I suppose.
>
> I can do first and third, probably with someone.
>
> Cheers,
> fijal
>
> On Tue, May 20, 2008 at 4:36 PM, Jacob Hall?n <jacob at openend.se> wrote:
>> On tisdagen den 20 maj 2008, Beatrice During wrote:
>>> Hi there
>>>
>>> Thursday 22nd of May is the submission deadline for
>>> EuroPython 2008 (http://www.europython.org/FrontPage, 7-12th of July).
>>>
>>> Question 1: Are any pypy people going to submit talks?
>>
>> I have submitted a high level talk about PyPy - essentially the architecture
>> overview I presented on the US tour, plus information about where we stand
>> today.
>>
>> Jacob
>> _______________________________________________
>> pypy-dev at codespeak.net
>> http://codespeak.net/mailman/listinfo/pypy-dev
>>
>

From fijall at gmail.com  Wed May 21 21:37:34 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Wed, 21 May 2008 21:37:34 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <693bc9ab0805211236t2b7fcbffw33e7b923473f797d@mail.gmail.com>
References: <4832BA4E.7030309@changemaker.nu>
	<200805201636.55610.jacob@openend.se>
	<693bc9ab0805210806t65a67d57h678dfe6b2459b72e@mail.gmail.com>
	<693bc9ab0805211236t2b7fcbffw33e7b923473f797d@mail.gmail.com>
Message-ID: <693bc9ab0805211237j1992907dlec309fd7cadea4b9@mail.gmail.com>

Bah, sent in the middle

Due to short time, I'll just submit 2 more talks with this structure
(assuming that jacob reserved the "architecture" one). We can discuss
and
change it probably further (but 3 slots sounds like a reasonable
solution anyway).

Cheers,
fijal

From jacob at openend.se  Wed May 21 22:04:50 2008
From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Wed, 21 May 2008 22:04:50 +0200
Subject: [pypy-dev] Europython 2008 - talks?
In-Reply-To: <693bc9ab0805211237j1992907dlec309fd7cadea4b9@mail.gmail.com>
References: <4832BA4E.7030309@changemaker.nu>
	<693bc9ab0805211236t2b7fcbffw33e7b923473f797d@mail.gmail.com>
	<693bc9ab0805211237j1992907dlec309fd7cadea4b9@mail.gmail.com>
Message-ID: <200805212204.50610.jacob@openend.se>

onsdagen den 21 maj 2008 skrev Maciej Fijalkowski:
> Bah, sent in the middle
>
> Due to short time, I'll just submit 2 more talks with this structure
> (assuming that jacob reserved the "architecture" one). We can discuss
> and
> change it probably further (but 3 slots sounds like a reasonable
> solution anyway).
>
> Cheers,
> fijal

This is what I have submitted:

Title: PyPy for the rest of us - A lightweight introduction for people who 
want to understand the big picture

Target audience: intermediate (concerning Python), novice (concerning PyPy)

Theme: Python Language

Talk format: 30 minutes

Summary: PyPy is a new implementation of Python that has been worked on for 
about 4 years. It has been usable for a couple of years, but there hasn't 
been a compelling reason for Python users to use PyPy so far. We expect this 
to change in the next 6-12 months, so now is the time to find out what PyPy 
is about, This talk explains the big picture of what PyPy is and isn't, in a 
way that is accessible to everyone. If you want to understand how a JIT 
compiler generator works, this talk is not for you. If you want to understand 
where PyPy stands today and what hurdles remain to make your code go faster, 
you should attend.

Speaker: Jacob Hall?n

Biography: Jacob has been involved with the PyPy project since its first 
sprint. He has done a lot of auxilliary work like coding library modules, 
writing funding proposals and doing project management tasks. This means that 
he has had to bend his mind around the intricate results of the genius coders 
that surround him.

From scott+pypy-dev at scottdial.com  Wed May 21 23:44:14 2008
From: scott+pypy-dev at scottdial.com (Scott Dial)
Date: Wed, 21 May 2008 17:44:14 -0400
Subject: [pypy-dev] r55005 - in pypy/branch/win32port/pypy: lib
 lib/_ctypes module/_rawffi module/_rawffi/test rlib rlib/test
In-Reply-To: <g11k05$i5p$1@ger.gmane.org>
References: <20080520141729.3216716853A@codespeak.net>
	<g11k05$i5p$1@ger.gmane.org>
Message-ID: <483497AE.7070009@scottdial.com>

Thomas Heller wrote:
>> Hardcoded "libc.so.6" on posix platforms.
>> There must be a better way.
> 
> May be that dl_iterate_phdr(3) can help?

But, from the man page:

"The dl_iterate_phdr() function is Linux-specific and should be avoided 
in portable applications."

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From cfbolz at gmx.de  Fri May 23 12:18:12 2008
From: cfbolz at gmx.de (Carl Friedrich Bolz)
Date: Fri, 23 May 2008 12:18:12 +0200
Subject: [pypy-dev] video recordings S3
Message-ID: <483699E4.7030507@gmx.de>

Hi all,

seems that the video recordings of S3 are available:
http://www.swa.hpi.uni-potsdam.de/s3/program/

The talk slides of the talk I gave there can be found here:

http://codespeak.net/svn/pypy/extradoc/talk/s3-2008/talk.pdf

The paper that we submitted can be found here:

http://codespeak.net/svn/pypy/extradoc/paper/spy-s3.pdf


Cheers,

Carl Friedrich

From jacob at openend.se  Mon May 26 00:30:10 2008
From: jacob at openend.se (Jacob =?utf-8?q?Hall=C3=A9n?=)
Date: Mon, 26 May 2008 00:30:10 +0200
Subject: [pypy-dev] Development philosophy
Message-ID: <200805260030.10255.jacob@openend.se>

Hi everyone,

I have been following several discussions that have been going on in the past 
couple of weeks concering the speed of certain parts of the code and the size 
of the the code base. While I think the sentiments about the different things 
are absolutely right, I think that the limited resources that we have require 
us to make choices in what we do and when we do it.

My views are that:

- Apart from a few libraries, like socket() and I/O, it doesn't matter that 
the library code is extremely slow, for the time being. We want to find some 
application where we can make a big difference early, rather than try to be 
everything to everyone from the start. Things like zlib and cryptography can 
be very slow, as long as they are correct.

- For core language functionality, speed matters a lot. Apart from the JIT 
work, which still is a long term project, I can see a couple of useful 
avenues for improving speed. One is optimising calls, which Samuele has more 
thoughts about. The other one is to dig into the list operation 
microbenchmarks, where our performance is a lot worse than CPython 2.4. List 
operations are a very large part of most real Python applications (like all 
the zillion web frameworks) and improving them may increase our chances of 
adoption before the JIT work is usable.

- I agree that the size of the project in lines of code is getting 
problematic, but I think it would be a mistake to start doing anything about 
it at this point in time. We are still not doing well enough for completeness 
and correctness to start simplifying the code. we need to tackle one aspect 
at a time. Otherwise we won't be able to see if we are making progress or 
just making things worse.

Our planned work to make Django run is very much about completeness and 
correctness. Let's keep our focus there for a while.

Jacob
 

From simon at arrowtheory.com  Mon May 26 21:49:31 2008
From: simon at arrowtheory.com (Simon Burton)
Date: Mon, 26 May 2008 15:49:31 -0400
Subject: [pypy-dev] video recordings S3
In-Reply-To: <483699E4.7030507@gmx.de>
References: <483699E4.7030507@gmx.de>
Message-ID: <20080526154931.2338063b.simon@arrowtheory.com>

On Fri, 23 May 2008 12:18:12 +0200
Carl Friedrich Bolz <cfbolz at gmx.de> wrote:

> Hi all,
> 
> seems that the video recordings of S3 are available:
> http://www.swa.hpi.uni-potsdam.de/s3/program/
> 

Does this work for anyone ?

Simon.

From saluk64007 at gmail.com  Mon May 26 22:02:23 2008
From: saluk64007 at gmail.com (Patrick Mullen)
Date: Mon, 26 May 2008 13:02:23 -0700
Subject: [pypy-dev] video recordings S3
In-Reply-To: <20080526154931.2338063b.simon@arrowtheory.com>
References: <483699E4.7030507@gmx.de>
	<20080526154931.2338063b.simon@arrowtheory.com>
Message-ID: <c3f69bf60805261302i61b88a84m7879f83774cc647d@mail.gmail.com>

The files work for me, very enlightening series of talks.  I download and
play from media player classic.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://codespeak.net/pipermail/pypy-dev/attachments/20080526/d9f685c2/attachment.htm 

From simon at arrowtheory.com  Mon May 26 22:14:37 2008
From: simon at arrowtheory.com (Simon Burton)
Date: Mon, 26 May 2008 16:14:37 -0400
Subject: [pypy-dev] video recordings S3
In-Reply-To: <c3f69bf60805261302i61b88a84m7879f83774cc647d@mail.gmail.com>
References: <483699E4.7030507@gmx.de>
	<20080526154931.2338063b.simon@arrowtheory.com>
	<c3f69bf60805261302i61b88a84m7879f83774cc647d@mail.gmail.com>
Message-ID: <20080526161437.379e4b41.simon@arrowtheory.com>

On Mon, 26 May 2008 13:02:23 -0700
"Patrick Mullen" <saluk64007 at gmail.com> wrote:

> The files work for me, very enlightening series of talks.  I download and
> play from media player classic.
> 

Can you send me a direct link ? Maybe it's the browser getting confused.

thanks,

Simon.

From santagada at gmail.com  Thu Jun  5 04:49:57 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Wed, 4 Jun 2008 23:49:57 -0300
Subject: [pypy-dev] Problem on OOType for jspypy
Message-ID: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com>

Directly after annotation the targetjsstandalone.py gives an error on  
ootype. I cant see what might be the problem, and yes the code is  
translating to c fine. Here I am talking about the new code on a  
branch, but hopefully when antocuni and others in europe read this the  
code is going to be merged to trunk (if it is not wait a little for it).

the error is:
[translation:ERROR]  Exception: unexpected prebuilt constant:  
<staticmethod object at 0x5937b0>

but the full trace is on:
http://paste.pocoo.org/show/62193/

thanks and I will be tomorrow on the channel.

--
Leonardo Santagada







From anto.cuni at gmail.com  Thu Jun  5 11:38:27 2008
From: anto.cuni at gmail.com (Antonio Cuni)
Date: Thu, 05 Jun 2008 11:38:27 +0200
Subject: [pypy-dev] Problem on OOType for jspypy
In-Reply-To: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com>
References: <4A239E04-1A9A-4FF1-B9DC-B3877D0B7503@gmail.com>
Message-ID: <4847B413.6070803@gmail.com>

Leonardo Santagada wrote:
> Directly after annotation the targetjsstandalone.py gives an error on  
> ootype. I cant see what might be the problem, and yes the code is  
> translating to c fine. Here I am talking about the new code on a  
> branch, but hopefully when antocuni and others in europe read this the  
> code is going to be merged to trunk (if it is not wait a little for it).
> 
> the error is:
> [translation:ERROR]  Exception: unexpected prebuilt constant:  
> <staticmethod object at 0x5937b0>
> 
> but the full trace is on:
> http://paste.pocoo.org/show/62193/
> 
> thanks and I will be tomorrow on the channel.

staticmethods are not really supported by rpython. I would say they work 
by chance with lltype, but you shouldn't use them.


ciao,
Anto

From yunmao at gmail.com  Sat Jun  7 22:30:51 2008
From: yunmao at gmail.com (Yun Mao)
Date: Sat, 7 Jun 2008 16:30:51 -0400
Subject: [pypy-dev] dead loop compilation
In-Reply-To: <mailman.2651.1212870661.18038.pypy-dev@codespeak.net>
References: <mailman.2651.1212870661.18038.pypy-dev@codespeak.net>
Message-ID: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com>

Hi, I was playing with pypy svn branch, and one of the first functions
i tried was the following:

def f():
   while 1: pass

it looks like pypy was trying to evaluate this and has gone dead too.
Is this what I suppose to see? thanks,

Yun

From santagada at gmail.com  Sat Jun  7 23:43:56 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Sat, 7 Jun 2008 18:43:56 -0300
Subject: [pypy-dev] dead loop compilation
In-Reply-To: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com>
References: <mailman.2651.1212870661.18038.pypy-dev@codespeak.net>
	<7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com>
Message-ID: <2A44F6E4-696C-41AA-8EBC-902D1DD4DB80@gmail.com>


On 07/06/2008, at 17:30, Yun Mao wrote:

> Hi, I was playing with pypy svn branch, and one of the first functions
> i tried was the following:
>
> def f():
>   while 1: pass
>
> it looks like pypy was trying to evaluate this and has gone dead too.
> Is this what I suppose to see? thanks,


I asked about it on the #pypy channel on irc.freenode.net and the  
answer is that this is a known issue but won't be fixed. The reason  
for that is that rpython is a tool to write interpreters and that is  
not a useful feature (to be able to create infinite loops).

I ask you to come to the irc channel and chat about what you are  
trying to do with pypy and someone there will give you some advice.


See you there,
--
Leonardo Santagada




From arigo at tunes.org  Mon Jun  9 11:01:12 2008
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 9 Jun 2008 11:01:12 +0200
Subject: [pypy-dev] dead loop compilation
In-Reply-To: <7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com>
References: <mailman.2651.1212870661.18038.pypy-dev@codespeak.net>
	<7cffadfa0806071330h5c9b246fs5de6fa765888829d@mail.gmail.com>
Message-ID: <20080609090112.GA4711@code0.codespeak.net>

Hi,

On Sat, Jun 07, 2008 at 04:30:51PM -0400, Yun Mao wrote:
> Hi, I was playing with pypy svn branch, and one of the first functions
> i tried was the following:
> 
> def f():
>    while 1: pass
> 
> it looks like pypy was trying to evaluate this and has gone dead too.
> Is this what I suppose to see? thanks,

We need a bit more context to know where you are trying to put this
code.  I tried to type it in a "pypy-c" executable -- is that what you
have in mind?

In "pypy-c" I can interrupt the infinite loop with Ctrl-C, which is what
I expected.  (To get this result you may need a "pypy-c" translated with
the '--allworkingmodules' option, otherwise I'm not sure signal handling
is enabled.)

If you mean that you tried to put an infinite loop in an RPython
function and translate that, then no chance.  The translation toolchain
is not really good with strange inputs: it only does something sane if
the input is sane (which means, an RPython that actually works and was
reasonably well tested first).


A bientot,

Armin

From fijall at gmail.com  Mon Jun  9 21:05:58 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 9 Jun 2008 21:05:58 +0200
Subject: [pypy-dev] django performance analysis
Message-ID: <693bc9ab0806091205u1dc73dafxbbb1784138097604@mail.gmail.com>

Analysis of pypy-c --oldstyle runtests.py --settings=settings -v 2
using new _lsprof module
These are analysis of a single test suite (test_client). In general,
ctypes costs are bigger (due to teardown/setup which takes most of
time).

http://codespeak.net/~fijal/django-calltree.png

Hereby I'll try to rewrite parts of ctypes (especially calling and
some basic objects) in RPython.

PS. I can upload kcachegrind input file if someone is interested.

From fijall at gmail.com  Tue Jun 10 07:41:03 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 10 Jun 2008 07:41:03 +0200
Subject: [pypy-dev] spotting pypy segfault
Message-ID: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>

I think this is the one about finalizers, because they raise errors,
but that might also be something with threads or anything, I don't
know. What to do:
1. Have pypy-c with threads (hybrid gc, faassen, threads,
allworkingmodules for me, svn v 55678)
2. Check out sqlalchemy

svn checkout http://svn.sqlalchemy.org/sqlalchemy/trunk sqlalchemy

go inside this directory (sqlalchemy), and run:

PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py

my gdb got confused, but hopefully you'll have better chance.

eventually, if this doesn't work (like you have different sqlalchemy
installed), do (inside sqlalchemy dir) python2.4 setup.py develop,
eventually as root depending on your system conf.

Cheers and have fun debugging,
fijal

From fijall at gmail.com  Tue Jun 10 07:50:22 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 10 Jun 2008 07:50:22 +0200
Subject: [pypy-dev] spotting pypy segfault
In-Reply-To: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>
References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>
Message-ID: <693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com>

It's a bit worse. It's infinite rec on C level somewhat. How did that
happen? A bug in our cycle analysis? Short part from gdb trace is
like:

#393 0x08cabd32 in pypy_g_W_Super_getattribute (l_self_19118=0xb72f37d8,
    l_name_136=0x97c3144) at implement_7.c:35032
#394 0x08bcf5c5 in pypy_g_BuiltinActivation_UwS_W_Super_ObjSpace_str__run_ (
    l_self_670=0x983ae94, l_scope_w_55=0xb71de804) at implement_6.c:174279
#395 0x081e9510 in pypy_g_BuiltinCode_funcrun (l_self_174=0x980a040,
    l_func_8=0x97ef6c0, l_args_28=0xb71de7ec) at implement.c:233924
#396 0x0805ff37 in pypy_g_funccall_star2 (l_self_2634=0x97ef6c0,
    l_stararg0_6=0xb72f37d8, l_stararg1_1=0xb71de7c4) at implement.c:11482
#397 0x0807f7f1 in pypy_g_get_and_call_function_star1 (l_w_descr_13=0x97ef6c0,
    l_w_obj_846=0xb72f37d8, l_stararg0_10=0xb71de7c4) at implement.c:30312
#398 0x0807dafd in pypy_g__handle_getattribute (l_w_descr_2=0x97ef6c0,
    l_w_obj_845=0xb72f37d8, l_w_name_60=0xb71de7c4) at implement.c:29216
#399 0x08062baf in pypy_g_getattr (l_w_obj_841=0xb72f37d8,
    l_w_name_59=0xb71de7c4) at implement.c:13316
#400 0x08cabd32 in pypy_g_W_Super_getattribute (l_self_19118=0xb72f37d8,
    l_name_136=0x97c3144) at implement_7.c:35032

How did that happen?

Cheers,
fijal

From fijall at gmail.com  Tue Jun 10 07:54:42 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 10 Jun 2008 07:54:42 +0200
Subject: [pypy-dev] spotting pypy segfault
In-Reply-To: <693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com>
References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>
	<693bc9ab0806092250p788958d6q46ba824fb8fd726c@mail.gmail.com>
Message-ID: <693bc9ab0806092254h409160bcifbd59f592bb5d38f@mail.gmail.com>

Argh! I think I know how that happened. Suppose we have a cycle, which
allocates n bytes on each cycle. So on each iteration we check if
stack is at least size of n. But let's suppose we have somewhere along
the cycle path of length m > n which returns. It means that our stack
check might not catch it since we have n bytes left. But we actually
need m, boom.
(Traceback resembles a bit this situation, on the very top is stuff
that is slightly different than repeating pattern).

Cheers,
fijal

From arigo at tunes.org  Tue Jun 10 12:19:45 2008
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 10 Jun 2008 12:19:45 +0200
Subject: [pypy-dev] spotting pypy segfault
In-Reply-To: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>
References: <693bc9ab0806092241x57b8d1e5u3abaecad3677c062@mail.gmail.com>
Message-ID: <20080610101944.GA9646@code0.codespeak.net>

Hi Maciej,

On Tue, Jun 10, 2008 at 07:41:03AM +0200, Maciej Fijalkowski wrote:
> PYTHONPATH=test gdb --args pypy-c --oldstyle test/ext/orderinglist.py

ImportError: No module named pysqlite2._sqlite

Where can we get the current version of sqlite-ctypes?


Armin

From fijall at gmail.com  Tue Jun 10 17:34:24 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 10 Jun 2008 17:34:24 +0200
Subject: [pypy-dev] spotting pypy segfa