From santagada at gmail.com Thu Oct 5 07:47:12 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 5 Oct 2006 02:47:12 -0300 Subject: [pypy-dev] PyParser Message-ID: <2f2e5f950610042247y61b6225ep3df5aaf7b1755770@mail.gmail.com> As some of you already know I will start to make the javascript interpreter. I am thinking about using the pyparser module that is in pypy, does someone knows if it is still working? Some people on the #pypy were talking about a modified ply that generates RPython, if someones knows about that also would be nice. Also the documentation of the pyparser module seems to not be in a good shape, I could do something about it if it is desired/allowed :) -- Leonardo Santagada (http://www.lomohomes.com/retype) N?o se preocupe com o que os outros v?o fazer. O melhor jeito de prever o futuro ? inventa-lo. - Alan Kay From jdoda at sympatico.ca Fri Oct 6 18:28:11 2006 From: jdoda at sympatico.ca (Jonathan Doda) Date: Fri, 06 Oct 2006 12:28:11 -0400 Subject: [pypy-dev] benchmarking pypy with pybench Message-ID: <4526841B.4040206@sympatico.ca> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - From what I understand, neither pystone nor richards are very representative of idiomatic python, but pybench is (as much as is possible for a synthetic benchmark, in any case). So, I thought it would be interesting to see how pypy compared to cpython using pybench. Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench reports details of the machine and python installation using the various functions in the platform module, but these functions don't appear to work in pypy, so the reporting was disabled. The UnicodeProperties test also had to be disabled due to pypy not having a unicodedata module. All other tests ran successfully, and pybench was otherwise unmodified. pypy revision 32952 was used, but pypy was translated using the default options, so I imagine these results could be improved on. The tests are run ten times, the first group of three columns shows the best times, and the second group shows the average times. Within each group, the first column is the pypy time, the second the python 2.4.3 time, and the third is the ratio. I hope these results are of interest, and if anyone has any advice on how to improve the test setup, or translation time options I should try, please let me know. Jonathan Doda Test minimum average - ------------------------------------------------------------------------ BuiltinFunctionCalls: 343ms 178ms 1.93 368ms 179ms 2.06 BuiltinMethodLookup: 554ms 161ms 3.44 567ms 174ms 3.26 CompareFloats: 285ms 172ms 1.66 295ms 173ms 1.71 CompareFloatsIntegers: 1492ms 171ms 8.73 1576ms 171ms 9.22 CompareIntegers: 410ms 160ms 2.56 425ms 160ms 2.66 CompareInternedStrings: 414ms 164ms 2.52 421ms 165ms 2.55 CompareLongs: 251ms 137ms 1.83 261ms 138ms 1.89 CompareStrings: 279ms 145ms 1.92 293ms 146ms 2.01 CompareUnicode: 315ms 145ms 2.17 320ms 146ms 2.19 ConcatStrings: 2230ms 201ms 11.09 2251ms 209ms 10.77 ConcatUnicode: 5515ms 209ms 26.39 5570ms 213ms 26.15 CreateInstances: 953ms 180ms 5.29 1000ms 189ms 5.29 CreateNewInstances: 712ms 165ms 4.32 746ms 166ms 4.49 CreateStringsWithConcat: 1067ms 175ms 6.10 1078ms 178ms 6.06 CreateUnicodeWithConcat: 905ms 167ms 5.42 919ms 175ms 5.25 DictCreation: 661ms 136ms 4.86 686ms 136ms 5.04 DictWithFloatKeys: 811ms 191ms 4.25 826ms 192ms 4.30 DictWithIntegerKeys: 482ms 154ms 3.13 493ms 155ms 3.18 DictWithStringKeys: 584ms 143ms 4.08 592ms 144ms 4.11 ForLoops: 898ms 150ms 5.99 905ms 152ms 5.95 IfThenElse: 388ms 135ms 2.87 396ms 136ms 2.91 ListSlicing: 423ms 106ms 3.99 454ms 107ms 4.24 NestedForLoops: 1137ms 176ms 6.46 1141ms 178ms 6.41 NormalClassAttribute: 559ms 156ms 3.58 567ms 157ms 3.61 NormalInstanceAttribute: 935ms 143ms 6.54 945ms 144ms 6.56 PythonFunctionCalls: 1048ms 133ms 7.88 1057ms 134ms 7.89 PythonMethodCalls: 1275ms 168ms 7.59 1285ms 169ms 7.60 Recursion: 1599ms 185ms 8.64 1607ms 186ms 8.64 SecondImport: 685ms 137ms 5.00 694ms 138ms 5.03 SecondPackageImport: 602ms 146ms 4.12 618ms 149ms 4.15 SecondSubmoduleImport: 987ms 179ms 5.51 996ms 182ms 5.47 SimpleComplexArithmetic: 745ms 197ms 3.78 750ms 201ms 3.73 SimpleDictManipulation: 668ms 151ms 4.42 670ms 153ms 4.38 SimpleFloatArithmetic: 396ms 172ms 2.30 407ms 182ms 2.24 SimpleIntFloatArithmetic: 363ms 131ms 2.77 376ms 133ms 2.83 SimpleIntegerArithmetic: 367ms 132ms 2.78 377ms 133ms 2.83 SimpleListManipulation: 384ms 121ms 3.17 399ms 123ms 3.24 SimpleLongArithmetic: 651ms 144ms 4.52 660ms 145ms 4.55 SmallLists: 371ms 140ms 2.65 372ms 142ms 2.62 SmallTuples: 2122ms 148ms 14.34 2144ms 150ms 14.29 SpecialClassAttribute: 545ms 154ms 3.54 573ms 155ms 3.70 SpecialInstanceAttribute: 939ms 182ms 5.16 971ms 184ms 5.28 StringMappings: 517ms 170ms 3.04 539ms 172ms 3.13 StringPredicates: 491ms 160ms 3.07 518ms 168ms 3.08 StringSlicing: 994ms 154ms 6.45 1039ms 160ms 6.49 TryExcept: 424ms 156ms 2.72 444ms 157ms 2.83 TryRaiseExcept: 666ms 182ms 3.66 687ms 184ms 3.73 TupleSlicing: 521ms 163ms 3.20 547ms 165ms 3.32 UnicodeMappings: 389ms 168ms 2.32 403ms 170ms 2.37 UnicodePredicates: 433ms 152ms 2.85 449ms 157ms 2.86 UnicodeSlicing: 1874ms 180ms 10.41 1922ms 186ms 10.33 Totals: 42658ms 8125ms 5.25 43595ms 8261ms 5.28 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.2.2 (GNU/Linux) iD8DBQFFJoQbWEgeeLQ8a74RAh36AJ9J0MwiaaM6aqAnvQidrTCCeSyKMQCfWvRZ vyz4YzrY0oEYzaAgBd9Ny+0= =Qr6G -----END PGP SIGNATURE----- From cfbolz at gmx.de Fri Oct 6 20:15:49 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 06 Oct 2006 20:15:49 +0200 Subject: [pypy-dev] benchmarking pypy with pybench In-Reply-To: <4526841B.4040206@sympatico.ca> References: <4526841B.4040206@sympatico.ca> Message-ID: <45269D55.3080307@gmx.de> Hi Jonathan! Thanks for doing this, we used pybench a while ago but didn't pursue if further. This is indeed very interesting! Jonathan Doda wrote: > - From what I understand, neither pystone nor richards are very > representative of idiomatic python, but pybench is (as much as is > possible for a synthetic benchmark, in any case). So, I thought it would > be interesting to see how pypy compared to cpython using pybench. > > Pybench 2.0 was used, downloaded from the python.org svn repo. Pybench > reports details of the machine and python installation using the various > functions in the platform module, but these functions don't appear to > work in pypy, so the reporting was disabled. The UnicodeProperties test > also had to be disabled due to pypy not having a unicodedata module. All > other tests ran successfully, and pybench was otherwise unmodified. PyPy has a unicodedata module, it is not compiled in by default, though. It's not clear to me why the platform module does not work, though. > pypy revision 32952 was used, but pypy was translated using the default > options, so I imagine these results could be improved on. > > The tests are run ten times, the first group of three columns shows the > best times, and the second group shows the average times. Within each > group, the first column is the pypy time, the second the python 2.4.3 > time, and the third is the ratio. > > I hope these results are of interest, and if anyone has any advice on > how to improve the test setup, or translation time options I should try, > please let me know. You could try to pass in some translation-options that try to optimize common string operations and small integers: python translate.py targetpypystandalone.py --objspace-std-withsmallint --objspace-std-withstrslice --objspace-std-withstrjoin Other interesting options are --objspace-std-withstrdict (to enable optimizations for dictionaries with string keys), --usemodules=unicodedata (to enable the unicode data module). The options are mutually compatible, so you can do a super-build with all of them (which unfortunately increases executable size quite a bit). Cheers, Carl Friedrich Bolz From simon at arrowtheory.com Sun Oct 8 21:50:20 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 8 Oct 2006 12:50:20 -0700 Subject: [pypy-dev] py.test bug Message-ID: <20061008125020.72f527e0.simon@arrowtheory.com> Traceback (most recent call last): File "/home/simon/bin/py.test", line 4, in ? py.test.cmdline.main() File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main failures = session.main(args) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main return super(TerminalSession, self).main(args) File "/home/simon/local/pypy/py/test/session.py", line 52, in main self.footer(colitems) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer self.failures() File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures self.repr_failure(colitem, outcome) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure handler(item, excinfo, traceback) File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong firstsourceline = entry.getfirstlinesource() AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' Simon. From fijal at genesilico.pl Sun Oct 8 22:55:44 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 08 Oct 2006 22:55:44 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008125020.72f527e0.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> Message-ID: <452965D0.8070800@genesilico.pl> Simon Burton wrote: >Traceback (most recent call last): > File "/home/simon/bin/py.test", line 4, in ? > py.test.cmdline.main() > File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main > failures = session.main(args) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main > return super(TerminalSession, self).main(args) > File "/home/simon/local/pypy/py/test/session.py", line 52, in main > self.footer(colitems) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer > self.failures() > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures > self.repr_failure(colitem, outcome) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure > handler(item, excinfo, traceback) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong > firstsourceline = entry.getfirstlinesource() >AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' > > >Simon. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > You seem to have old version of py.test. Please update to svn trunk. From fijal at genesilico.pl Sun Oct 8 22:59:19 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 08 Oct 2006 22:59:19 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008125020.72f527e0.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> Message-ID: <452966A7.7080309@genesilico.pl> Simon Burton wrote: >Traceback (most recent call last): > File "/home/simon/bin/py.test", line 4, in ? > py.test.cmdline.main() > File "/home/simon/local/pypy/py/test/cmdline.py", line 17, in main > failures = session.main(args) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 34, in main > return super(TerminalSession, self).main(args) > File "/home/simon/local/pypy/py/test/session.py", line 52, in main > self.footer(colitems) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 167, in footer > self.failures() > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 262, in failures > self.repr_failure(colitem, outcome) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 280, in repr_failure > handler(item, excinfo, traceback) > File "/home/simon/local/pypy/py/test/terminal/terminal.py", line 297, in repr_failure_tblong > firstsourceline = entry.getfirstlinesource() >AttributeError: 'TracebackEntry' object has no attribute 'getfirstlinesource' > > >Simon. >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > Ugh. That seems that I'm the one with old version of py.test :-( From simon at arrowtheory.com Mon Oct 9 01:48:09 2006 From: simon at arrowtheory.com (Simon Burton) Date: Sun, 8 Oct 2006 16:48:09 -0700 Subject: [pypy-dev] py.test bug In-Reply-To: <452966A7.7080309@genesilico.pl> References: <20061008125020.72f527e0.simon@arrowtheory.com> <452966A7.7080309@genesilico.pl> Message-ID: <20061008164809.099459a3.simon@arrowtheory.com> On Sun, 08 Oct 2006 22:59:19 +0200 Maciek Fijalkowski wrote: > > > > Ugh. That seems that I'm the one with old version of py.test :-( I think i fixed it.. (it's been commited) Simon. From anthony at interlink.com.au Mon Oct 9 12:42:07 2006 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon, 9 Oct 2006 20:42:07 +1000 Subject: [pypy-dev] benchmarking pypy with pybench In-Reply-To: <45269D55.3080307@gmx.de> References: <4526841B.4040206@sympatico.ca> <45269D55.3080307@gmx.de> Message-ID: <200610092042.09163.anthony@interlink.com.au> On Saturday 07 October 2006 04:15, Carl Friedrich Bolz wrote: > PyPy has a unicodedata module, it is not compiled in by default, though. > It's not clear to me why the platform module does not work, though. 'platform.py' also doesn't work with IronPython. I posted a patch on SF for this - www.python.org/sf/1563842 There's another patch with pybench fixes for IronPython that might be related. Don't have the patch# right now. Making it work with pypy is probably worthwhile, too. Someone working on the project might want to add a note to the patch. -- Anthony Baxter It's never too late to have a happy childhood. From arigo at tunes.org Tue Oct 10 00:52:17 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 Oct 2006 00:52:17 +0200 Subject: [pypy-dev] py.test bug In-Reply-To: <20061008164809.099459a3.simon@arrowtheory.com> References: <20061008125020.72f527e0.simon@arrowtheory.com> <452966A7.7080309@genesilico.pl> <20061008164809.099459a3.simon@arrowtheory.com> Message-ID: <20061009225217.GA14831@code0.codespeak.net> Hi Simon, On Sun, Oct 08, 2006 at 04:48:09PM -0700, Simon Burton wrote: > > Ugh. That seems that I'm the one with old version of py.test :-( > > I think i fixed it.. (it's been commited) Sorry for the trouble, I forgot to checkin code.py together with the rest of my changes. A bientot, Armin From jacob at strakt.com Wed Oct 11 01:53:04 2006 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Wed, 11 Oct 2006 01:53:04 +0200 Subject: [pypy-dev] Slight worries Message-ID: <200610110153.07659.jacob@strakt.com> I try to follow the discussions on IRC regularly, and I have some worries after what I have seen in the last couple of weeks. There seem to be too many problems with broken checkins. The translation breaks, tests stop working etc. While I understand that running tests takes an enormous amount of time, it worries me that we have to fix things on the trunk so very often. I think this eats up time that could be better spent. If people need to leave testing for automated tests, there ought to be a less disruptive way of doing this. Of course I may be wrong, and this is really the optimal way of moving forward, but the I would like to be reassured that it is that way. A second thing that is bothering me is that there does not seem to be a canonical list of software that our development system is designed to work with. I feel that this is important. It is difficult enough to make things work with a specific set of tools. Currently we should be making sure that things work with one set of tools and the generalization to various different versions of pygame, grapviz, gcc and whatnot can happen later. I know this is not quite attainable, since preferred versions of the various tools don't work on all platforms, but I hope you see the core point. Trying to make things work with all possible combinations of peripheral software will just generate lots and lots of work. We need a public stance on what we support. Jacob Hall?n From exarkun at divmod.com Wed Oct 11 02:41:17 2006 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Tue, 10 Oct 2006 20:41:17 -0400 Subject: [pypy-dev] Slight worries In-Reply-To: <200610110153.07659.jacob@strakt.com> Message-ID: <20061011004117.1717.1169292187.divmod.quotient.73138@ohm> On Wed, 11 Oct 2006 01:53:04 +0200, Jacob Hall?n wrote: >I try to follow the discussions on IRC regularly, and I have some worries >after what I have seen in the last couple of weeks. > >There seem to be too many problems with broken checkins. The translation >breaks, tests stop working etc. While I understand that running tests takes >an enormous amount of time, it worries me that we have to fix things on the >trunk so very often. I think this eats up time that could be better spent. > >If people need to leave testing for automated tests, there ought to be a less >disruptive way of doing this. > >Of course I may be wrong, and this is really the optimal way of moving >forward, but the I would like to be reassured that it is that way. I'd like to echo this sentement. It's really amazing how much progress the PyPy project has made over the last year or so, but it's frustrating to never be sure if updating to the latest revision is going to eliminate my ability to do anything with it. The translation process is currently immensely expensive, so I can certainly understand that people want to commit to trunk without doing extensive testing (I can imagine such testing taking up an entire day or more in some cases). But as well all know, the introduction of regressions into trunk comes with costs too. I'm not going to suggest any specific course of action, because I'm confident everyone involved with the PyPy project can recognize this problem and consider ways to address it. Howeverr, if there's anything I can do to help come up with or enact a solution, I offer what assistance I can. For starters, if it would be helpful, I may be able to offer some hardware to run the automated test suite. This aside, PyPy has come a long way, and everyone involved should be very proud of what you guys have created. I have confidence that these growing pains will soon be behind the project. > >A second thing that is bothering me is that there does not seem to be a >canonical list of software that our development system is designed to work >with. I feel that this is important. It is difficult enough to make things >work with a specific set of tools. Currently we should be making sure that >things work with one set of tools and the generalization to various different >versions of pygame, grapviz, gcc and whatnot can happen later. I know this is >not quite attainable, since preferred versions of the various tools don't >work on all platforms, but I hope you see the core point. Trying to make >things work with all possible combinations of peripheral software will just >generate lots and lots of work. We need a public stance on what we support. > >Jacob Hall?n Jean-Paul From fijal at genesilico.pl Thu Oct 12 19:58:29 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 12 Oct 2006 19:58:29 +0200 Subject: [pypy-dev] [pypy-svn] r33236 - pypy/dist/pypy/objspace/test In-Reply-To: <20061012172440.DAAAB1007C@code0.codespeak.net> References: <20061012172440.DAAAB1007C@code0.codespeak.net> Message-ID: <452E8245.1090601@genesilico.pl> auc at codespeak.net wrote: >Author: auc >Date: Thu Oct 12 19:24:38 2006 >New Revision: 33236 > >Modified: > pypy/dist/pypy/objspace/test/test_logicobjspace.py >Log: >some needed adjustement (most notably, removed explicit scheduling calls) > > >Modified: pypy/dist/pypy/objspace/test/test_logicobjspace.py >============================================================================== >--- pypy/dist/pypy/objspace/test/test_logicobjspace.py (original) >+++ pypy/dist/pypy/objspace/test/test_logicobjspace.py Thu Oct 12 19:24:38 2006 >@@ -6,6 +6,11 @@ > # we might be called from _test_logic_build > # if not, check your paths > >+try: >+ is_interpreted() >+except: >+ def is_interpreted(): return True >+ > > > try: from pypy.conftest import gettestobjspace from py.test import skip except ImportError: pass # we might be called from _test_logic_build # if not, check your paths try: is_interpreted() except: def is_interpreted(): return True This is really cool kind of code. If we cannot import pypy, # check your paths is in comment (no print, nothing). If we cannot import py.test, we probably won't have is_interpreted, so if is_interpreted(): py.test.skip("dsa") will just break. And simply calling py.test test_logicobjspace.py fails. (It should skip if there are no blablabla available). This attempts of having broken tests makes py.test work harder (because I always have to check if tests are broken by default or not). From aurelien.campeas at logilab.fr Fri Oct 13 09:43:41 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Fri, 13 Oct 2006 09:43:41 +0200 Subject: [pypy-dev] [pypy-svn] r33236 - pypy/dist/pypy/objspace/test In-Reply-To: <452E8245.1090601@genesilico.pl> References: <20061012172440.DAAAB1007C@code0.codespeak.net> <452E8245.1090601@genesilico.pl> Message-ID: <20061013074341.GA19980@crater.logilab.fr> On Thu, Oct 12, 2006 at 07:58:29PM +0200, Maciek Fijalkowski wrote: > auc at codespeak.net wrote: > > >Author: auc > >Date: Thu Oct 12 19:24:38 2006 > >New Revision: 33236 > > > >Modified: > > pypy/dist/pypy/objspace/test/test_logicobjspace.py > >Log: > >some needed adjustement (most notably, removed explicit scheduling calls) > > > > > >Modified: pypy/dist/pypy/objspace/test/test_logicobjspace.py > >============================================================================== > >--- pypy/dist/pypy/objspace/test/test_logicobjspace.py (original) > >+++ pypy/dist/pypy/objspace/test/test_logicobjspace.py Thu Oct 12 19:24:38 2006 > >@@ -6,6 +6,11 @@ > > # we might be called from _test_logic_build > > # if not, check your paths > > > >+try: > >+ is_interpreted() > >+except: > >+ def is_interpreted(): return True > >+ > > > > > > > try: > from pypy.conftest import gettestobjspace > from py.test import skip > except ImportError: > pass > # we might be called from _test_logic_build > # if not, check your paths > > try: > is_interpreted() > except: > def is_interpreted(): return True > > This is really cool kind of code. If we cannot import pypy, # check your > paths is in comment (no print, nothing). If we cannot import py.test, we > probably won't have is_interpreted, so if is_interpreted(): Wrong. Ok this deserves an explanation : whenever we want to test a pypy logic build, using _test_logic_build.py, is_interpreted is defined and returns False ... (yes it is not too explicit nor pretty). > py.test.skip("dsa") will just break. And simply calling py.test > test_logicobjspace.py fails. (It should skip if there are no blablabla > available). What are you trying to do ? > > This attempts of having broken tests makes py.test work harder (because > I always have to check if tests are broken by default or not). > Please change the tone of your remarks. From niko at alum.mit.edu Fri Oct 13 11:45:49 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 13 Oct 2006 11:45:49 +0200 Subject: [pypy-dev] CLI tests on Mac OS X Message-ID: Hey, I am wondering whether anyone has had success running the CLI tests on Mac OS X? I am having a problem where the tests seem to lock up in random places. I have tried letting them run all night without any progress. In each case, "top" reveals that the mono process is sucking up 60-80% of my CPU time, and if I kill the mono process then the test fails but the test process continues. When I run the tests on my linux machine, it seems to work --- however, since I use the mac for most of my development, it is a bit of pain to move the patches back and forth for testing, especially if new files are involved. I have mono installed through DarwinPorts, and it claims to be version 1.1.16.1. This is on Mac OS X 10.4, and using Python 2.4 (also from DarwinPorts). Has anyone else seen similar behavior or know a solution? thanks! Niko From cfbolz at gmx.de Fri Oct 13 13:02:50 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 13 Oct 2006 13:02:50 +0200 Subject: [pypy-dev] [pypy-svn] r33249 - in pypy/dist/pypy/translator: cli jvm jvm/src oosupport In-Reply-To: <20061013104610.BCC32100BD@code0.codespeak.net> References: <20061013104610.BCC32100BD@code0.codespeak.net> Message-ID: <452F725A.2000005@gmx.de> Hi Niko niko at codespeak.net wrote: > Author: niko > Date: Fri Oct 13 12:46:09 2006 > New Revision: 33249 > > Added: > pypy/dist/pypy/translator/jvm/ > pypy/dist/pypy/translator/jvm/generator.py > pypy/dist/pypy/translator/jvm/genjvm.py > pypy/dist/pypy/translator/jvm/opcodes.py > pypy/dist/pypy/translator/jvm/src/ > pypy/dist/pypy/translator/jvm/src/PyPy.java > pypy/dist/pypy/translator/jvm/types.py > Modified: > pypy/dist/pypy/translator/cli/function.py > pypy/dist/pypy/translator/cli/metavm.py > pypy/dist/pypy/translator/oosupport/metavm.py > Log: > first stabs at a jvm backend --- doesn't run or anything yet, but contains > at least a plausible implementation of the scalar operations > > also refactor a few things from cli into oosupport (mostly deleting duplicates > like getfield and setfield) and add a few (possibly oververbose :) comments. > changes to cli are very minor, but ran cli tests in any case and received > no regressions. [snip] I know that it is not much fun to do it this early in the life of a backend, but please try to write some tests. I will probably be annoying but you will get payoffs very quickly and thinking about how to test something (ideally before you start coding) helps you understand the upcoming tasks. Can't really comment on the technical quality of your checkin since I don't know any Java (nor much of the ootypesystem, for that matter :-). "Untested code is not working" Cheers, Carl Friedrich From niko at alum.mit.edu Fri Oct 13 13:07:53 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 13 Oct 2006 13:07:53 +0200 Subject: [pypy-dev] [pypy-svn] r33249 - in pypy/dist/pypy/translator: cli jvm jvm/src oosupport In-Reply-To: <452F725A.2000005@gmx.de> References: <20061013104610.BCC32100BD@code0.codespeak.net> <452F725A.2000005@gmx.de> Message-ID: <324B6388-8878-432F-8BFB-27D8C6E1E709@alum.mit.edu> > I know that it is not much fun to do it this early in the life of a > backend, but please try to write some tests. I will probably be > annoying > but you will get payoffs very quickly and thinking about how to test > something (ideally before you start coding) helps you understand the > upcoming tasks. Agreed. Actually, my next goal was to write start writing tests for the code I had written so far, but I thought I would check-in what I had done already before I started them. Niko From fijal at genesilico.pl Sat Oct 21 13:35:25 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 21 Oct 2006 13:35:25 +0200 Subject: [pypy-dev] py.test distributed Message-ID: <453A05FD.8090307@genesilico.pl> I'm gathering issues that you're all missing in py.test distributed. There is as well LSession (--session=L) which by default runs locally boxed version of tests (with segfault catching, applevel output catching and such). From cfbolz at gmx.de Sat Oct 21 23:52:39 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 21 Oct 2006 23:52:39 +0200 Subject: [pypy-dev] range list implementation Message-ID: <453A96A7.5080005@gmx.de> Hi all! Just a few words about the range list implementation I wrote during the last few days (when I was fed up with the config branch...). There is a new list implementation W_RangeListObject. The idea is that calling range will not immediately allocate the whole list but only store start, stop and step into the range list object. The range list object behaves like a perfectly normal list to the user. For some operations the range list is 'forced', that is the full list is generated. This is especially the case for operations that mutate the list. The following operations on the range list keep it in its non-forced memory-saving form: * iteration * len * getitem * getitem with a slice (which returns a new range list) * iter * repr * reverse * sort The reverse and sort method are admittedly somewhat useless (who constructs a range and sorts it?) but were easy to do. It would be possible to add support for more operations, but I doubt that it would be useful. I have not timed the result extensively, but you can pull tricks like that: >>>> import sys >>>> r = range(sys.maxint) >>>> len(r) 2147483647 >>>> iter(r) >>>> print r[1000:2000:30] [1000, 1030, 1060, 1090, 1120, 1150, 1180, 1210, 1240, 1270, 1300, 1330, 1360, 1390, 1420, 1450, 1480, 1510, 1540, 1570, 1600, 1630, 1660, 1690, 1720, 1750, 1780, 1810, 1840, 1870, 1900, 1930, 1960, 1990] Some very trivial timings: 2.877 seconds CPython versus 0.026 seconds pypy-c-withrangelist for the following snippet which is of course written in a way to showcase the advantages of range lists. Of course CPython uses a lot more memory too :-) import time t1 = time.time() result = [] for i in range(10000): result.append(range(i)) t2 = time.time() print t2 - t1 It remains to be seen what sort of benefits this gives for realistic applications. Cheers, Carl Friedrich From len-l at telus.net Sun Oct 22 01:43:57 2006 From: len-l at telus.net (Lenard Lindstrom) Date: Sat, 21 Oct 2006 16:43:57 -0700 Subject: [pypy-dev] range list implementation In-Reply-To: <453A96A7.5080005@gmx.de> Message-ID: <453A4E4D.22498.969B45@len-l.telus.net> On 21 Oct 2006 at 23:52, Carl Friedrich Bolz wrote: > Hi all! > > Just a few words about the range list implementation I wrote during the > last few days (when I was fed up with the config branch...). There is a > new list implementation W_RangeListObject. The idea is that calling > range will not immediately allocate the whole list but only store start, > stop and step into the range list object. The range list object behaves > like a perfectly normal list to the user. For some operations the range > list is 'forced', that is the full list is generated. This is especially > the case for operations that mutate the list. The following operations > on the range list keep it in its non-forced memory-saving form: > > * iteration > * len > * getitem > * getitem with a slice (which returns a new range list) > * iter > * repr > * reverse > * sort > Early versions of the xrange type supported slicing. Slicing was dropped from xrange in Python 2.2. As I recall reading somewhere there just wasn't enough interest to justify the cost of maintaining the C code. But maybe it makes more sense when coded in Python. Lenard Lindstrom From fijal at genesilico.pl Sun Oct 22 14:10:56 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 22 Oct 2006 14:10:56 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <20061021233702.32CB210070@code0.codespeak.net> References: <20061021233702.32CB210070@code0.codespeak.net> Message-ID: <453B5FD0.70101@genesilico.pl> niko at codespeak.net wrote: >Author: niko >Date: Sun Oct 22 01:36:59 2006 >New Revision: 33528 > >Added: > pypy/dist/pypy/translator/jvm/__init__.py > pypy/dist/pypy/translator/jvm/option.py > pypy/dist/pypy/translator/jvm/test/ > pypy/dist/pypy/translator/jvm/test/__init__.py > pypy/dist/pypy/translator/jvm/test/runtest.py > pypy/dist/pypy/translator/jvm/test/test_bool.py > pypy/dist/pypy/translator/jvm/typesystem.py >Removed: > pypy/dist/pypy/translator/jvm/types.py >Modified: > pypy/dist/pypy/translator/jvm/ (props changed) > pypy/dist/pypy/translator/jvm/conftest.py > pypy/dist/pypy/translator/jvm/database.py > pypy/dist/pypy/translator/jvm/generator.py > pypy/dist/pypy/translator/jvm/genjvm.py > pypy/dist/pypy/translator/jvm/node.py > pypy/dist/pypy/translator/jvm/opcodes.py > pypy/dist/pypy/translator/jvm/src/PyPy.java > pypy/dist/pypy/translator/oosupport/metavm.py >Log: >further work towards working jvm tests --- this is an intermediate >check-in, still evolving the JVM TypeSystem and database. > > > >Added: pypy/dist/pypy/translator/jvm/__init__.py >============================================================================== > >Modified: pypy/dist/pypy/translator/jvm/conftest.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/conftest.py (original) >+++ pypy/dist/pypy/translator/jvm/conftest.py Sun Oct 22 01:36:59 2006 >@@ -13,6 +13,10 @@ > ## help='View the graphs before they are generated'), > Option('--wd', action='store_true', dest='wd', default=False, > help='Output to current directory instead of /tmp'), >+ Option('--noassemble', action='store_true', dest="noasm", default=False, >+ help="don't assemble jasmin files"), >+ Option('--norun', action='store_true', dest="norun", default=False, >+ help="don't run the compiled executable"), > Option('--package', action='store', dest='package', default='pypy', > help='Package to output generated classes into') > #Option('--opt', action='XXX', dest='YYY', default=DEF, help='HELP') > >Modified: pypy/dist/pypy/translator/jvm/database.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/database.py (original) >+++ pypy/dist/pypy/translator/jvm/database.py Sun Oct 22 01:36:59 2006 >@@ -1,11 +1,85 @@ > """ >-For now, a simple worklist abstraction. >+The database tracks which graphs have already been generated, and maintains >+a worklist. It also contains a pointer to the type system. It is passed >+into every node for generation along with the generator. > """ >+from cStringIO import StringIO >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.typesystem import jvm_method_desc >+from pypy.translator.jvm import node >+import pypy.translator.jvm.generator as jvmgen >+import pypy.translator.jvm.typesystem as jvmtypes > > class Database: >- def __init__(self): >- self._pending = [] >+ def __init__(self, ts): >+ # Public attributes: >+ self.type_system = ts >+ >+ # Private attributes: >+ self._classes = {} # Maps ootype class objects to node.Class objects >+ self._counter = 0 # Used to create unique names >+ self._pending = [] # Worklist >+ >+ def _make_unique_name(self, nm): >+ cnt = self._counter >+ self._counter += 1 >+ return nm + "_" + str(cnt) + "_" >+ >+ def get_class_for(self, ooclass): >+ """ Given an OOTypeSystem Instance object representing a user >+ defined class (ooclass), returns a node.Class object representing >+ its jvm counterpart. """ >+ >+ # Create class object if it does not already exist: >+ if ooclass in self._classes: >+ return self._classes[ooclass] >+ clname = self._make_unique_name(ooclass._name) >+ clobj = self._classes[ooclass] = node.Class(clname) >+ >+ # Add fields: >+ for fieldnm, (fieldty, fielddef) in ooclass._fields.iteritems(): >+ if ftype is ootype.Void: continue >+ fieldnm = self._make_unique_name(fieldnm) >+ fieldty = self.type_system.ootype_to_jvm(ftype) >+ clobj.add_field(fieldty, fieldnm) # TODO --- fielddef?? >+ >+ # Add methods: >+ for mname, mimpl in ooclass._methods.iteritems(): >+ if not hasattr(mimpl, 'graph'): >+ # Abstract method >+ TODO >+ else: >+ # if the first argument's type is not a supertype of >+ # this class it means that this method this method is >+ # not really used by the class: don't render it, else >+ # there would be a type mismatch. >+ args = m_meth.graph.getargs() >+ SELF = args[0].concretetype >+ if not ootype.isSubclass(ooclass, SELF): continue >+ mobj = _method_for_graph(clobj, False, mimpl.graph) >+ clobj.add_method(mobj) >+ >+ return clobj >+ >+ def _method_for_graph(self, classobj, is_static, graph): > >+ """ >+ Creates a node.Function object for a particular graph. Adds the >+ method to 'classobj', which should be a node.Class object. >+ """ >+ >+ # Build up a func object >+ func_name = self._make_unique_name(graph.name) >+ argtypes = [arg.concretetype for arg in graph.getargs() >+ if arg.concretetype is not ootype.Void] >+ jargtypes = [self.type_system.ootype_to_jvm(argty) >+ for argty in argtypes] >+ rettype = graph.getreturnvar().concretetype >+ jrettype = self.type_system.ootype_to_jvm(rettype) >+ funcobj = self._translated[cachekey] = node.Function( >+ classobj, func_name, jargtypes, jrettype, graph, is_static) >+ return funcobj >+ > def pending_node(self, node): > self._pending.append(node) > >@@ -14,6 +88,3 @@ > > def pop(self): > return self._pending.pop() >- >- def method_for_graph(self, graph): >- > >Modified: pypy/dist/pypy/translator/jvm/generator.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/generator.py (original) >+++ pypy/dist/pypy/translator/jvm/generator.py Sun Oct 22 01:36:59 2006 >@@ -1,6 +1,7 @@ >+import os # >+from pypy.objspace.flow import model as flowmodel > from pypy.translator.oosupport.metavm import Generator >- >-__all__ = ['JasminGenerator'] >+from pypy.translator.jvm.typesystem import JvmType > > # ___________________________________________________________________________ > # JVM Opcode Flags: >@@ -8,11 +9,11 @@ > # Indicates certain properties of each opcode. Used mainly for debugging > # assertions > >-NOFLAGS = 0 >-BRANCH = 1 # Opcode is a branching opcode (implies a label argument) >-INTARG = 2 # Opcode has an integer argument >-CONST = 6 # Opcode has specialized variants (implies INTARG) >-INVOKE = 8 # Opcode is some kind of method invocation >+NOFLAGS = 0 >+BRANCH = 1 # Opcode is a branching opcode (implies a label argument) >+INTARG = 2 # Opcode has an integer argument >+CONSTSPEC = 6 # Opcode has specialized variants (implies INTARG) >+INVOKE = 8 # Opcode is some kind of method invocation > > # ___________________________________________________________________________ > # JVM Opcodes: >@@ -27,6 +28,48 @@ > """ > self.flags = flags > self.jvmstr = jvmstr >+ >+class OpcodeFamily(object): >+ """ >+ Many opcodes in JVM have variants that depend on the type of the >+ operands; for example, one must choose the correct ALOAD, ILOAD, >+ or DLOAD depending on whether one is loading a reference, integer, >+ or double variable respectively. Each instance of this class >+ defines one 'family' of opcodes, such as the LOAD family shown >+ above, and produces Opcode objects specific to a particular type. >+ """ >+ def __init__(self, flags, suffix): >+ """ >+ flags is a set of flags (see above) that describe opcode >+ jvmstr is the name for jasmin printouts >+ """ >+ self.flags = flags >+ self.suffix = suffix >+ self.cache = {} >+ >+ def _o(self, prefix): >+ try: >+ return self.cache[prefix] >+ except KeyError: >+ self.cache[prefix] = obj = Opcode(self.flags, prefix+self.suffix) >+ return obj >+ >+ def for_type(self, argtype): >+ """ Returns a customized opcode of this family appropriate to >+ 'argtype', a JvmType object. """ >+ >+ # These are always true: >+ if self[0] == 'L': return self._o("A") # Objects >+ if self[0] == '[': return self._o("A") # Arrays >+ if self == 'I': return self._o("I") # Integers >+ if self == 'J': return self._o("L") # Integers >+ if self == 'D': return self._o("D") # Doubles >+ if self == 'C': return self._o("C") # Characters >+ if self == 'B': return self._o("B") # Bytes >+ if self == 'V': return self._o("") # Void [used by RETURN] >+ >+ # TODO --- extend? etc >+ unimplemented > > # Define the opcodes for IFNE, IFEQ, IFLT, IF_ICMPLT, etc. The IFxx > # variants compare a single integer arg against 0, and the IF_ICMPxx >@@ -48,11 +91,11 @@ > > # Other opcodes > GOTO = Opcode(BRANCH, 'goto') >-ICONST = Opcode(CONST, 'iconst') >-DCONST_0 = Opcode(CONST, 'dconst_0') >-DCONST_1 = Opcode(CONST, 'dconst_1') >-LCONST_0 = Opcode(CONST, 'lconst_0') >-LCONST_1 = Opcode(CONST, 'lconst_1') >+ICONST = Opcode(CONSTSPEC, 'iconst') >+DCONST_0 = Opcode(NOFLAGS, 'dconst_0') >+DCONST_1 = Opcode(NOFLAGS, 'dconst_1') >+LCONST_0 = Opcode(NOFLAGS, 'lconst_0') >+LCONST_1 = Opcode(NOFLAGS, 'lconst_1') > GETFIELD = Opcode(NOFLAGS, 'getfield') > PUTFIELD = Opcode(NOFLAGS, 'putfield') > GETSTATIC = Opcode(NOFLAGS, 'getstatic') >@@ -66,16 +109,56 @@ > IDIV = Opcode(NOFLAGS, 'idiv') > IREM = Opcode(NOFLAGS, 'irem') > IAND = Opcode(NOFLAGS, 'iand') >+IOR = Opcode(NOFLAGS, 'ior') > ISHL = Opcode(NOFLAGS, 'ishl') > ISHR = Opcode(NOFLAGS, 'ishr') >+IUSHR = Opcode(NOFLAGS, 'iushr') > DCMPG = Opcode(NOFLAGS, 'dcmpg') > DCMPL = Opcode(NOFLAGS, 'dcmpl') > NOP = Opcode(NOFLAGS, 'nop') > I2D = Opcode(NOFLAGS, 'i2d') > I2L = Opcode(NOFLAGS, 'i2l') >+D2I= Opcode(NOFLAGS, 'd2i') >+L2I = Opcode(NOFLAGS, 'l2i') >+ATHROW = Opcode(NOFLAGS, 'athrow') >+DNEG = Opcode(NOFLAGS, 'dneg') >+DADD = Opcode(NOFLAGS, 'dadd') >+DSUB = Opcode(NOFLAGS, 'dsub') >+DMUL = Opcode(NOFLAGS, 'dmul') >+DDIV = Opcode(NOFLAGS, 'ddiv') >+DREM = Opcode(NOFLAGS, 'drem') >+LNEG = Opcode(NOFLAGS, 'lneg') >+LADD = Opcode(NOFLAGS, 'ladd') >+LSUB = Opcode(NOFLAGS, 'lsub') >+LMUL = Opcode(NOFLAGS, 'lmul') >+LDIV = Opcode(NOFLAGS, 'ldiv') >+LREM = Opcode(NOFLAGS, 'lrem') >+LAND = Opcode(NOFLAGS, 'land') >+LOR = Opcode(NOFLAGS, 'lor') >+LXOR = Opcode(NOFLAGS, 'lxor') >+LSHL = Opcode(NOFLAGS, 'lshl') >+LSHR = Opcode(NOFLAGS, 'lshr') >+LUSHR = Opcode(NOFLAGS, 'lushr') >+ >+# Loading/storing local variables >+LOAD = OpcodeFamily(INTARG, "load") >+STORE = OpcodeFamily(INTARG, "store") >+RETURN = OpcodeFamily(NOFLAGS, "return") >+ >+# Loading/storing from arrays >+# *NOTE*: This family is characterized by the type of the ELEMENT, >+# not the type of the ARRAY. >+# >+# Also: here I break from convention by naming the objects ARRLOAD >+# rather than ALOAD, even though the suffix is 'aload'. This is to >+# avoid confusion with the ALOAD opcode. >+ARRLOAD = OpcodeFamily(NOFLAGS, "aload") >+ARRSTORE = OpcodeFamily(NOFLAGS, "astore") > > # ___________________________________________________________________________ > # Helper Method Information >+# >+# These are used by code outside of this module as well. > > class Method(object): > def __init__(self, classnm, methnm, desc, opcode=INVOKESTATIC): >@@ -95,15 +178,40 @@ > PYPYUINTTODOUBLE = Method('pypy.PyPy', 'uint_to_double', '(I)D') > PYPYDOUBLETOUINT = Method('pypy.PyPy', 'double_to_uint', '(D)I') > PYPYLONGBITWISENEGATE = Method('pypy.PyPy', 'long_bitwise_negate', '(L)L') >+PYPYARRAYTOLIST = Method('pypy.PyPy', 'array_to_list', >+ '([Ljava/lang/Object;)Ljava/util/List;') >+PYPYSTRTOINT = Method('pypy.PyPy', 'str_to_int', >+ '([Ljava/lang/String;)I') >+PYPYSTRTOUINT = Method('pypy.PyPy', 'str_to_uint', >+ '([Ljava/lang/String;)I') >+PYPYSTRTOLONG = Method('pypy.PyPy', 'str_to_long', >+ '([Ljava/lang/String;)J') >+PYPYSTRTOULONG = Method('pypy.PyPy', 'str_to_ulong', >+ '([Ljava/lang/String;)J') >+PYPYSTRTOBOOL = Method('pypy.PyPy', 'str_to_bool', >+ '([Ljava/lang/String;)B') >+PYPYSTRTODOUBLE = Method('pypy.PyPy', 'str_to_double', >+ '([Ljava/lang/String;)D') >+PYPYSTRTOCHAR = Method('pypy.PyPy', 'str_to_char', >+ '([Ljava/lang/String;)C') > > class JVMGenerator(Generator): > > """ Base class for all JVM generators. Invokes a small set of '_' > methods which indicate which opcodes to emit; these can be >- translated by a subclass into Jasmin assembly, binary output, etc.""" >+ translated by a subclass into Jasmin assembly, binary output, etc. >+ Must be inherited from to specify a particular output format; >+ search for the string 'unimplemented' to find the methods that >+ must be overloaded. """ >+ >+ def __init__(self, type_system): >+ self.type_system = type_system > > # __________________________________________________________________ > # JVM specific methods to be overloaded by a subclass >+ # >+ # If the name does not begin with '_', it will be called from >+ # outside the generator. > > def begin_class(self, classnm): > """ >@@ -114,64 +222,213 @@ > def end_class(self): > unimplemented > >- def begin_function(self, funcname, argtypes, static=False): >+ def add_field(self, fname, ftype): >+ """ >+ fname --- name of the field (a string) >+ ftype --- JvmType for the field >+ """ >+ # TODO --- should fdesc be an ootype?? >+ unimplemented >+ >+ def begin_function(self, funcname, argvars, argtypes, rettype, >+ static=False): > """ > funcname --- name of the function >- argtypes --- types of each argument (in what format??) >+ argvars --- list of objects passed to load() that represent arguments; >+ should be in order, or () if load() will not be used >+ argtypes --- JvmType for each argument >+ rettype --- JvmType for the return value > static --- keyword, if true then a static func is generated >+ >+ This function also defines the scope for variables passed to >+ load()/store(). > """ >- unimplemented >+ # Compute the indicates of each argument in the local variables >+ # for the function. Note that some arguments take up two slots >+ # depending on their type [this is compute by type_width()] >+ self.next_offset = 0 >+ self.local_vars = {} >+ for idx, var in enumerate(argvars): >+ self.local_vars[var] = self.next_offset >+ self.next_offset += argtypes[idx].type_width() >+ # Prepare a map for the local variable indices we will add >+ # Let the subclass do the rest of the work; note that it does >+ # not need to know the argvars parameter, so don't pass it >+ self._begin_function(funcname, argtypes, rettype, static) >+ >+ def _begin_function(self, funcname, argtypes, rettype, static): >+ """ >+ Main implementation of begin_function. The begin_function() >+ does some generic handling of args. >+ """ >+ unimplemented > > def end_function(self): >+ del self.next_offset >+ del self.local_vars >+ self._end_function() >+ >+ def _end_function(self): >+ unimplemented >+ >+ def mark(self, lbl): >+ """ Marks the point that a label indicates. """ > unimplemented > >- def _unique_label(self, desc): >+ def _instr(self, opcode, *args): >+ """ Emits an instruction with the given opcode and arguments. >+ The correct opcode and their types depends on the opcode. """ >+ unimplemented >+ >+ def return_val(self, vartype): >+ """ Returns a value from top of stack of the JvmType 'vartype' """ >+ self._instr(RETURN.for_type(vartype)) >+ >+ def load_jvm_var(self, vartype, varidx): >+ """ Loads from jvm slot #varidx, which is expected to hold a value of >+ type vartype """ >+ self._instr(LOAD.for_type(vartype), varidx) >+ >+ def store_jvm_var(self, vartype, varidx): >+ """ Loads from jvm slot #varidx, which is expected to hold a value of >+ type vartype """ >+ self._instr(STORE.for_type(vartype), varidx) >+ >+ def load_from_array(self, elemtype): >+ """ Loads something from an array; the result will be of type 'elemtype' >+ (and hence the array is of type 'array_of(elemtype)'), where >+ 'elemtype' is a JvmType. Assumes that the array ref and index are >+ already pushed onto stack (in that order). """ >+ self._instr(ARRLOAD.for_type(elemtype)) >+ >+ def store_to_array(self, elemtype): >+ """ Stores something into an array; the result will be of type >+ 'elemtype' (and hence the array is of type >+ 'array_of(elemtype)'), where 'elemtype' is a JvmType. Assumes >+ that the array ref, index, and value are already pushed onto >+ stack (in that order).""" >+ self._instr(ARRLOAD.for_type(elemtype)) >+ >+ def unique_label(self, desc, mark=False): > """ Returns an opaque, unique label object that can be passed an >- argument for branching opcodes, or the _mark instruction. >+ argument for branching opcodes, or the mark instruction. > > 'desc' should be a comment describing the use of the label. > It is for decorative purposes only and should be a valid C >- identifier.""" >+ identifier. >+ >+ 'mark' --- if True, then also calls self.mark() with the new lbl """ > labelnum = len(self._labels) > self._labels.append(desc) >- return ('Label', labelnum) >+ res = ('Label', labelnum) >+ if mark: >+ self.mark(res) >+ return res >+ >+ # __________________________________________________________________ >+ # Exception Handling > >- def _mark(self, lbl): >- """ Marks the point that a label indicates. """ >- unimplemented >+ def begin_try(self): >+ """ >+ Begins a try/catch region. Must be followed by a call to end_try() >+ after the code w/in the try region is complete. >+ """ >+ self.begintrylbl = self.unique_label("begin_try", mark=True) > >- def _instr(self, opcode, *args): >- """ Emits an instruction with the given opcode and arguments. >- The correct opcode and their types depends on the opcode. """ >- unimplemented >+ def end_try(self): >+ """ >+ Ends a try/catch region. Must be followed immediately >+ by a call to begin_catch(). >+ """ >+ self.endtrylbl = self.unique_label("end_try", mark=True) >+ >+ def begin_catch(self, excclsty): >+ """ >+ Begins a catch region corresponding to the last try; there can >+ be more than one call to begin_catch, in which case the last >+ try region is reused. >+ 'excclsty' --- a JvmType for the class of exception to be caught >+ """ >+ catchlbl = self.unique_label("catch") >+ self.mark(catchlbl, mark=True) >+ self.try_catch_region( >+ excclsty, self.begintrylbl, send.endtrylbl, catchlbl) > >+ def end_catch(self): >+ """ >+ Ends a catch region. >+ (Included for CLI compatibility) >+ """ >+ return >+ >+ def try_catch_region(self, excclsty, trystartlbl, tryendlbl, catchlbl): >+ """ >+ Indicates a try/catch region: >+ 'excclsty' --- a JvmType for the class of exception to be caught >+ 'trystartlbl', 'tryendlbl' --- labels marking the beginning and end >+ of the try region >+ 'catchlbl' --- label marking beginning of catch region >+ """ >+ unimplemented >+ > # __________________________________________________________________ > # Generator methods and others that are invoked by MicroInstructions > # > # These translate into calls to the above methods. > > def emit(self, instr, *args): >- """ 'instr' in our case must be the name of another method, or >- a JVM opcode (as named above) """ >- >- if hasattr(self, instr): >+ """ 'instr' in our case must be either a string, in which case >+ it is the name of a method to invoke, or an Opcode/Method >+ object (defined above).""" >+ >+ if isinstance(instr, str): > return getattr(self, instr)(*args) >- >- glob = globals() >- if instr in glob: >- val = glob[instr] >- if isinstance(val, Opcode): >- self._instr(glob[opcode], *args) >- else if isinstance(val, Method): >- val.invoke(self) > >- assert False >+ if isinstance(instr, Opcode): >+ return self._instr(instr, *args) >+ >+ if isinstance(instr, Method): >+ return instr.invoke(self) >+ >+ raise Exception("Unknown object in call to emit(): "+repr(instr)) >+ >+ def _var_data(self, v): >+ # Determine java type: >+ jty = self.type_system.ootype_to_jvm(v.concretetype) >+ # Determine index in stack frame slots: >+ # note that arguments and locals can be treated the same here >+ if v in self.local_vars: >+ idx = self.local_vars[v] >+ else: >+ idx = self.local_vars[v] = self.next_offset >+ self.next_offset += jty.type_width() >+ return jty, idx > > def load(self, v): >- unimplemented >+ if isinstance(v, flowmodel.Variable): >+ jty, idx = _var_data(v) >+ return self.load_jvm_var(jty, idx) >+ >+ if isinstance(v, flowmodel.Constant): >+ # TODO: Refactor and complete this code? Maybe more like cli code? >+ if TYPE is ootype.Void: >+ pass >+ elif TYPE is ootype.Bool: >+ self._instr(ICONST, int(value)) >+ elif TYPE is ootype.Char or TYPE is ootype.UniChar: >+ self._instr(ICONST, ord(value)) >+ elif isinstance(value, CDefinedIntSymbolic): >+ self._instr(ICONST, DEFINED_INT_SYMBOLICS[value.expr]) >+ elif TYPE in (ootype.Signed, ootype.Unsigned): >+ self._instr(ICONST, value) # handle Unsigned better! >+ >+ raise Exception('Unexpected type for v in load(): '+v) > > def store(self, v): >- unimplemented >+ if isinstance(v, flowmodel.Variable): >+ jty, idx = _var_data(v) >+ return self.store_jvm_var(jty, idx) >+ raise Exception('Unexpected type for v in store(): '+v) > > def set_field(self, concretetype, value): > self._instr(SETFIELD, concretetype, value) >@@ -185,6 +442,13 @@ > # __________________________________________________________________ > # Methods invoked directly by strings in jvm/opcode.py > >+ def goto(self, lbl): >+ self._instr(GOTO, lbl) >+ >+ def throw(self): >+ """ Throw the object from top of the stack as an exception """ >+ self._instr(ATHROW) >+ > def iabs(self): > MATHIABS.invoke(self) > >@@ -196,6 +460,10 @@ > self._instr(ICONST, -1) > self._instr(IXOR) > >+ def goto_if_true(self, label): >+ """ Jumps if the top of stack is true """ >+ self._instr(IFNE, label) >+ > ##### Comparison methods > > def _compare_op(self, cmpopcode): >@@ -206,14 +474,14 @@ > instruction equals zero]. Consumes as many operands from the > stack as the cmpopcode consumes, typically 1 or 2. > """ >- midlbl = self._unique_label() >- endlbl = self._unique_label() >+ midlbl = self.unique_label('cmpop') >+ endlbl = self.unique_label('cmpop') > self._instr(cmpopcode, midlbl) > self._instr(ICONST, 0) > self._instr(GOTO, endlbl) >- self._mark(midlbl) >+ self.mark(midlbl) > self._instr(ICONST, 1) >- self._mark(endlbl) >+ self.mark(endlbl) > > logical_not = lambda self: self._compare_op(IFEQ) > equals_zero = logical_not >@@ -271,4 +539,72 @@ > ulong_greater_equals = lambda self: self._ulong_compare_op(IFGE) > > class JasminGenerator(JVMGenerator): >- pass >+ >+ def __init__(self, outdir, package): >+ self.outdir = outdir >+ >+ def begin_class(self, classnm): >+ """ >+ classnm --- full Java name of the class (i.e., "java.lang.String") >+ """ >+ >+ iclassnm = classnm.replace('.', '/') >+ jfile = "%s/%s.j" % (self.outdir, iclassnm) >+ >+ try: >+ jdir = jfile[:jfile.rindex('/')] >+ os.makedirs(jdir) >+ except OSError: pass >+ self.out = open(jfile, 'w') >+ >+ # Write the JasminXT header >+ self.out.write(".bytecode XX\n") >+ #self.out.write(".source \n") >+ self.out.write(".class public %s\n" % iclassnm) >+ self.out.write(".super java/lang/Object\n") # ? >+ >+ def end_class(self): >+ self.out.close() >+ self.out = None >+ >+ def add_field(self, fname, fdesc): >+ # TODO --- Signature for generics? >+ # TODO --- these must appear before methods, do we want to buffer >+ # them up to allow out of order calls to add_field()? >+ assert isinstance(fdesc, JvmType) >+ self.out.write('.field public %s %s\n' % (fname, fdesc)) >+ >+ def _begin_function(self, funcname, argtypes, rettype, static): >+ # Throws clause? Only use RuntimeExceptions? >+ kw = ['public'] >+ if static: kw.append('static') >+ self.out.write('.method %s %s (%s)%s\n' % ( >+ funcname, " ".join(kw), >+ "".join(argtypes), rettype)) >+ >+ def _end_function(self): >+ self.out.write('.end method\n') >+ >+ def mark(self, lbl): >+ """ Marks the point that a label indicates. """ >+ _, lblnm = lbl >+ assert _ == "Label" >+ self.out.write(' %s:\n' % lblnm) >+ >+ def _instr(self, opcode, *args): >+ jvmstr = opcode.jvmstr >+ >+ # Hack: this should be somewhere else, just not sure where yet >+ if opcode.flags & CONSTSPEC: >+ if args[0] == -1: >+ jvmstr += "_m1" >+ elif args[0] >= 0 and args[0] <= 5: >+ jvmstr += "_%d" % args[0] >+ >+ self.out.write(' %s %s\n' % ( >+ jvmstr, " ".join([str(s) for s in args]))) >+ >+ def try_catch_region(self, excclsty, trystartlbl, tryendlbl, catchlbl): >+ self.out.write(' .catch %s from %s to %s using %s\n' % ( >+ excclsty.int_class_name(), trystartlbl, tryendlbl, catchlbl)) >+ > >Modified: pypy/dist/pypy/translator/jvm/genjvm.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/genjvm.py (original) >+++ pypy/dist/pypy/translator/jvm/genjvm.py Sun Oct 22 01:36:59 2006 >@@ -11,7 +11,9 @@ > from pypy.translator.jvm.generator import JasminGenerator > from pypy.translator.jvm.option import getoption > from pypy.translator.jvm.database import Database >+from pypy.translator.jvm.typesystem import JvmTypeSystem > from pypy.translator.jvm.log import log >+from pypy.translator.jvm.node import EntryPoint > > class JvmError(Exception): > """ Indicates an error occurred in the JVM runtime """ >@@ -26,9 +28,13 @@ > > For those interested in the location of the files, the following > attributes exist: >- tmpdir --- root directory from which all files can be found >- javadir --- the directory containing *.java >- classdir --- the directory where *.class will be generated >+ tmpdir --- root directory from which all files can be found (py.path obj) >+ javadir --- the directory containing *.java (py.path obj) >+ classdir --- the directory where *.class will be generated (py.path obj) >+ package --- a string with the name of the package (i.e., 'java.util') >+ >+ The following attributes also exist to find the state of the sources: >+ compiled --- True once the sources have been compiled successfully > """ > > def __init__(self, tmpdir, package): >@@ -39,11 +45,12 @@ > """ > self.tmpdir = tmpdir > self.package = package >+ self.compiled = False > > # Compute directory where .java files are > self.javadir = self.tmpdir > for subpkg in package.split('.'): >- self.srcdir = os.path.join(self.srcdir, subpkg) >+ self.javadir = self.javadir.join(subpkg) > > # Compute directory where .class files should go > self.classdir = self.javadir >@@ -51,12 +58,14 @@ > def compile(self): > """ > Compiles the .java sources into .class files, ready for execution. >+ Raises a JvmError if compilation fails. > """ > javac = getoption('javac') >- javafiles = [f for f in os.listdir(self.javadir) >+ javafiles = [f for f in self.javadir.listdir() > if f.endswith('.java')] > res = subprocess.call([javac] + javafiles) > if res: raise JvmError('Failed to compile!') >+ else: self.compiled = True > > def execute(self, args): > """ >@@ -64,6 +73,7 @@ > output as a string. The 'args' are provided as arguments, > and will be converted to strings. > """ >+ assert self.compiled > strargs = [str(a) for a in args] > cmd = [getoption('java'), '%s.Main' % self.package] > pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE).stdout >@@ -81,12 +91,12 @@ > t = TranslationContext() > ann = t.buildannotator() > ann.build_types(func, annotation) >- t.buildrtype(type_system="ootype").specialize() >+ t.buildrtyper(type_system="ootype").specialize() > main_graph = t.graphs[0] > if getoption('view'): t.view() > if getoption('wd'): tmpdir = py.path.local('.') > else: tmpdir = udir >- jvm = GenJvm(tmpdir, t) >+ jvm = GenJvm(tmpdir, t, entrypoint=EntryPoint(main_graph, True)) > return jvm.generate_source() > > class GenJvm(object): >@@ -104,19 +114,18 @@ > 'entrypoint' --- if supplied, an object with a render method > """ > self.jvmsrc = JvmGeneratedSource(tmpdir, getoption('package')) >- self.db = Database() >+ self.type_system = JvmTypeSystem() >+ self.db = Database(self.type_system) > if entrypoint: > self.db.pending_node(entrypoint) >+ else: >+ self.db.pending_node(EntryPoint(translator.graphs[0], False)) > > def generate_source(self): > """ Creates the sources, and returns a JvmGeneratedSource object > for manipulating them """ > generator = self._create_generator() > >- # Deal with entry point >- if not self.db.len_pending(): >- # XXX default entry point >- > # Drain worklist > n = 0 > while self.db.len_pending(): >@@ -129,13 +138,12 @@ > (n, total, n*100.0/total)) > > # Return the source object once we have finished >- generator.all_done() > return self.jvmsrc > > def _create_generator(self): > """ Creates and returns a Generator object according to the > configuration. Right now, however, there is only one kind of > generator: JasminGenerator """ >- return JasminGenerator(self.jvmsrc.javadir) >+ return JasminGenerator(self.jvmsrc.javadir, self.jvmsrc.package) > > > >Modified: pypy/dist/pypy/translator/jvm/node.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/node.py (original) >+++ pypy/dist/pypy/translator/jvm/node.py Sun Oct 22 01:36:59 2006 >@@ -1,3 +1,16 @@ >+""" >+Rendering nodes for the JVM. I suspect that a lot of this could be >+made to be common between CLR and JVM. >+""" >+ >+ >+from pypy.rpython.lltypesystem import lltype >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.typesystem import jStringArray, jVoid, jThrowable >+from pypy.translator.jvm.typesystem import jvm_for_class >+import pypy.translator.jvm.generator as jvmgen >+from pypy.translator.jvm.opcodes import opcodes >+ > class Node(object): > def render(self, db, generator): > unimplemented >@@ -10,22 +23,256 @@ > testing (see __init__) > """ > >- def __init__(self, graph): >+ def __init__(self, graph, expandargs): > """ > 'graph' --- The initial graph to invoke from main() >+ 'expandargs' --- controls whether the arguments passed to main() >+ are passed as a list, or expanded to match each argument to the graph >+ >+ The 'expandargs' option deserves explanation: >+ >+ it will be false for a standalone build, because in that >+ case we want to convert the String[] array that main() receives >+ into a corresponding python List of string objects. >+ >+ it will (generally) be true when compiling individual >+ functions, in which case we might be compiling an entry >+ point with a signature like (a:int,b:float) in which case >+ argv[1] should be converted to an integer, and argv[2] >+ should be converted to a float. > """ > self.graph = graph >+ self.expand_arguments = expandargs > pass > >+ # XXX --- perhaps this table would be better placed in typesystem.py >+ # so as to constrain the knowledge of lltype and ootype >+ _type_conversion_methods = { >+ ootype.Signed:jvmgen.PYPYSTRTOINT, >+ ootype.Unsigned:jvmgen.PYPYSTRTOUINT, >+ lltype.SignedLongLong:jvmgen.PYPYSTRTOLONG, >+ lltype.UnsignedLongLong:jvmgen.PYPYSTRTOULONG, >+ ootype.Bool:jvmgen.PYPYSTRTOBOOL, >+ ootype.Float:jvmgen.PYPYSTRTODOUBLE, >+ ootype.Char:jvmgen.PYPYSTRTOCHAR >+ } >+ > def render(self, db, gen): > gen.begin_class('pypy.Main') >- gen.begin_function('main', 'String[]', static=True) >+ gen.begin_function('main', (), [jStringArray], jVoid, static=True) > >- # XXX --- handle arguments somehow! (will probably need some options) >+ # Handle arguments: >+ if self.expand_arguments: >+ # Convert each entry into the array to the desired type by >+ # invoking an appropriate helper function on each one >+ for i, arg in enumerate(self.graph.getargs()): >+ gen.emit(jvmgen.ICONST, i) >+ gen.emit(self._type_conversion_methods[arg.concretetype]) >+ else: >+ # Convert the array of strings to a List as the >+ # python method expects >+ arg0 = self.graph.getargs()[0] >+ assert isinstance(arg0.concretetype, ootype.List), str(arg0.concretetype) >+ assert arg0._ITEMTYPE is ootype.String >+ gen.load_jvm_var(0) >+ gen.emit(jvmgen.PYPYARRAYTOLIST) > > # Generate a call to this method >- db.method_for_graph(self.graph).invoke(gen) >+ gen.emit(db.method_for_graph(self.graph, static=True)) > > gen.end_function() > gen.end_class() > >+class Function(object): >+ >+ """ Represents a function to be emitted. *Note* that it is not a >+ descendant of Node: it cannot be entered into the database >+ worklist. This is because in Java, all functions occur w/in a >+ class: therefore classes as a whole must be placed on the >+ worklist. """ >+ >+ def __init__(self, classobj, name, jargtypes, jrettype, graph, is_static): >+ """ >+ classobj: the Class object this is a part of (even static >+ functions have a class) >+ name: the name of the function >+ jargtypes: JvmType of each argument >+ jrettype: JvmType this function returns >+ graph: the graph representing the body of the function >+ is_static: boolean flag indicate whether func is static (!) >+ """ >+ self.classnm = classnm >+ self.name = name >+ self.graph = graph >+ self.jargtypes = jargtypes >+ self.jrettype = jrettype >+ self.is_static = is_static >+ >+ def method(self): >+ """ Returns a jvmgen.Method that can invoke this function """ >+ if self.is_static: opcode = jvmgen.INVOKESTATIC >+ else: opcode = jvmgen.INVOKEVIRTUAL >+ mdesc = jvm_method_desc(self.jargtypes, self.jrettype) >+ return jvmgen.Method(classnm, self.func_name, mdesc, opcode=opcode) >+ >+ def render_func(self, db, gen): >+ if getattr(self.graph.func, 'suggested_primitive', False): >+ assert False, 'Cannot render a suggested_primitive' >+ >+ # Prepare argument lists for begin_function call >+ jargvars = [] >+ jargtypes = [] >+ for arg in self.graph.getargs(): >+ if arg.concretetype is ootype.Void: continue >+ jargvars.append(arg) >+ jargtypes.append(db.type_system.ootype_to_jvm(arg.concretetype)) >+ >+ # Determine return type >+ jrettype = db.type_system.ootype_to_jvm( >+ self.graph.getreturnvar().concretetype) >+ >+ # Start the function definition >+ gen.begin_function(self.name, jargvars, jargtypes, jrettype, >+ static=self.is_static) >+ >+ # Go through each block and create a label for it; if the >+ # block will be used to catch an exception, add a second label >+ # to catch_labels >+ block_labels = {} >+ #catch_labels = {} >+ for ctr, block in enumerate(graph.iterblocks()): >+ blbl = gen.unique_label('Block_'+ctr) >+ block_labels[block] = blbl >+ >+ ## Go through the blocks we may toss exceptions to >+ #if block.exitswitch == flowmodel.c_last_exception: >+ # for link in block.exits: >+ # if link.exitcase is None: continue # return >+ # if link.target not in catch_labels: >+ # catch_labels[link.target] = gen.unique_label('catch') >+ >+ # Iterate through the blocks and generate code for them >+ return_blocks = [] >+ for block in graph.iterblocks(): >+ >+ # Mark the beginning of the block, render all the ops, and >+ # then mark the end. >+ gen.mark(block_labels[block][0]) >+ >+ # Determine whether the last oper in this block may raise an exc >+ handle_exc = (block.exitswitch == flowmodel.c_last_exception) >+ >+ # Render the operations; create labels for a try/catch >+ # region around the last operation >+ if block.operations: >+ for op in block.operations[:-1]: >+ self._render_op(op) >+ if handle_exc: trybeglbl = gen.unique_label('try', mark=True) >+ self._render_op(block.operations[-1]) >+ if handle_exc: tryendlbl = gen.unique_label('try', mark=True) >+ >+ # Handle 'return' blocks: in this case, we return the >+ # variable specified >+ if self._is_return_block(block): >+ return_var = block.inputargs[0] >+ return_ty = ootype_to_jvm(return_var.concretetype) >+ if return_var.concretetype is not Void: >+ self.load(return_var) >+ gen.return_val(return_ty) >+ >+ # Handle 'raise' blocks: in this case, we just throw the >+ # variable specified >+ if self._is_raise_block(block): >+ exc = block.inputargs[1] >+ self.load(exc) >+ gen.throw() >+ >+ if handle_exc: >+ # search for the "default" block to be executed when >+ # no exception is raised >+ for link in block.exits: >+ if link.exitcase is None: >+ self._copy_link_vars(gen, link) >+ gen.goto(block_labels[link.target]) >+ >+ # TODO: proper exception handling; we may not want to >+ # use the same model as CLR >+ else: >+ # no exception handling, determine correct link to follow >+ for link in block.exits: >+ self._copy_link_vars(gen, link) >+ target_label = block_labels[link.target] >+ if link.exitcase is None or link is block.exits[-1]: >+ gen.goto(target_label) >+ else: >+ assert type(link.exitcase is bool) >+ assert block.exitswitch is not None >+ gen.load(block.exitswitch) >+ gen.goto_if_true(target_label) >+ >+ gen.end_function() >+ >+ def _render_op(self, op): >+ instr_list = opcodes.get(op.opname, None) >+ assert getoption('nostop') or instr_list is not None >+ if instr_list: instr_list.render(self, op) >+ >+ def _copy_link_vars(self, gen, link): >+ target = link.target >+ for to_load, to_store in zip(link.args, target.inputargs): >+ if to_load.concretetype is not Void: >+ gen.load(to_load) >+ gen.store(to_store) >+ >+ def _is_return_block(self, block): >+ return (not block.exits) and len(block.inputargs) == 1 >+ >+ def _is_raise_block(self, block): >+ return (not block.exits) and len(block.inputargs) == 2 >+ >+class Class(Node): >+ >+ """ Represents a class to be emitted. Note that currently, classes >+ are emitted all in one shot, not piecemeal. """ >+ >+ def __init__(self, name): >+ """ >+ 'name' should be a fully qualified Java class name like >+ "java.lang.String" >+ """ >+ self.name = name >+ self.fields = [] >+ self.methods = {} # Maps graph -> Function >+ self.rendered = False >+ >+ def jvm_type(self): >+ return jvm_for_class(self.name) >+ >+ def add_field(self, fieldty, fieldnm): >+ """ Creates a new field in this with type 'fieldty' (a >+ JvmType) and with the name ;fieldnm; (a String). Must be called >+ before render().""" >+ assert not self.rendered >+ self.fields.append((fieldty, fieldnm)) >+ >+ def has_method_for(self, graph): >+ return graph in self.methods >+ >+ def add_method(self, func): >+ """ Creates a new method in this class, represented by the >+ Function object 'func'. Must be called before render(); >+ intended to be invoked by the database.""" >+ assert not self.rendered >+ self.methods[func.graph] = func >+ >+ def render(self, db, gen): >+ self.rendered = True >+ gen.begin_class(self.name) >+ >+ for fieldty, fieldnm in self.fields: >+ gen.add_field(fieldty, fieldnm) >+ >+ for method in self.methods.values(): >+ method.render_func(db, gen) >+ >+ gen.end_class(self.name) > >Modified: pypy/dist/pypy/translator/jvm/opcodes.py >============================================================================== >--- pypy/dist/pypy/translator/jvm/opcodes.py (original) >+++ pypy/dist/pypy/translator/jvm/opcodes.py Sun Oct 22 01:36:59 2006 >@@ -5,41 +5,48 @@ > > """ > >-from pypy.translator.cli.metavm import Call, CallMethod, RuntimeNew, \ >- IndirectCall, GetField, SetField, CastTo, OOString, DownCast, NewCustomDict,\ >- CastWeakAdrToPtr, MapException >-from pypy.translator.oosupport.metavm import PushArg, PushAllArgs, StoreResult, InstructionList,\ >- New >-from pypy.translator.cli.cts import WEAKREF >+from pypy.translator.oosupport.metavm import \ >+ PushArg, PushAllArgs, StoreResult, InstructionList, New, DoNothing >+import pypy.translator.jvm.generator as jvmgen >+ >+def _check_zer(op): >+ # TODO >+ return op >+ >+def _check_ovf(op): >+ # TODO >+ return op > >+# This table maps the opcodes to micro-ops for processing them. >+# It is post-processed by a function to be found below. > opcodes = { > # __________ object oriented operations __________ >- 'new': [New], >- 'runtimenew': [RuntimeNew], >- 'oosetfield': [SetField], >- 'oogetfield': [GetField], >- 'oosend': [CallMethod], >- 'ooupcast': DoNothing, >- 'oodowncast': [DownCast], >- 'oois': 'ceq', >- 'oononnull': [PushAllArgs, 'ldnull', 'ceq']+Not, >- 'instanceof': [CastTo, 'ldnull', 'cgt.un'], >- 'subclassof': [PushAllArgs, 'call bool [pypylib]pypy.runtime.Utils::SubclassOf(class [mscorlib]System.Type, class[mscorlib]System.Type)'], >- 'ooidentityhash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >- 'oohash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >- 'oostring': [OOString], >- 'ooparse_int': [PushAllArgs, 'call int32 [pypylib]pypy.runtime.Utils::OOParseInt(string, int32)'], >- 'oonewcustomdict': [NewCustomDict], >- >- 'same_as': DoNothing, >- 'hint': [PushArg(0), StoreResult], >- 'direct_call': [Call], >- 'indirect_call': [IndirectCall], >- >- 'cast_ptr_to_weakadr': [PushAllArgs, 'newobj instance void class %s::.ctor(object)' % WEAKREF], >- 'cast_weakadr_to_ptr': [CastWeakAdrToPtr], >- 'gc__collect': 'call void class [mscorlib]System.GC::Collect()', >- 'resume_point': Ignore, >+ #'new': [New], >+ #'runtimenew': [RuntimeNew], >+ #'oosetfield': [SetField], >+ #'oogetfield': [GetField], >+ #'oosend': [CallMethod], >+ #'ooupcast': DoNothing, >+ #'oodowncast': [DownCast], >+ #'oois': 'ceq', >+ #'oononnull': [PushAllArgs, 'ldnull', 'ceq']+Not, >+ #'instanceof': [CastTo, 'ldnull', 'cgt.un'], >+ #'subclassof': [PushAllArgs, 'call bool [pypylib]pypy.runtime.Utils::SubclassOf(class [mscorlib]System.Type, class[mscorlib]System.Type)'], >+ #'ooidentityhash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >+ #'oohash': [PushAllArgs, 'callvirt instance int32 object::GetHashCode()'], >+ #'oostring': [OOString], >+ #'ooparse_int': [PushAllArgs, 'call int32 [pypylib]pypy.runtime.Utils::OOParseInt(string, int32)'], >+ #'oonewcustomdict': [NewCustomDict], >+ # >+ #'same_as': DoNothing, >+ #'hint': [PushArg(0), StoreResult], >+ #'direct_call': [Call], >+ #'indirect_call': [IndirectCall], >+ # >+ #'cast_ptr_to_weakadr': [PushAllArgs, 'newobj instance void class %s::.ctor(object)' % WEAKREF], >+ #'cast_weakadr_to_ptr': [CastWeakAdrToPtr], >+ #'gc__collect': 'call void class [mscorlib]System.GC::Collect()', >+ #'resume_point': Ignore, > > # __________ numeric operations __________ > >@@ -56,126 +63,132 @@ > 'unichar_ne': 'not_equals', > > 'int_is_true': 'not_equals_zero', >- 'int_neg': 'INEG', >+ 'int_neg': jvmgen.INEG, > 'int_neg_ovf': None, # How to handle overflow? > 'int_abs': 'iabs', > 'int_abs_ovf': _check_ovf('iabs'), > 'int_invert': 'bitwise_negate', > >- 'int_add': 'IADD', >- 'int_sub': 'ISUB', >- 'int_mul': 'IMUL', >- 'int_floordiv': 'IDIV', >- 'int_floordiv_zer': _check_zer('IDIV'), >- 'int_mod': 'IREM', >+ 'int_add': jvmgen.IADD, >+ 'int_sub': jvmgen.ISUB, >+ 'int_mul': jvmgen.IMUL, >+ 'int_floordiv': jvmgen.IDIV, >+ 'int_floordiv_zer': _check_zer(jvmgen.IDIV), >+ 'int_mod': jvmgen.IREM, > 'int_lt': 'less_than', > 'int_le': 'less_equals', > 'int_eq': 'equals', > 'int_ne': 'not_equals', > 'int_gt': 'greater_than', > 'int_ge': 'greater_equals', >- 'int_and': 'IAND', >- 'int_or': 'IOR', >- 'int_lshift': 'ISHL', >- 'int_rshift': 'ISHR', >- 'int_xor': 'IXOR', >- 'int_add_ovf': _check_ovf('IADD'), >- 'int_sub_ovf': _check_ovf('ISUB'), >- 'int_mul_ovf': _check_ovf('IMUL'), >- 'int_floordiv_ovf': 'IDIV', # these can't overflow! >- 'int_mod_ovf': 'IREM', >+ 'int_and': jvmgen.IAND, >+ 'int_or': jvmgen.IOR, >+ 'int_lshift': jvmgen.ISHL, >+ 'int_rshift': jvmgen.ISHR, >+ 'int_xor': jvmgen.IXOR, >+ 'int_add_ovf': _check_ovf(jvmgen.IADD), >+ 'int_sub_ovf': _check_ovf(jvmgen.ISUB), >+ 'int_mul_ovf': _check_ovf(jvmgen.IMUL), >+ 'int_floordiv_ovf': jvmgen.IDIV, # these can't overflow! >+ 'int_mod_ovf': jvmgen.IREM, > 'int_lt_ovf': 'less_than', > 'int_le_ovf': 'less_equals', > 'int_eq_ovf': 'equals', > 'int_ne_ovf': 'not_equals', > 'int_gt_ovf': 'greater_than', > 'int_ge_ovf': 'greater_equals', >- 'int_and_ovf': 'IAND', >- 'int_or_ovf': 'IOR', >+ 'int_and_ovf': jvmgen.IAND, >+ 'int_or_ovf': jvmgen.IOR, > >- 'int_lshift_ovf': _check_ovf('ISHL'), >- 'int_lshift_ovf_val': _check_ovf('ISHL'), # VAL?? >+ 'int_lshift_ovf': _check_ovf(jvmgen.ISHL), >+ 'int_lshift_ovf_val': _check_ovf(jvmgen.ISHL), # VAL?? > >- 'int_rshift_ovf': 'ISHR', # these can't overflow! >- 'int_xor_ovf': 'IXOR', >- 'int_floordiv_ovf_zer': _check_zer('IDIV'), >- 'int_mod_ovf_zer': _check_zer('IREM'), >+ 'int_rshift_ovf': jvmgen.ISHR, # these can't overflow! >+ 'int_xor_ovf': jvmgen.IXOR, >+ 'int_floordiv_ovf_zer': _check_zer(jvmgen.IDIV), >+ 'int_mod_ovf_zer': _check_zer(jvmgen.IREM), > > 'uint_is_true': 'not_equals_zero', > 'uint_invert': 'bitwise_negate', > >- 'uint_add': 'IADD', >- 'uint_sub': 'ISUB', >- 'uint_mul': 'IMUL', >- 'uint_div': 'IDIV', # valid? >+ 'uint_add': jvmgen.IADD, >+ 'uint_sub': jvmgen.ISUB, >+ 'uint_mul': jvmgen.IMUL, >+ 'uint_div': jvmgen.IDIV, # valid? > 'uint_truediv': None, # TODO >- 'uint_floordiv': 'IDIV', # valid? >- 'uint_mod': 'IREM', # valid? >+ 'uint_floordiv': jvmgen.IDIV, # valid? >+ 'uint_mod': jvmgen.IREM, # valid? > 'uint_lt': 'u_less_than', > 'uint_le': 'u_less_equals', > 'uint_eq': 'u_equals', > 'uint_ne': 'u_not_equals', > 'uint_gt': 'u_greater_than', > 'uint_ge': 'u_greater_equals', >- 'uint_and': 'IAND', >- 'uint_or': 'IOR', >- 'uint_lshift': 'ISHL', >- 'uint_rshift': 'IUSHR', >- 'uint_xor': 'IXOR', >- >- 'float_is_true': [PushAllArgs, 'DCONST_0', 'dbl_not_equals'], >- 'float_neg': 'DNEG', >+ 'uint_and': jvmgen.IAND, >+ 'uint_or': jvmgen.IOR, >+ 'uint_lshift': jvmgen.ISHL, >+ 'uint_rshift': jvmgen.IUSHR, >+ 'uint_xor': jvmgen.IXOR, >+ >+ 'float_is_true': [PushAllArgs, >+ jvmgen.DCONST_0, >+ 'dbl_not_equals'], >+ 'float_neg': jvmgen.DNEG, > 'float_abs': 'dbl_abs', > >- 'float_add': 'DADD', >- 'float_sub': 'DSUB', >- 'float_mul': 'DMUL', >- 'float_truediv': 'DDIV', >- 'float_mod': 'DREM', # use Math.IEEEremainder? >+ 'float_add': jvmgen.DADD, >+ 'float_sub': jvmgen.DSUB, >+ 'float_mul': jvmgen.DMUL, >+ 'float_truediv': jvmgen.DDIV, >+ 'float_mod': jvmgen.DREM, # use Math.IEEEremainder? > 'float_lt': 'dbl_less_than', > 'float_le': 'dbl_less_equals', > 'float_eq': 'dbl_equals', > 'float_ne': 'dbl_not_equals', > 'float_gt': 'dbl_greater_than', > 'float_ge': 'dbl_greater_equals', >- 'float_floor': 'MATHFLOOR', >- 'float_fmod': 'DREM', # DREM is akin to fmod() in C >+ 'float_floor': jvmgen.MATHFLOOR, >+ 'float_fmod': jvmgen.DREM, # DREM is akin to fmod() in C > >- 'llong_is_true': [PushAllArgs, 'LCONST_0', 'long_not_equals'], >- 'llong_neg': 'LNEG', >- 'llong_neg_ovf': _check_ovf('LNEG'), >- 'llong_abs': 'MATHLABS', >- 'llong_invert': 'PYPYLONGBITWISENEGATE', >- >- 'llong_add': 'LADD', >- 'llong_sub': 'LSUB', >- 'llong_mul': 'LMUL', >- 'llong_div': 'LDIV', >+ 'llong_is_true': [PushAllArgs, >+ jvmgen.LCONST_0, >+ 'long_not_equals'], >+ 'llong_neg': jvmgen.LNEG, >+ 'llong_neg_ovf': _check_ovf(jvmgen.LNEG), >+ 'llong_abs': jvmgen.MATHLABS, >+ 'llong_invert': jvmgen.PYPYLONGBITWISENEGATE, >+ >+ 'llong_add': jvmgen.LADD, >+ 'llong_sub': jvmgen.LSUB, >+ 'llong_mul': jvmgen.LMUL, >+ 'llong_div': jvmgen.LDIV, > 'llong_truediv': None, # TODO >- 'llong_floordiv': 'LDIV', >- 'llong_mod': 'LREM', >+ 'llong_floordiv': jvmgen.LDIV, >+ 'llong_mod': jvmgen.LREM, > 'llong_lt': 'long_less_than', > 'llong_le': 'long_less_equals', > 'llong_eq': 'long_equals', > 'llong_ne': 'long_not_equals', > 'llong_gt': 'long_greater_than', > 'llong_ge': 'long_greater_equals', >- 'llong_and': 'LAND', >- 'llong_or': 'LOR', >- 'llong_lshift': 'LSHL', >- 'llong_rshift': 'LSHR', >- 'llong_xor': 'LXOR', >- >- 'ullong_is_true': [PushAllArgs, 'LCONST_0', 'long_not_equals'], >- 'ullong_invert': 'PYPYLONGBITWISENEGATE', >- >- 'ullong_add': 'LADD', >- 'ullong_sub': 'LSUB', >- 'ullong_mul': 'LMUL', >- 'ullong_div': 'LDIV', # valid? >+ 'llong_and': jvmgen.LAND, >+ 'llong_or': jvmgen.LOR, >+ 'llong_lshift': jvmgen.LSHL, >+ 'llong_rshift': jvmgen.LSHR, >+ 'llong_xor': jvmgen.LXOR, >+ >+ 'ullong_is_true': [PushAllArgs, >+ jvmgen.LCONST_0, >+ 'long_not_equals'], >+ 'ullong_invert': jvmgen.PYPYLONGBITWISENEGATE, >+ >+ 'ullong_add': jvmgen.LADD, >+ 'ullong_sub': jvmgen.LSUB, >+ 'ullong_mul': jvmgen.LMUL, >+ 'ullong_div': jvmgen.LDIV, # valid? > 'ullong_truediv': None, # TODO >- 'ullong_floordiv': 'LDIV', # valid? >- 'ullong_mod': 'LREM', # valid? >+ 'ullong_floordiv': jvmgen.LDIV, # valid? >+ 'ullong_mod': jvmgen.LREM, # valid? > 'ullong_lt': 'ulong_less_than', > 'ullong_le': 'ulong_less_equals', > 'ullong_eq': 'ulong_equals', >@@ -189,19 +202,24 @@ > # trick. > 'cast_bool_to_int': DoNothing, > 'cast_bool_to_uint': DoNothing, >- 'cast_bool_to_float': [PushAllArgs, 'not_equals_zero', 'I2D'], >+ 'cast_bool_to_float': [PushAllArgs, 'not_equals_zero', jvmgen.I2D], > > 'cast_char_to_int': DoNothing, > 'cast_unichar_to_int': DoNothing, > 'cast_int_to_char': DoNothing, > 'cast_int_to_unichar': DoNothing, > 'cast_int_to_uint': DoNothing, >- 'cast_int_to_float': 'I2D', >- 'cast_int_to_longlong': 'I2L', >+ 'cast_int_to_float': jvmgen.I2D, >+ 'cast_int_to_longlong': jvmgen.I2L, > 'cast_uint_to_int': DoNothing, >- 'cast_uint_to_float': PYPYUINTTODOUBLE, >- 'cast_float_to_int': 'D2I', >- 'cast_float_to_uint': PYPYDOUBLETOUINT, >- 'truncate_longlong_to_int': 'L2I', >+ 'cast_uint_to_float': jvmgen.PYPYUINTTODOUBLE, >+ 'cast_float_to_int': jvmgen.D2I, >+ 'cast_float_to_uint': jvmgen.PYPYDOUBLETOUINT, >+ 'truncate_longlong_to_int': jvmgen.L2I, > > } >+ >+for opc in opcodes: >+ val = opcodes[opc] >+ if not isinstance(val, list): >+ val = [PushAllArgs, val] > >Added: pypy/dist/pypy/translator/jvm/option.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/option.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,4 @@ >+from pypy.translator.jvm.conftest import option >+ >+def getoption(name): >+ return getattr(option, name) > >Modified: pypy/dist/pypy/translator/jvm/src/PyPy.java >============================================================================== >--- pypy/dist/pypy/translator/jvm/src/PyPy.java (original) >+++ pypy/dist/pypy/translator/jvm/src/PyPy.java Sun Oct 22 01:36:59 2006 >@@ -1,12 +1,35 @@ > package pypy; > >+import java.util.List; >+import java.util.ArrayList; >+ >+/** >+ * Class with a number of utility routines. >+ */ > public class PyPy { > >+ /** >+ * Compares two unsigned integers (value1 and value2) and returns >+ * a value greater than, equal to, or less than zero if value 1 is >+ * respectively greater than, equal to, or less than value2. The >+ * idea is that you can do the following: >+ * >+ * Call uint_cmp(value1, value2) >+ * IFLT ... // jumps if value1 < value2 >+ * IFEQ ... // jumps if value1 == value2 >+ * IFGT ... // jumps if value1 > value2 >+ * etc >+ */ > public static int uint_cmp(int value1, int value2) { >- final int VALUE2BIGGER = -1; > final int VALUE1BIGGER = 1; >+ final int VALUE2BIGGER = -1; > final int EQUAL = 0; > >+ if (((value1 | value2) & Integer.MIN_VALUE) == 0) { >+ // neither is negative, presumably the common case >+ return value1 - value2; >+ } >+ > if (value1 == value2) > return EQUAL; > >@@ -23,13 +46,8 @@ > return VALUE1BIGGER; > } > } >- else if (value2 < 0) { >- // value1 is not neg, value2 is neg >- return VALUE2BIGGER; >- } >- >- if (value1 > value2) >- return VALUE1BIGGER; >+ >+ // value1 is not neg, value2 is neg > return VALUE2BIGGER; > } > >@@ -92,4 +110,76 @@ > public static long long_bitwise_negate(long value) { > return ~value; > } >+ >+ public static List array_to_list(Object[] array) { >+ List l = new ArrayList(); >+ for (Object o : array) { >+ l.add(o); >+ } >+ return l; >+ } >+ >+ public static int str_to_int(String s) { >+ try { >+ return Integer.parseInt(s); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static int str_to_uint(String s) { >+ try { >+ long l = Long.parseLong(s); >+ if (l < Integer.MAX_VALUE) >+ return l; >+ int lowerword = l & 0xFFFF; >+ int upperword = l >> 16; >+ return lowerword + (upperword << 16); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static long str_to_long(String s) { >+ try { >+ return Long.parseLong(s); >+ } catch (NumberFormatException fe) { >+ throw new RuntimeException(fe); >+ } >+ } >+ >+ public static long str_to_ulong(String s) { >+ // oh bother >+ throw new RuntimeException("TODO--- str to ulong"); >+ } >+ >+ public static boolean str_to_bool(String s) { >+ // not sure what are considered valid boolean values... >+ // let's be very accepting and take both strings and numbers >+ if (s.equalsIgnoreCase("true")) >+ return true; >+ if (s.equalsIgnoreCase("false")) >+ return false; >+ >+ try { >+ int i = Integer.parseInt(s); >+ return i != 0; >+ } catch (NumberFormatException ex) { >+ throw new RuntimeException(ex); >+ } >+ } >+ >+ public static double str_to_double(String s) { >+ try { >+ return Double.parseDouble(s); >+ } catch (NumberFormatException ex) { >+ throw new RuntimeException(ex); >+ } >+ } >+ >+ public static char str_to_char(String s) { >+ if (s.length() != 1) >+ throw new RuntimeException("String not single character: '"+s+"'"); >+ return s.charAt(0); >+ } > } >\ No newline at end of file > >Added: pypy/dist/pypy/translator/jvm/test/__init__.py >============================================================================== > >Added: pypy/dist/pypy/translator/jvm/test/runtest.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/test/runtest.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,119 @@ >+import os >+import platform >+ >+import py >+from py.compat import subprocess >+from pypy.tool.udir import udir >+from pypy.rpython.test.tool import BaseRtypingTest, OORtypeMixin >+from pypy.rpython.lltypesystem.lltype import typeOf >+from pypy.rpython.ootypesystem import ootype >+from pypy.annotation.model import lltype_to_annotation >+from pypy.translator.translator import TranslationContext >+from pypy.translator.jvm.genjvm import generate_source_for_function >+from pypy.translator.jvm.option import getoption >+ >+FLOAT_PRECISION = 8 >+ >+# CLI duplicate >+class StructTuple(tuple): >+ def __getattr__(self, name): >+ if name.startswith('item'): >+ i = int(name[len('item'):]) >+ return self[i] >+ else: >+ raise AttributeError, name >+ >+# CLI duplicate >+class OOList(list): >+ def ll_length(self): >+ return len(self) >+ >+ def ll_getitem_fast(self, i): >+ return self[i] >+ >+# CLI duplicate >+class ExceptionWrapper: >+ def __init__(self, class_name): >+ self.class_name = class_name >+ >+ def __repr__(self): >+ return 'ExceptionWrapper(%s)' % repr(self.class_name) >+ >+# CLI could-be duplicate >+class JvmGeneratedSourceWrapper(object): >+ def __init__(self, gensrc): >+ """ gensrc is an instance of JvmGeneratedSource """ >+ self.gensrc = gensrc >+ >+ def __call__(self, *args): >+ if not self.gensrc.compiled: >+ py.test.skip("Assembly disabled") >+ >+ if getoption('norun'): >+ py.test.skip("Execution disabled") >+ >+ resstr = self.gensrc.execute(args) >+ res = eval(resstr) >+ if isinstance(res, tuple): >+ res = StructTuple(res) # so tests can access tuple elements with .item0, .item1, etc. >+ elif isinstance(res, list): >+ res = OOList(res) >+ return res >+ >+class JvmTest(BaseRtypingTest, OORtypeMixin): >+ def __init__(self): >+ self._func = None >+ self._ann = None >+ self._jvm_src = None >+ >+ def _compile(self, fn, args, ann=None): >+ if ann is None: >+ ann = [lltype_to_annotation(typeOf(x)) for x in args] >+ if self._func is fn and self._ann == ann: >+ return JvmGeneratedSourceWrapper(self._jvm_src) >+ else: >+ self._func = fn >+ self._ann = ann >+ self._jvm_src = generate_source_for_function(fn, ann) >+ if not getoption('noasm'): >+ self._jvm_src.compile() >+ return JvmGeneratedSourceWrapper(self._jvm_src) >+ >+ def _skip_win(self, reason): >+ if platform.system() == 'Windows': >+ py.test.skip('Windows --> %s' % reason) >+ >+ def interpret(self, fn, args, annotation=None): >+ py.test.skip("jvm tests don't work yet") >+ src = self._compile(fn, args, annotation) >+ res = src(*args) >+ if isinstance(res, ExceptionWrapper): >+ raise res >+ return res >+ >+ def interpret_raises(self, exception, fn, args): >+ import exceptions # needed by eval >+ try: >+ self.interpret(fn, args) >+ except ExceptionWrapper, ex: >+ assert issubclass(eval(ex.class_name), exception) >+ else: >+ assert False, 'function did not raise any exception at all' >+ >+ def float_eq(self, x, y): >+ return round(x, FLOAT_PRECISION) == round(y, FLOAT_PRECISION) >+ >+ def ll_to_string(self, s): >+ return s >+ >+ def ll_to_list(self, l): >+ return l >+ >+ def class_name(self, value): >+ return value.class_name.split(".")[-1] >+ >+ def is_of_instance_type(self, val): >+ return isinstance(val, InstanceWrapper) >+ >+ def read_attr(self, obj, name): >+ py.test.skip('read_attr not supported on genjvm tests') > >Added: pypy/dist/pypy/translator/jvm/test/test_bool.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/test/test_bool.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,7 @@ >+import py >+from pypy.translator.jvm.test.runtest import JvmTest >+from pypy.rpython.test.test_rbool import BaseTestRbool >+ >+class TestJvmBool(JvmTest, BaseTestRbool): >+ pass >+ > >Added: pypy/dist/pypy/translator/jvm/typesystem.py >============================================================================== >--- (empty file) >+++ pypy/dist/pypy/translator/jvm/typesystem.py Sun Oct 22 01:36:59 2006 >@@ -0,0 +1,157 @@ >+""" >+Translation between PyPy ootypesystem and JVM type system. >+ >+Here are some tentative non-obvious decisions: >+ >+Signed scalar types mostly map as is. >+ >+Unsigned scalar types are a problem; the basic idea is to store them >+as signed values, but execute special code when working with them. Another >+option would be to use classes, or to use the "next larger" type and remember to use appropriate modulos. The jury is out on >+this. Another idea would be to add a variant type system that does >+not have unsigned values, and write the required helper and conversion >+methods in RPython --- then it could be used for multiple backends. >+ >+Python strings are mapped to byte arrays, not Java Strings, since >+Python strings are really sets of bytes, not unicode code points. >+Jury is out on this as well; this is not the approach taken by cli, >+for example. >+ >+Python Unicode strings, on the other hand, map directly to Java Strings. >+ >+WeakRefs can hopefully map to Java Weak References in a straight >+forward fashion. >+ >+Collections can hopefully map to Java collections instances. Note >+that JVM does not have an idea of generic typing at its lowest level >+(well, they do have signature attributes, but those don't really count >+for much). >+ >+""" >+from pypy.rpython.lltypesystem import lltype, llmemory >+from pypy.rpython.ootypesystem import ootype >+from pypy.translator.jvm.option import getoption >+from pypy.translator.jvm.log import log >+ >+class JvmType(str): >+ """ >+ The class we use to represent JVM types; it is just a string with >+ the JVM type descriptor at the moment. Using JvmType allows us to >+ use isinstance, however. The grammar for type descriptors can be >+ read about here: >+ http://java.sun.com/docs/books/vmspec/2nd-edition/html/ClassFile.doc.html >+ """ >+ def is_scalar(self): >+ return self[0] != 'L' and self[0] != '[' >+ def is_reference(self): >+ return not self.is_scalar() >+ def is_array(self): >+ return self[0] == '[' >+ def int_class_name(self): >+ """ Converts a descriptor like Ljava/lang/Object; to >+ java/lang/Object """ >+ assert self[0] == 'L' and self[-1] == ';' >+ return self[1:-1] >+ def type_width(self): >+ """ Returns number of JVM words this type takes up. JVM words >+ are a theoretically abstract quantity that basically >+ represents 32 bits; so most types are 1, but longs and doubles >+ are 2. """ >+ if self[0] == 'J' or self[0] == 'D': >+ return 2 >+ return 1 >+ >+# JVM type functions >+ >+def jvm_array_of(jtype): >+ """ Returns a JvmType representing an array of 'jtype', which must be >+ another JvmType """ >+ assert isinstance(jtype, JvmType) >+ return JvmType('['+str(jtype)) >+ >+def jvm_for_class(classnm): >+ """ Returns a JvmType representing a particular class 'classnm', which >+ should be a fully qualified java class name (i.e., 'java.lang.String') """ >+ return JvmType('L%s;' % classnm.replace('.','/')) >+ >+# Common JVM types >+jVoid = JvmType('V') >+jInt = JvmType('I') >+jLong = JvmType('J') >+jBool = JvmType('Z') >+jDouble = JvmType('D') >+jByte = JvmType('B') >+jByteArray = jvm_array_of(jByte) >+jChar = JvmType('C') >+jThrowable = jvm_for_class('java.lang.Throwable') >+jObject = jvm_for_class('java.lang.Object') >+jString = jvm_for_class('java.lang.String') >+jStringArray = jvm_array_of(jString) >+jArrayList = jvm_for_class('java.util.ArrayList') >+jHashMap = jvm_for_class('java.util.HashMap') >+jIterator = jvm_for_class('java.util.Iterator') >+jClass = jvm_for_class('java.lang.Class') >+jStringBuilder = jvm_for_class('java.lang.StringBuilder') >+ >+# Map from OOType to an internal JVM type descriptor >+_lltype_to_jvm = { >+ ootype.Void: jVoid, >+ ootype.Signed: jInt, >+ ootype.Unsigned: jInt, >+ lltype.SignedLongLong: jLong, >+ lltype.UnsignedLongLong: jLong, >+ ootype.Bool: jBool, >+ ootype.Float: jDouble, >+ ootype.Char: jByte, >+ ootype.UniChar: jChar, >+ ootype.String: jByteArray, >+ ootype.ROOT: jObject, >+ >+ # We may want to use PyPy wrappers here later: >+ llmemory.WeakGcAddress: jObject, # XXX >+ ootype.StringBuilder: jStringBuilder, >+ ootype.Class: jClass, >+ ootype.List: jArrayList, >+ ootype.Dict: jHashMap, >+ ootype.DictItemsIterator:jIterator >+ } >+ >+# Method descriptor construction >+def jvm_method_desc(argtypes, rettype): >+ """ A Java method has a descriptor, which is a string specified >+ its argument and return types. This function converts a list of >+ argument types (JvmTypes) and the return type (also a JvmType), >+ into one of these descriptor strings. """ >+ return "(%s)%s" % ("".join(argtypes), rettype) >+ >+class JvmTypeSystem(object): >+ >+ """ This object translates between the OOTypeSystem and JVM type >+ descriptors. """ >+ >+ def enforce_jvm(self, typ): >+ if isinstance(typ, JvmType): >+ return typ >+ return self.ootype_to_jvm(typ) >+ >+ def ootype_to_jvm(self, oot): >+ """ Returns an instance of JvmType corresponding to the given >+ OOType """ >+ >+ # Check the easy cases >+ if oot in _lltype_to_jvm: >+ return _lltype_to_jvm[oot] >+ >+ # Now handle the harder ones >+ if isinstance(oot, lltype.Ptr) and isinstance(t.TO, lltype.OpaqueType): >+ return jObject >+ if isinstance(oot, ootype.Instance): >+ return XXX >+ if isinstance(oot, ootype.Record): >+ return XXX >+ if isinstance(oot, ootype.StaticMethod): >+ return XXX >+ >+ # Uh-oh >+ unhandled_case >+ > >Modified: pypy/dist/pypy/translator/oosupport/metavm.py >============================================================================== >--- pypy/dist/pypy/translator/oosupport/metavm.py (original) >+++ pypy/dist/pypy/translator/oosupport/metavm.py Sun Oct 22 01:36:59 2006 >@@ -98,6 +98,10 @@ > > def __call__(self, *args): > return self.render(*args) >+ >+class _DoNothing(MicroInstruction): >+ def render(self, generator, op): >+ pass > > class PushArg(MicroInstruction): > """ Pushes a given operand onto the stack. """ >@@ -227,3 +231,4 @@ > SetField = _SetField() > GetField = _GetField() > DownCast = _DownCast() >+DoNothing = _DoNothing() >_______________________________________________ >pypy-svn mailing list >pypy-svn at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-svn > > Ugh. You've broken a lot of stuff by just copying conftest.py from cli. Basically you cannot do that, because both are loaded at the same time (the reason for that is unclear) and options are conflicting. From fijal at genesilico.pl Sun Oct 22 14:38:21 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 22 Oct 2006 14:38:21 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <20061021233702.32CB210070@code0.codespeak.net> References: <20061021233702.32CB210070@code0.codespeak.net> Message-ID: <453B663D.3040108@genesilico.pl> As well as some of your commits have broken JS tests. I'm not totally convinced that my attempt (returning void) is better than your (not returning void), but please at least run JS tests and check if nothing is broken. If you brake something, but you're totally convinced that I'm the one who should fix something please at least contact me, I would be happy to sort things out. From fijal at genesilico.pl Mon Oct 23 00:26:04 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 00:26:04 +0200 Subject: [pypy-dev] ECMAScript and ML Message-ID: <453BEFFC.2020704@genesilico.pl> Reposted from ES4 discussion (about ECMAScript): > We seem to have chosen ML (OCaml with call/cc, probably) as the meta- > language for the ES4 spec. This is hot of the presses, but it looks > like the right choice, and any change that moves away from it will > take unlikely effort and a better candidate language. We were faced > with the arduous task of inventing our own sound meta-language, and > it started looking like ML with customizations. > This means we will have a reference implementation, with all the > software engineering overhead that implies. But it will be an > implementation whose goal is clarity and soundness, not space or time > performance. And with some work, as Cormac Flanagan points out, the > reference code (or probably a subset of it) could be used with Coq to > do automated proofs. > We will make the reference implementation available in due course, as > open source. We will welcome contributors (Hi, Nicolas! ;-). I think it's quite interesting how it influences our attempt to write down JS interpreter. From niko at alum.mit.edu Mon Oct 23 09:49:28 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Mon, 23 Oct 2006 09:49:28 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453B663D.3040108@genesilico.pl> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> Message-ID: <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> > As well as some of your commits have broken JS tests. I'm not totally > convinced that my attempt (returning void) is better than your (not > returning void), but please at least run JS tests and check if nothing > is broken. If you brake something, but you're totally convinced > that I'm > the one who should fix something please at least contact me, I > would be > happy to sort things out. My sincere apologies! I was trying to be careful not to break things, but evidently not careful enough. I will make sure to run the JS tests in the future --- I didn't in the past because I assumed I did not have the required software. Regarding the change to get_ and set_field to make it ignore Void arguments, I ported that from the CLI code --- I mentioned it on IRC and people thought it was a good idea. I don't really understand too well whether it is necessary or not, but I am afraid that your change might break the CLI tests, as previously they relied upon their own version of the GetField MetaVM op which *did* ignore the Void operations. Again, my apologies for breaking things. :) Niko From fijal at genesilico.pl Mon Oct 23 11:50:01 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 11:50:01 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> Message-ID: <453C9049.6050602@genesilico.pl> Niko Matsakis wrote: >> As well as some of your commits have broken JS tests. I'm not totally >> convinced that my attempt (returning void) is better than your (not >> returning void), but please at least run JS tests and check if nothing >> is broken. If you brake something, but you're totally convinced that >> I'm >> the one who should fix something please at least contact me, I would be >> happy to sort things out. > > > My sincere apologies! I was trying to be careful not to break > things, but evidently not careful enough. I will make sure to run > the JS tests in the future --- I didn't in the past because I assumed > I did not have the required software. > > Regarding the change to get_ and set_field to make it ignore Void > arguments, I ported that from the CLI code --- I mentioned it on IRC > and people thought it was a good idea. I don't really understand too > well whether it is necessary or not, but I am afraid that your change > might break the CLI tests, as previously they relied upon their own > version of the GetField MetaVM op which *did* ignore the Void > operations. > > Again, my apologies for breaking things. :) > > > Niko Sorry for being too agressive :-] Actually I do agree with you and I'll make JS working that way. Just please inform me each time you change something which breaks tests explicitely. Cheers, Fijal From cfbolz at gmx.de Mon Oct 23 11:59:00 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 23 Oct 2006 11:59:00 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453C9049.6050602@genesilico.pl> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> <453C9049.6050602@genesilico.pl> Message-ID: <453C9264.4070200@gmx.de> Maciek Fijalkowski wrote: > Niko Matsakis wrote: > >>> As well as some of your commits have broken JS tests. I'm not totally >>> convinced that my attempt (returning void) is better than your (not >>> returning void), but please at least run JS tests and check if nothing >>> is broken. If you brake something, but you're totally convinced that >>> I'm >>> the one who should fix something please at least contact me, I would be >>> happy to sort things out. >> >> My sincere apologies! I was trying to be careful not to break >> things, but evidently not careful enough. I will make sure to run >> the JS tests in the future --- I didn't in the past because I assumed >> I did not have the required software. >> >> Regarding the change to get_ and set_field to make it ignore Void >> arguments, I ported that from the CLI code --- I mentioned it on IRC >> and people thought it was a good idea. I don't really understand too >> well whether it is necessary or not, but I am afraid that your change >> might break the CLI tests, as previously they relied upon their own >> version of the GetField MetaVM op which *did* ignore the Void >> operations. >> >> Again, my apologies for breaking things. :) I think it happened to everybody at one point. > Sorry for being too agressive :-] Actually I do agree with you and I'll > make JS working that way. Just please inform me each time you change > something which breaks tests explicitely. Well, actually your fix broke the cli tests :-). Cheers, Carl Friedrich From fijal at genesilico.pl Mon Oct 23 12:01:48 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Oct 2006 12:01:48 +0200 Subject: [pypy-dev] [pypy-svn] r33528 - in pypy/dist/pypy/translator: jvm jvm/src jvm/test oosupport In-Reply-To: <453C9264.4070200@gmx.de> References: <20061021233702.32CB210070@code0.codespeak.net> <453B663D.3040108@genesilico.pl> <316816B7-1417-4CD0-9259-BFA9ECF0A961@alum.mit.edu> <453C9049.6050602@genesilico.pl> <453C9264.4070200@gmx.de> Message-ID: <453C930C.5090009@genesilico.pl> Carl Friedrich Bolz wrote: >Maciek Fijalkowski wrote: > > >>Niko Matsakis wrote: >> >> >> >>>>As well as some of your commits have broken JS tests. I'm not totally >>>>convinced that my attempt (returning void) is better than your (not >>>>returning void), but please at least run JS tests and check if nothing >>>>is broken. If you brake something, but you're totally convinced that >>>>I'm >>>>the one who should fix something please at least contact me, I would be >>>>happy to sort things out. >>>> >>>> >>>My sincere apologies! I was trying to be careful not to break >>>things, but evidently not careful enough. I will make sure to run >>>the JS tests in the future --- I didn't in the past because I assumed >>>I did not have the required software. >>> >>>Regarding the change to get_ and set_field to make it ignore Void >>>arguments, I ported that from the CLI code --- I mentioned it on IRC >>>and people thought it was a good idea. I don't really understand too >>>well whether it is necessary or not, but I am afraid that your change >>>might break the CLI tests, as previously they relied upon their own >>>version of the GetField MetaVM op which *did* ignore the Void >>>operations. >>> >>>Again, my apologies for breaking things. :) >>> >>> > >I think it happened to everybody at one point. > > > > >>Sorry for being too agressive :-] Actually I do agree with you and I'll >>make JS working that way. Just please inform me each time you change >>something which breaks tests explicitely. >> >> > >Well, actually your fix broke the cli tests :-). > >Cheers, > >Carl Friedrich > > Yeah, I know. Didn't know that CLI uses oosupport, I've fixed all by now I guess. My apologies this time ;-) From jeff at taupro.com Tue Oct 31 07:19:14 2006 From: jeff at taupro.com (Jeff Rush) Date: Tue, 31 Oct 2006 00:19:14 -0600 Subject: [pypy-dev] Invitation to Present at PyCon 2007 Message-ID: <4546EAE2.7010005@taupro.com> The deadline for submitting a talk for PyCon 2007, to be held Feb 23-25 in Addison (Dallas), Texas is upon us. I've been looking through the database of submitted talk proposals and I don't see anything related to PyPy. I would like to invite the PyPy community to present on some aspect of their project, as they did at PyCon 2006 and hopefully make some new converts. The deadline for talk submission is midnight Oct 31. To give a talk, go to the following page, create a speaker account and provide a brief proposal. http://us.pycon.org/TX2007/CallForProposals We're also collecting ideas on talks in general on the following wiki page, in case anyone is looking for inspiration. http://us.pycon.org/TX2007/TalkIdeas And there will be presentations on the Parrot VM, with which a comparison against PyPy would be quite interesting. I hope to see many of you in Dallas! Jeff Rush Co-Chair PyCon 2007 From alexandre.fayolle at logilab.fr Thu Nov 2 15:38:10 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 2 Nov 2006 15:38:10 +0100 Subject: [pypy-dev] IRC server unreachable Message-ID: <20061102143809.GC26886@crater.logilab.fr> Hi, am I the only one unable to join any freenode IRC servers ? I get connection refused errors on servers listed on http://freenode.net/irc_servers.shtml Or has the IRC network for pypy changed ? -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061102/b199d727/attachment.pgp From sdouche at gmail.com Thu Nov 2 16:00:25 2006 From: sdouche at gmail.com (Sebastien Douche) Date: Thu, 2 Nov 2006 16:00:25 +0100 Subject: [pypy-dev] IRC server unreachable In-Reply-To: <20061102143809.GC26886@crater.logilab.fr> References: <20061102143809.GC26886@crater.logilab.fr> Message-ID: <5e1183fa0611020700r60c56374i95ef1dd20969fe3c@mail.gmail.com> On 11/2/06, Alexandre Fayolle wrote: > Hi, > > am I the only one unable to join any freenode IRC servers ? I get > connection refused errors on servers listed on > http://freenode.net/irc_servers.shtml Hi Alexandre, hugh DDOS on Freenode since 11h. -- S?bastien Douche From arigo at tunes.org Thu Nov 2 17:52:02 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 2 Nov 2006 17:52:02 +0100 Subject: [pypy-dev] schedule() In-Reply-To: <20060918084726.GB31821@crater.logilab.fr> References: <20060918075428.GA19879@code0.codespeak.net> <20060918084726.GB31821@crater.logilab.fr> Message-ID: <20061102165201.GA8864@code0.codespeak.net> Hi Aurelien, On Mon, Sep 18, 2006 at 10:47:26AM +0200, Aur?lien Camp?as wrote: > > I have to ask again. Do you want me to add a hook in the logic objspace > > that calls schedule() every Nth bytecode instruction? > > Sure. See interpreter/test/test_executioncontext.py, or look at how module/thread/__init__.py adds an action that is called back every Nth bytecode instruction. A bientot, Armin From alexandre.fayolle at logilab.fr Thu Nov 2 21:34:28 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 2 Nov 2006 21:34:28 +0100 Subject: [pypy-dev] schedule() In-Reply-To: <20061102165201.GA8864@code0.codespeak.net> References: <20060918075428.GA19879@code0.codespeak.net> <20060918084726.GB31821@crater.logilab.fr> <20061102165201.GA8864@code0.codespeak.net> Message-ID: <20061102203428.GB10288@crater.logilab.fr> On Thu, Nov 02, 2006 at 05:52:02PM +0100, Armin Rigo wrote: > Hi Aurelien, > > On Mon, Sep 18, 2006 at 10:47:26AM +0200, Aur?lien Camp?as wrote: > > > I have to ask again. Do you want me to add a hook in the logic objspace > > > that calls schedule() every Nth bytecode instruction? > > > > Sure. > > See interpreter/test/test_executioncontext.py, or look at how > module/thread/__init__.py adds an action that is called back every Nth > bytecode instruction. Great ! Thanks a lot, Armin. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061102/c481b9e1/attachment-0001.pgp From cfbolz at gmx.de Sun Nov 5 15:43:38 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 05 Nov 2006 15:43:38 +0100 Subject: [pypy-dev] Duesseldorf sprint report Message-ID: <454DF89A.3090007@gmx.de> Hi all! it is with a familiar level of tiredness that we bring you these lines. We are again sitting in one of the rooms of the Institut f?r Informatik, after 6 days of sprinting. As usual, it has been a busy and productive (and sometimes strange) sprint. One of the new developments of the sprint was the work of Leonardo Santadaga, our "Summer" of PyPy student from Brazil. Leonardo proposed to write a JavaScript interpreter, had his proposal accepted and now gets his travel to sprints funded. This work has seen good progress every day, so that we now have an interpreter that handles simple snippet of JavaScript code. Leonardo had help from various other people changing over the course of the sprint such as Maciek, Guido (the reluctant Master of JavaScript) and Stephan. The parser is currently stolen from them Narcissus project, and the interpreter does not translate yet. For less than a weeks work though we think are doing pretty well (we are trying not to distract ourselves with crazy thoughts like translating a JS interpreter to JS or wondering how fast it would be after applying the magic JIT technology). Although Leonardo will be flying back to Brazil soon he will continue working on it (at least if he finds sufficient time between caring for his beloved new MacBook). The other PyPy sprint virgin was Niko Matsakis, a graduate student at ETH Z?rich. To start with he worked with help of Antonio on the fledgling JVM backend. Antonio and Niko worked on moving code out of the CLI backend to be shared with other object-oriented backends. They got as far as supporting nearly everything except constants (which as usual turns out to be the hardest thing to support). The team was split up later in the week to work on Other Things. Niko only found out after he _left_ the sprint that Samuele is one of the authors of Jython, which he is interested in since some of his students are working on it. Another topic that received attention all week was the JIT (dramatic music). The architecture of the JIT has now crystallized enough to be able to split up the work into the (for mere mortals largely incomprehensible) "time shifting" front end and the (much more straightforward) code generating backends. Samuele and Arre started the week by adding some missing pieces for things called "portals" in the front end. A portal is the part of your code that marks the entry into JIT-land. For the PyPy interpreter it is expected to be the bytecode dispatcher (or in general the main loop of an interpreter). The JIT tests now take so long that they also had time to work on translating rsocket, which is an RPython-level socket library, and using it for implementing the socket module. For the first few days of the sprint Armin and Richard took a DFA engine (deterministic finite automaton) that Carl Friedrich wrote to lex Prolog code and adapted it to be jittable. On Wednesday Armin tried cast some light on the dark internals of the JIT with a short talk about this example. The DFA engine is viewed as interpreting the specification of the automaton and then turned into a compiler by the timeshifter. This compiler does not work in advance. Instead calls to the compiler are inserted into the transformed version of the DFA engine. If the DFA engine is used to match a string against the automaton, the compiler produces somewhat efficient machine code for doing that. Richard's work (with the help of Armin and Arre) for the rest of the sprint can be summarized as improving the timeshifter to be able to "un-adapt" this DFA engine until it at least resembled the original code. This involved adding a new piece of terminology to the ever-growing JIT jargon "deep-freezing" (something to do with structures that are so constant that even their contents are constant). Concurrently, and with its own set of problems, Michael worked on the JIT backend all week. He started together with Armin to document all the sometimes non-intuitively named methods of the backend API, which has only emerged in the last month or so. They renamed the most strange ones afterwards. After that Michael and Eric did a little polishing of the PowerPC backend and began to investigate writing an LLVM backend (C++ is hard). Eric had to leave mid-sprint so Michael continued with Niko (another Mac fanatic with insufficient funds to have a MacBook) to improve the PowerPC backend. This involved fights with the calling conventions, writing a greedy register allocator and lots of time waiting for tests (they didn't manage to find a nice side-project). At the end of the week the PPC backend can actually handle no more programs than it could at the beginning, but we are very much happier about its foundations. After the break-day Samuele and Maciek started working on the project that would occupy them for the rest of the sprint: adding "transparent proxy objects" to the PyPy interpreter. A transparent proxy is something that claims to be of a specific type (e.g. a list) but all operations are forwarded to a controller which can choose how to implement those in an arbitrary way, such as retrieving the values from a database, a remote host or similar. For most operations the multi-method based implementation of the standard object space made this relatively straightforward. For the black-sheep operations this was some more work but this is still something that, according to Samuele, "if you tried this with CPython it would take so long that you wouldn't want it anymore when you're finished". The motivation for writing these proxy thingies is to experiment with transparent remote objects, orthogonal persistence and security (there is also a plan to steal the pygame from CPython by using them). There was a little work on the side on the upcoming "apigen" tool, which is a new part of the py-lib. This is something Maciek, Guido and Brian Dorsey started before the sprint. It runs all the tests of a package (currently only the py-lib) and observes the behaviour of the running code. From this it generates API documentation with additional information like what types the arguments of the functions had, which attributes of instances were changed... Carl Friedrich spent quite some time on the deeply brain-taxing task of populating the new rlib directory by moving various files there and fixing all the imports. The rlib directory is meant to contain a library useable from RPython programs, so the code in there is all RPython and is supposed to make up in some way for the lack of a standard library in RPython. Such modules have been accumulating in various places where they don't really belong. Samuele, Anto and Carl Friedrich had a planning meeting for the integration of the .NET framework with the Python interpreter that Anto plans to do. This means being able to call .NET functions and use .NET classes from pypy.net. Samuele presented the two major approaches: The simpler, but more annoying one is to write wrapper classes that have the actual .net object as an attribute. For this you need to do conversions at the language boundaries and thus suffer a performance penalty. The other approach is to identify some of the classes of the PyPy interpreter with .NET classes so that you can pass them around freely. Guido and Carl Friedrich worked a small bit on the build tool, which supposedly gives people a way to build PyPy on remote hosts. This is now somewhat working modulo lots of annoying real-life details (error-handling being one of them). Together with Samuele they had a discussion about what to do in this area in the future. Various realistic ideas (such as a web frontend) and unrealistic ideas (having a quake configuration tool where you can hunt down and kill configuration settings that you don't want) were discussed. A very positive aspect of the sprint is that Christian who has been ill for quite a while is back with us and now finding all the things we have broken on windows in the meantime. Carl Friedrich with the help of Anto and Guido spent a big chunk of time near the end of the sprint in refactoring PyPy's file implementation. The first step was converting the stream implementation which was living at app-level before to be RPython and thus useable by other RPython programs (such as Prolog interpreters, hint hint). Afterwards they converted the file implementation to use the new facilities, which involved the usual fighting with the annotator to make it work. This should also make it possible to remove some horrendous hacks from the .NET backend, as the new file implementation should make it easier to move away from using a POSIX-like model (actually the .NET backend contained a POSIX compatibility layer written by Anto just for this purpose). Since we cannot seem to come up with a witty closing remark, we just stop. Ade, Carl Friedrich & mwh -- "... and the end result of all this is a dating site that matches people according to the sort of PyPy they compile." -- Samuele Pedroni From jacob at strakt.com Sun Nov 5 16:44:55 2006 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Sun, 5 Nov 2006 16:44:55 +0100 Subject: [pypy-dev] Duesseldorf sprint report In-Reply-To: <454DF89A.3090007@gmx.de> References: <454DF89A.3090007@gmx.de> Message-ID: <200611051644.55257.jacob@strakt.com> Congratulations to you all. You have been extremely productive and seem to have managed to have fun at the same time. Wish I could have been there. Jacob Hall?n From paul.degrandis at gmail.com Sun Nov 5 18:46:29 2006 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Sun, 5 Nov 2006 12:46:29 -0500 Subject: [pypy-dev] A new idea? Message-ID: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> Hi everyone, I started to work on PyPy about a year to a year and a half ago, but had to quickly drop the project because of school and academic research I'm involved in. I really feel that even though PyPy is still at a stage of rapid development and evolution it's about time that people start really illustrating the true power of pypy. I'd like to be that guy for you. My idea is simple, I'll write small but useful applications that really illustrate the powers of pypy. These small applications can be loosely strung together as tutorials of language features or just examples of pypy's superiority. For some time now, I've developed small extensions to CPython in form of models, c extensions, and decorations to add features to the language that I find useful. However, Pypy's architecture naturally lends itself to these tweaks and in some cases supports them already. So basically, I want to know your thoughts on this. And if you have any ideas for applications or features you'd like to see illustrated please let me know! Regards, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061105/fd549a6c/attachment.htm From cfbolz at gmx.de Wed Nov 8 16:06:09 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 08 Nov 2006 16:06:09 +0100 Subject: [pypy-dev] Summer of PyPy Call for Proposals Message-ID: <4551F261.6050108@gmx.de> Last chance to join the Summer of PyPy! ======================================= Hopefully by now you have heard of the "Summer of PyPy", our program for funding the expenses of attending a sprint for students. If not, you've just read the essence of the idea :-) However, the PyPy EU funding period is drawing to an end and there is now only one sprint left where we can sponsor the travel costs of interested students within our program. This sprint will probably take place in Leysin, Switzerland from 8th-14th of January 2007. So, as explained in more detail at: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html we would encourage any interested students to submit a proposal in the next month or so. If you're stuck for ideas, you can find some at http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html but please do not feel limited in any way by this list! Cheers, Carl Friedrich Bolz and the PyPy team From mistobaan at gmail.com Thu Nov 9 15:22:11 2006 From: mistobaan at gmail.com (Fabrizio Milo aka misto) Date: Thu, 9 Nov 2006 15:22:11 +0100 Subject: [pypy-dev] JIT improvements for RISC processors Message-ID: Hi to everyone, I am really interested in code machine generation for risc processors. In order to have an idea of the work that can be done, and to create a proposal I will ask some questions: - What is already done? where is it? (paths, file names please) - What ideas are already defined and need just to be implemented? - Material that I have to / should read? I have a rough background on RISC assembler, the most comes from MIPS and IBM's PowerPC some material I am aware of is http://tinyurl.com/9dw43 Don't let me escape for the second (last?) time! :) ---------------------------- * P~ython Addicted * www.fabriziomilo.it From pedronis at strakt.com Thu Nov 9 15:53:58 2006 From: pedronis at strakt.com (Samuele Pedroni) Date: Thu, 09 Nov 2006 15:53:58 +0100 Subject: [pypy-dev] interesting paper on a object-oriented language for very constrained embedded systems Message-ID: <45534106.6010101@strakt.com> http://compilers.cs.ucla.edu/virgil/virgil-oopsla06.pdf RPython for different reasons or by different means does similar things already (devirtualisation, separate flexible load time) and could be adapted to implement other aspects/optimisations as well, under the no new allocations assumption. regards. From mwh at python.net Wed Nov 15 12:11:22 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 15 Nov 2006 12:11:22 +0100 Subject: [pypy-dev] JIT improvements for RISC processors References: Message-ID: <87wt5xotgl.fsf@starship.python.net> I was kind of hoping you would be in #pypy sometime in the last couple of days to talk about this. "Fabrizio Milo aka misto" writes: > Hi to everyone, > > I am really interested in code machine generation for risc processors. > In order to have an idea of the work that can be done, and to create > a proposal I will ask some questions: > > - What is already done? where is it? (paths, file names please) Well, the code generation code is all in pypy.jit.codegen; pypy.jit.codegen.model is a good starting place. There is a somewhat complete x86 backend and a less complete PowerPC backend (although I'm working on that right now and it will maybe have caught up by the end of the week, in terms of functionality at least). > - What ideas are already defined and need just to be implemented? Well, the interface in the model file just referenced is more or less defined, although there are a few details that it would be good to change to make decent register allocation possible. > - Material that I have to / should read? > > I have a rough background on RISC assembler, > the most comes from MIPS and IBM's PowerPC > > some material I am aware of is http://tinyurl.com/9dw43 Also worth reading are IBM's "Compiler Writer's Guide" and Apple's "LowLevelABI.pdf": http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/852569B20050FF7785256996007558C6 http://gemma.apple.com/documentation/developertools/Conceptual/LowLevelABI/LowLevelABI.pdf For a Summer of PyPy project there needs to be some reasonably clearly defined goals. An obvious and useful target is improving the quality of the generated PowerPC code -- I'm making it work at the moment, I'm certainly not making it fast. Another good thing would be sharing some code between the PPC and x86 code generation (not so much the actual code generation, but there are things about memory management of code buffers and so on that are likely to be similar between any machine code backend). And once this is done, adding another backend for say MIPS should be reasonably straightforward. Cheers, mwh -- Those who have deviant punctuation desires should take care of their own perverted needs. -- Erik Naggum, comp.lang.lisp From edreamleo at charter.net Wed Nov 29 02:59:15 2006 From: edreamleo at charter.net (Edward K. Ream) Date: Tue, 28 Nov 2006 19:59:15 -0600 Subject: [pypy-dev] Crash starting pypy on XP Message-ID: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> Hello all, Excellent documentation at: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html#downloading-running-the-pypy-0-9-release Alas, when I execute python pypy/bin/py.py from C:\pypy-0.9.0 on XP I get: C:\pypy-0.9.0>python pypy/bin/py.py Traceback (most recent call last): File "pypy/bin/py.py", line 207, in sys.exit(main_(sys.argv)) File "pypy/bin/py.py", line 78, in main_ space = make_objspace(Options) File "pypy/bin/py.py", line 66, in make_objspace compiler = cmdlineopt.compiler, File "C:\pypy-0.9.0\pypy\interpreter\baseobjspace.py", line 164, in __init__ self.initialize() File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 98, in initialize w_mod = self.setup_exceptions() File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 219, in setup_exceptions mod, w_dic = self.create_builtin_module('_exceptions.py', 'exceptions') File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 164, in create_builtin_module w_dic = PyPyCacheDir.build_applevelinterp_dict(fake, self) File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 840, in build_applevelinterp_dict cls._setup() File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 895, in _setup p = lp(pypy.__file__).new(basename='_cache').ensure(dir=1) File "C:\pypy-0.9.0\py\path\local\local.py", line 276, in ensure return p._ensuredirs() File "C:\pypy-0.9.0\py\path\local\local.py", line 265, in _ensuredirs if self.check(dir=0): File "C:\pypy-0.9.0\py\path\common.py", line 102, in check return self.Checkers(self)._evaluate(kw) File "C:\pypy-0.9.0\py\path\common.py", line 75, in _evaluate if bool(value) ^ bool(meth()) ^ invert: File "C:\pypy-0.9.0\py\path\local\local.py", line 35, in dir return stat.S_ISDIR(self._stat().mode) File "C:\pypy-0.9.0\py\path\local\local.py", line 29, in _stat self._statcache = self.path.stat() File "C:\pypy-0.9.0\py\path\local\local.py", line 285, in stat stat = self._callex(os.stat, self.strpath) File "C:\pypy-0.9.0\py\path\common.py", line 205, in _callex return func(*args) WindowsError: [Error 2] The system cannot find the file specified: 'C:\\pypy-0.9.0\\pypy\\_cache' Any help getting started would be appreciated. Thanks. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at charter.net Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From santagada at gmail.com Wed Nov 29 10:02:07 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 29 Nov 2006 07:02:07 -0200 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> Message-ID: <2f2e5f950611290102n68d9d5f7i519b1b5d3958df@mail.gmail.com> it seems to be a bug on py.path... it seems to be doing stat caches... strange, maybe you can just coment out the stat caching for win32. Just so you know I never worked with py lib internals... just giving my 2 cents. -- Leonardo Santagada (http://www.lomohomes.com/retype) From mwh at python.net Wed Nov 29 10:29:55 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 29 Nov 2006 10:29:55 +0100 Subject: [pypy-dev] Crash starting pypy on XP References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <2f2e5f950611290102n68d9d5f7i519b1b5d3958df@mail.gmail.com> Message-ID: <87k61eiorw.fsf@starship.python.net> "Leonardo Santagada" writes: > it seems to be a bug on py.path... it seems to be doing stat caches... > strange, maybe you can just coment out the stat caching for win32. Earth pinging Fijal... > Just so you know I never worked with py lib internals... just giving > my 2 cents. It makes some sense. I was confused for a while because Ed mentioned that he was using pypy-0.9.0... but of course, even if you check out the 0.9.0 tag (Ed, is this what you did?), you get the trunk py lib, because of the way svn:externals work. I guess the release procedures should include doing something about that, like svn cp-ing a particular snapshot of the py lib into the tag. Cheers, mwh -- 42. You can measure a programmer's perspective by noting his attitude on the continuing vitality of FORTRAN. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From shafranov at gmail.com Wed Nov 29 10:59:22 2006 From: shafranov at gmail.com (Alexander Shafranov) Date: Wed, 29 Nov 2006 12:59:22 +0300 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> Message-ID: <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> Hi, I also noticed this problem, but when I used python 2.4 instead of 2.5problem disappeared. On 11/29/06, Edward K. Ream wrote: > > Hello all, > > Excellent documentation at: > > > http://codespeak.net/pypy/dist/pypy/doc/getting-started.html#downloading-running-the-pypy-0-9-release > > Alas, when I execute python pypy/bin/py.py from C:\pypy-0.9.0 on XP I get: > > C:\pypy-0.9.0>python pypy/bin/py.py > Traceback (most recent call last): > File "pypy/bin/py.py", line 207, in > sys.exit(main_(sys.argv)) > File "pypy/bin/py.py", line 78, in main_ > space = make_objspace(Options) > File "pypy/bin/py.py", line 66, in make_objspace > compiler = cmdlineopt.compiler, > File "C:\pypy-0.9.0\pypy\interpreter\baseobjspace.py", line 164, in > __init__ > self.initialize() > File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 98, in initialize > w_mod = self.setup_exceptions() > File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 219, in > setup_exceptions > mod, w_dic = self.create_builtin_module('_exceptions.py', 'exceptions') > File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 164, in > create_builtin_module > w_dic = PyPyCacheDir.build_applevelinterp_dict(fake, self) > File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 840, in > build_applevelinterp_dict > cls._setup() > File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 895, in _setup > p = lp(pypy.__file__).new(basename='_cache').ensure(dir=1) > File "C:\pypy-0.9.0\py\path\local\local.py", line 276, in ensure > return p._ensuredirs() > File "C:\pypy-0.9.0\py\path\local\local.py", line 265, in _ensuredirs > if self.check(dir=0): > File "C:\pypy-0.9.0\py\path\common.py", line 102, in check > return self.Checkers(self)._evaluate(kw) > File "C:\pypy-0.9.0\py\path\common.py", line 75, in _evaluate > if bool(value) ^ bool(meth()) ^ invert: > File "C:\pypy-0.9.0\py\path\local\local.py", line 35, in dir > return stat.S_ISDIR(self._stat().mode) > File "C:\pypy-0.9.0\py\path\local\local.py", line 29, in _stat > self._statcache = self.path.stat() > File "C:\pypy-0.9.0\py\path\local\local.py", line 285, in stat > stat = self._callex(os.stat, self.strpath) > File "C:\pypy-0.9.0\py\path\common.py", line 205, in _callex > return func(*args) > WindowsError: [Error 2] The system cannot find the file specified: > 'C:\\pypy-0.9.0\\pypy\\_cache' > > Any help getting started would be appreciated. Thanks. > > Edward > -------------------------------------------------------------------- > Edward K. Ream email: edreamleo at charter.net > Leo: http://webpages.charter.net/edreamleo/front.html > -------------------------------------------------------------------- > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061129/de032641/attachment.htm From hpk at trillke.net Wed Nov 29 11:03:06 2006 From: hpk at trillke.net (holger krekel) Date: Wed, 29 Nov 2006 11:03:06 +0100 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> Message-ID: <20061129100306.GO11948@solar.trillke.net> On Wed, Nov 29, 2006 at 12:59 +0300, Alexander Shafranov wrote: > Hi, > > I also noticed this problem, but when I used python 2.4 instead of > 2.5problem disappeared. hum, let me note that i think the problem has to do with Windows Error conversions, see the somewhat icky _callex() function in py/path/common.py. Fixing this probably is easier with having an XP box and Python 2.5 around (thanks Alexander for pointing to it!). best, holger > > > On 11/29/06, Edward K. Ream wrote: > > > >Hello all, > > > >Excellent documentation at: > > > > > >http://codespeak.net/pypy/dist/pypy/doc/getting-started.html#downloading-running-the-pypy-0-9-release > > > >Alas, when I execute python pypy/bin/py.py from C:\pypy-0.9.0 on XP I > >get: > > > >C:\pypy-0.9.0>python pypy/bin/py.py > >Traceback (most recent call last): > >File "pypy/bin/py.py", line 207, in > > sys.exit(main_(sys.argv)) > >File "pypy/bin/py.py", line 78, in main_ > > space = make_objspace(Options) > >File "pypy/bin/py.py", line 66, in make_objspace > > compiler = cmdlineopt.compiler, > >File "C:\pypy-0.9.0\pypy\interpreter\baseobjspace.py", line 164, in > >__init__ > > self.initialize() > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 98, in > >initialize > > w_mod = self.setup_exceptions() > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 219, in > >setup_exceptions > > mod, w_dic = self.create_builtin_module('_exceptions.py', > > 'exceptions') > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 164, in > >create_builtin_module > > w_dic = PyPyCacheDir.build_applevelinterp_dict(fake, self) > >File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 840, in > >build_applevelinterp_dict > > cls._setup() > >File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 895, in _setup > > p = lp(pypy.__file__).new(basename='_cache').ensure(dir=1) > >File "C:\pypy-0.9.0\py\path\local\local.py", line 276, in ensure > > return p._ensuredirs() > >File "C:\pypy-0.9.0\py\path\local\local.py", line 265, in _ensuredirs > > if self.check(dir=0): > >File "C:\pypy-0.9.0\py\path\common.py", line 102, in check > > return self.Checkers(self)._evaluate(kw) > >File "C:\pypy-0.9.0\py\path\common.py", line 75, in _evaluate > > if bool(value) ^ bool(meth()) ^ invert: > >File "C:\pypy-0.9.0\py\path\local\local.py", line 35, in dir > > return stat.S_ISDIR(self._stat().mode) > >File "C:\pypy-0.9.0\py\path\local\local.py", line 29, in _stat > > self._statcache = self.path.stat() > >File "C:\pypy-0.9.0\py\path\local\local.py", line 285, in stat > > stat = self._callex(os.stat, self.strpath) > >File "C:\pypy-0.9.0\py\path\common.py", line 205, in _callex > > return func(*args) > >WindowsError: [Error 2] The system cannot find the file specified: > >'C:\\pypy-0.9.0\\pypy\\_cache' > > > >Any help getting started would be appreciated. Thanks. > > > >Edward > >-------------------------------------------------------------------- > >Edward K. Ream email: edreamleo at charter.net > >Leo: http://webpages.charter.net/edreamleo/front.html > >-------------------------------------------------------------------- > > > > > > > >_______________________________________________ > >pypy-dev at codespeak.net > >http://codespeak.net/mailman/listinfo/pypy-dev > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From shafranov at gmail.com Wed Nov 29 11:28:25 2006 From: shafranov at gmail.com (Alexander Shafranov) Date: Wed, 29 Nov 2006 13:28:25 +0300 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <20061129100306.GO11948@solar.trillke.net> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> <20061129100306.GO11948@solar.trillke.net> Message-ID: <74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> Hm, With 2.5 on XP box: D:\Tools\pypy\pypy-0.9.0\py\path>py.test inserting into sys.path: D:\Tools\pypy\pypy-0.9.0 Traceback (most recent call last): File "D:\Tools\pypy\pypy-0.9.0\py\bin\win32\\..\py.test", line 4, in py.test.cmdline.main() File "D:\Tools\pypy\pypy-0.9.0\py\test\cmdline.py", line 13, in main config, args = py.test.Config.parse(args) File "D:\Tools\pypy\pypy-0.9.0\py\initpkg.py", line 180, in __getattr__ result = self.__package__._resolve(extpy) File "D:\Tools\pypy\pypy-0.9.0\py\initpkg.py", line 68, in _resolve implmodule = self._loadimpl(fspath[:-3]) File "D:\Tools\pypy\pypy-0.9.0\py\initpkg.py", line 98, in _loadimpl return __import__(modpath, None, None, ['__doc__']) File "D:\Tools\pypy\pypy-0.9.0\py\test\config.py", line 6, in defaultconfig = py.magic.autopath().dirpath('defaultconftest.py') File "D:\Tools\pypy\pypy-0.9.0\py\magic\autopath.py", line 32, in autopath if basefile in current: File "D:\Tools\pypy\pypy-0.9.0\py\path\common.py", line 110, in __contains__ return self.join(other).check() File "D:\Tools\pypy\pypy-0.9.0\py\path\common.py", line 102, in check return self.Checkers(self)._evaluate(kw) File "D:\Tools\pypy\pypy-0.9.0\py\path\common.py", line 75, in _evaluate if bool(value) ^ bool(meth()) ^ invert: File "D:\Tools\pypy\pypy-0.9.0\py\path\local\local.py", line 41, in exists return self._stat() File "D:\Tools\pypy\pypy-0.9.0\py\path\local\local.py", line 29, in _stat self._statcache = self.path.stat() File "D:\Tools\pypy\pypy-0.9.0\py\path\local\local.py", line 285, in stat stat = self._callex(os.stat, self.strpath) File "D:\Tools\pypy\pypy-0.9.0\py\path\common.py", line 205, in _callex return func(*args) WindowsError: [Error 2] The system cannot find the file specified: 'D:\\Tools\\pypy\\pypy-0.9.0\\__init__.py' On 11/29/06, holger krekel wrote: > > On Wed, Nov 29, 2006 at 12:59 +0300, Alexander Shafranov wrote: > > Hi, > > > > I also noticed this problem, but when I used python 2.4 instead of > > 2.5problem disappeared. > > hum, let me note that i think the problem has to do > with Windows Error conversions, see the somewhat > icky _callex() function in py/path/common.py. > Fixing this probably is easier with having an XP box > and Python 2.5 around (thanks Alexander for pointing to it!). > > best, > > holger > > > > > > > On 11/29/06, Edward K. Ream wrote: > > > > > >Hello all, > > > > > >Excellent documentation at: > > > > > > > > > > http://codespeak.net/pypy/dist/pypy/doc/getting-started.html#downloading-running-the-pypy-0-9-release > > > > > >Alas, when I execute python pypy/bin/py.py from C:\pypy-0.9.0 on XP I > > >get: > > > > > >C:\pypy-0.9.0>python pypy/bin/py.py > > >Traceback (most recent call last): > > >File "pypy/bin/py.py", line 207, in > > > sys.exit(main_(sys.argv)) > > >File "pypy/bin/py.py", line 78, in main_ > > > space = make_objspace(Options) > > >File "pypy/bin/py.py", line 66, in make_objspace > > > compiler = cmdlineopt.compiler, > > >File "C:\pypy-0.9.0\pypy\interpreter\baseobjspace.py", line 164, in > > >__init__ > > > self.initialize() > > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 98, in > > >initialize > > > w_mod = self.setup_exceptions() > > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 219, in > > >setup_exceptions > > > mod, w_dic = self.create_builtin_module('_exceptions.py', > > > 'exceptions') > > >File "C:\pypy-0.9.0\pypy\objspace\std\objspace.py", line 164, in > > >create_builtin_module > > > w_dic = PyPyCacheDir.build_applevelinterp_dict(fake, self) > > >File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 840, in > > >build_applevelinterp_dict > > > cls._setup() > > >File "C:\pypy-0.9.0\pypy\interpreter\gateway.py", line 895, in _setup > > > p = lp(pypy.__file__).new(basename='_cache').ensure(dir=1) > > >File "C:\pypy-0.9.0\py\path\local\local.py", line 276, in ensure > > > return p._ensuredirs() > > >File "C:\pypy-0.9.0\py\path\local\local.py", line 265, in _ensuredirs > > > if self.check(dir=0): > > >File "C:\pypy-0.9.0\py\path\common.py", line 102, in check > > > return self.Checkers(self)._evaluate(kw) > > >File "C:\pypy-0.9.0\py\path\common.py", line 75, in _evaluate > > > if bool(value) ^ bool(meth()) ^ invert: > > >File "C:\pypy-0.9.0\py\path\local\local.py", line 35, in dir > > > return stat.S_ISDIR(self._stat().mode) > > >File "C:\pypy-0.9.0\py\path\local\local.py", line 29, in _stat > > > self._statcache = self.path.stat() > > >File "C:\pypy-0.9.0\py\path\local\local.py", line 285, in stat > > > stat = self._callex(os.stat, self.strpath) > > >File "C:\pypy-0.9.0\py\path\common.py", line 205, in _callex > > > return func(*args) > > >WindowsError: [Error 2] The system cannot find the file specified: > > >'C:\\pypy-0.9.0\\pypy\\_cache' > > > > > >Any help getting started would be appreciated. Thanks. > > > > > >Edward > > >-------------------------------------------------------------------- > > >Edward K. Ream email: edreamleo at charter.net > > >Leo: http://webpages.charter.net/edreamleo/front.html > > >-------------------------------------------------------------------- > > > > > > > > > > > >_______________________________________________ > > >pypy-dev at codespeak.net > > >http://codespeak.net/mailman/listinfo/pypy-dev > > > > > > _______________________________________________ > > pypy-dev at codespeak.net > > http://codespeak.net/mailman/listinfo/pypy-dev > > -- > merlinux GmbH Steinbergstr. 42 31139 Hildesheim > http://merlinux.de tel +49 5121 20800 75 (fax 77) > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061129/33afe042/attachment-0001.htm From arigo at tunes.org Wed Nov 29 12:15:52 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 29 Nov 2006 12:15:52 +0100 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> <20061129100306.GO11948@solar.trillke.net> <74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> Message-ID: <20061129111552.GA30301@code0.codespeak.net> Hi Alexander, On Wed, Nov 29, 2006 at 01:28:25PM +0300, Alexander Shafranov wrote: > With 2.5 on XP box: There are many details that are known not to work with Python 2.5. We fixed some since the 0.9 release, but there might be some left. You should use Python 2.4, or try your luck with PyPy's HEAD revision. A bientot, Armin From edreamleo at charter.net Wed Nov 29 16:35:32 2006 From: edreamleo at charter.net (Edward K. Ream) Date: Wed, 29 Nov 2006 09:35:32 -0600 Subject: [pypy-dev] Crash starting pypy on XP References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE><74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com><20061129100306.GO11948@solar.trillke.net><74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> <20061129111552.GA30301@code0.codespeak.net> Message-ID: <001f01c713cc$02178360$6400a8c0@EDWARDOFFICE> > You should use Python 2.4 Yes, that works. BTW, I think the getting started page is about the clearest, best such page I have ever seen. I look forward to playing with pypy and to meeting you all at the next sprint, whenever/wherever that will be. pypy looks like the most interesting computer language project in the world today. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at charter.net Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From paul.degrandis at gmail.com Wed Nov 29 16:42:18 2006 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Wed, 29 Nov 2006 10:42:18 -0500 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <001f01c713cc$02178360$6400a8c0@EDWARDOFFICE> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> <20061129100306.GO11948@solar.trillke.net> <74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> <20061129111552.GA30301@code0.codespeak.net> <001f01c713cc$02178360$6400a8c0@EDWARDOFFICE> Message-ID: <9c0bb8a00611290742v1b315858y8fa1bb63604c74c4@mail.gmail.com> > pypy looks like the most interesting computer language project in the world today. I also agree. Paul On 11/29/06, Edward K. Ream wrote: > > > You should use Python 2.4 > > Yes, that works. BTW, I think the getting started page is about the > clearest, best such page I have ever seen. > > I look forward to playing with pypy and to meeting you all at the next > sprint, whenever/wherever that will be. pypy looks like the most > interesting computer language project in the world today. > > Edward > -------------------------------------------------------------------- > Edward K. Ream email: edreamleo at charter.net > Leo: http://webpages.charter.net/edreamleo/front.html > -------------------------------------------------------------------- > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061129/1cf85c01/attachment.htm From santagada at gmail.com Wed Nov 29 17:39:53 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 29 Nov 2006 14:39:53 -0200 Subject: [pypy-dev] Crash starting pypy on XP In-Reply-To: <001f01c713cc$02178360$6400a8c0@EDWARDOFFICE> References: <001501c71359$f9671d10$6400a8c0@EDWARDOFFICE> <74fc943d0611290159k11a9690cgdc6fcc714e099d7c@mail.gmail.com> <20061129100306.GO11948@solar.trillke.net> <74fc943d0611290228y1baaa188ye274a4af0233bb77@mail.gmail.com> <20061129111552.GA30301@code0.codespeak.net> <001f01c713cc$02178360$6400a8c0@EDWARDOFFICE> Message-ID: <2f2e5f950611290839i626bb438h13cd2e08a089fbf0@mail.gmail.com> On 11/29/06, Edward K. Ream wrote: > I look forward to playing with pypy and to meeting you all at the next > sprint, whenever/wherever that will be. pypy looks like the most > interesting computer language project in the world today. You wrote Leo? I would really like if you could teach it to me on the switzerland sprint (that is, if I went there) :-). One more great mind on the project, welcome aboard. -- Leonardo Santagada (http://www.lomohomes.com/retype) N?o se preocupe com o que os outros v?o fazer. O melhor jeito de prever o futuro ? inventa-lo. - Alan Kay From arigo at tunes.org Wed Nov 29 22:37:48 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 29 Nov 2006 22:37:48 +0100 Subject: [pypy-dev] A new idea? In-Reply-To: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> References: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> Message-ID: <20061129213747.GA26408@code0.codespeak.net> Hi Paul, On Sun, Nov 05, 2006 at 12:46:29PM -0500, Paul deGrandis wrote: > idea is simple, I'll write small but useful applications that really > illustrate the powers of pypy. Sorry for the delay! Yes, this idea is attractive. Do you have particular projects in mind? Areas of interest? You are welcome to drop by in #pypy (in ~ european hours :-) and discuss this - unless of course you did so already and we missed the connexion between you and your nick :-) A bientot, Armin From paul.degrandis at gmail.com Thu Nov 30 00:03:24 2006 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Wed, 29 Nov 2006 18:03:24 -0500 Subject: [pypy-dev] A new idea? In-Reply-To: <20061129213747.GA26408@code0.codespeak.net> References: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> <20061129213747.GA26408@code0.codespeak.net> Message-ID: <9c0bb8a00611291503i59b2e570o2d8d828d12d31e05@mail.gmail.com> Hi Armin, Thank you for the input. I'm working on some interesting research projects but I have this project cooking to. I agree I think it is very attractive. I'm interested in general interesting applications of emerging technology like Pypy. Pypy has so many amazing features and aspects that there are a pool of interesting applications to create. Alternatively, there are also boring applications that when done with PyPy, offer an elegant solution. Other areas of interest are autonomic computing, situation aware computing (like application aware firewalls and situation aware protocols). I'm also currently involved in a Software Forensics research project with Drexel University. Paul On 11/29/06, Armin Rigo wrote: > > Hi Paul, > > On Sun, Nov 05, 2006 at 12:46:29PM -0500, Paul deGrandis wrote: > > idea is simple, I'll write small but useful applications that really > > illustrate the powers of pypy. > > Sorry for the delay! Yes, this idea is attractive. Do you have > particular projects in mind? Areas of interest? > > You are welcome to drop by in #pypy (in ~ european hours :-) and discuss > this - unless of course you did so already and we missed the connexion > between you and your nick :-) > > > A bientot, > > Armin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061129/fa14e6ae/attachment.htm From edreamleo at charter.net Thu Nov 30 03:50:44 2006 From: edreamleo at charter.net (Edward K. Ream) Date: Wed, 29 Nov 2006 20:50:44 -0600 Subject: [pypy-dev] Some new first thoughts about pypy Message-ID: <000501c7142a$54c66aa0$6400a8c0@EDWARDOFFICE> The pypy tools promise to move programming (not just python programming) in new directions. They may show a way to realize an ancient dream of mine, viz., to eliminate the gc for some (most?, all?) objects. For details, see: http://webpages.charter.net/edreamleo/whitepapers.html#allocating-storage-using-lifetimes Of course, nothing at all may come of this; it's just something to keep in the back of my mind. Your (gentle) comments are welcome. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at charter.net Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- From alexandre.fayolle at logilab.fr Thu Nov 30 10:56:38 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 30 Nov 2006 10:56:38 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 Message-ID: <20061130095638.GC24889@crater.logilab.fr> Hi, While running translations with various options on an AMD64, I encountered issues with the llvm backend: the translation fails when llvm-as is called. The error message is:: [translation:ERROR] llvm-as: :586398: Redefinition of function 'LL_os_fstat'! In entry_point.ll, I have a forward declaration of LL_os_fstat:: declare fastcc %structtype_tuple10* %LL_os_fstat(long) and indeed 2 definitions of the function. The first one is at line 449952, and it has a structure very similar to the definition I get when I translate in 32bit mode (but takes a long instead of an int as argument). The second one looks like:: internal fastcc %structtype_tuple10* %LL_os_fstat(long %tmp_13431) { %tmp_13433 = cast long %tmp_13431 to int %tmp_13432 = call fastcc %structtype_tuple10* %LL_os_fstat(int %tmp_13433) ret %structtype_tuple10* %tmp_13432 } If I comment out this second definition, I get a similar error regarding LL_os_lseek, LL_os_fork, etc. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061130/8a9a9c76/attachment.pgp From eric at vanrietpaap.nl Thu Nov 30 12:42:48 2006 From: eric at vanrietpaap.nl (Eric van Riet Paap) Date: Thu, 30 Nov 2006 12:42:48 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <20061130095638.GC24889@crater.logilab.fr> References: <20061130095638.GC24889@crater.logilab.fr> Message-ID: <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> Hi Alexandre, Thank you for trying this out! Do the llvm tests pass? pypy-llvm has not been properly tested on 64 bit systems! Some efforts were made but compatibilty was never maintained because of a lack of such a machine. If you could give me an account on your computer I could make sure that at least proper checks are in place. cheers, Eric On Nov 30, 2006, at 10:56 AM, Alexandre Fayolle wrote: > Hi, > > While running translations with various options on an AMD64, I > encountered issues with the llvm backend: the translation fails when > llvm-as is called. The error message is:: > > [translation:ERROR] llvm-as: :586398: Redefinition of > function 'LL_os_fstat'! > > In entry_point.ll, I have a forward declaration of LL_os_fstat:: > > declare fastcc %structtype_tuple10* %LL_os_fstat(long) > > and indeed 2 definitions of the function. The first one is at line > 449952, and it has a structure very similar to the definition I get > when > I translate in 32bit mode (but takes a long instead of an int as > argument). The second one looks like:: > > internal fastcc %structtype_tuple10* %LL_os_fstat(long %tmp_13431) { > %tmp_13433 = cast long %tmp_13431 to int > %tmp_13432 = call fastcc %structtype_tuple10* %LL_os_fstat > (int %tmp_13433) > ret %structtype_tuple10* %tmp_13432 > } > > > If I comment out this second definition, I get a similar error > regarding > LL_os_lseek, LL_os_fork, etc. > > -- > Alexandre Fayolle LOGILAB, Paris (France) > Formations Python, Zope, Plone, Debian: http://www.logilab.fr/ > formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/ > services > Informatique scientifique: http://www.logilab.fr/science > Reprise et maintenance de sites CPS: http://www.migration-cms.com/ > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From alexandre.fayolle at logilab.fr Thu Nov 30 13:29:04 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 30 Nov 2006 13:29:04 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> References: <20061130095638.GC24889@crater.logilab.fr> <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> Message-ID: <20061130122904.GE24889@crater.logilab.fr> On Thu, Nov 30, 2006 at 12:42:48PM +0100, Eric van Riet Paap wrote: > Hi Alexandre, > > Thank you for trying this out! > > Do the llvm tests pass? Unfortunately, no. See http://codespeak.net/~afayolle/summary.html for details. The installed llvm version is 1.8b. llvm-gcc --version says llvm-gcc (GCC) 3.4-llvm 20051104 (LLVM 1.7cvs) > pypy-llvm has not been properly tested on 64 bit systems! Some > efforts were made but compatibilty was never maintained because of a > lack of such a machine. > If you could give me an account on your computer I could make sure > that at least proper checks are in place. I am afraid this is going to be difficult. The machine is inside logilab's LAN, and only available through a VPN. However I'll be happy to help solving the issue (trying to patch the code if pointed in some initial direction, or applying patches and reporting back) -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061130/a6ffd059/attachment-0001.pgp From santagada at gmail.com Thu Nov 30 13:36:41 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 30 Nov 2006 10:36:41 -0200 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <20061130122904.GE24889@crater.logilab.fr> References: <20061130095638.GC24889@crater.logilab.fr> <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> <20061130122904.GE24889@crater.logilab.fr> Message-ID: <2f2e5f950611300436r764da43bq4f39a4d7046d6d23@mail.gmail.com> At least If I remember correctly what samuele told me is that if you really want to use llvm you should be using the cvs trunk version, this might fix some bugs for you (or open new ones :-)) On 11/30/06, Alexandre Fayolle wrote: > On Thu, Nov 30, 2006 at 12:42:48PM +0100, Eric van Riet Paap wrote: > > Hi Alexandre, > > > > Thank you for trying this out! > > > > Do the llvm tests pass? > > Unfortunately, no. See http://codespeak.net/~afayolle/summary.html for > details. > > The installed llvm version is 1.8b. llvm-gcc --version says > llvm-gcc (GCC) 3.4-llvm 20051104 (LLVM 1.7cvs) > > > pypy-llvm has not been properly tested on 64 bit systems! Some > > efforts were made but compatibilty was never maintained because of a > > lack of such a machine. > > If you could give me an account on your computer I could make sure > > that at least proper checks are in place. > > I am afraid this is going to be difficult. The machine is inside > logilab's LAN, and only available through a VPN. However I'll be happy > to help solving the issue (trying to patch the code if pointed in some > initial direction, or applying patches and reporting back) > > > -- > Alexandre Fayolle LOGILAB, Paris (France) > Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations > D?veloppement logiciel sur mesure: http://www.logilab.fr/services > Informatique scientifique: http://www.logilab.fr/science > Reprise et maintenance de sites CPS: http://www.migration-cms.com/ > > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.5 (GNU/Linux) > > iQEVAwUBRW7OkF6T+PKoJ87eAQL3zQgAhwXKdu4WAl6MK9LMTa7ncEdYigydQ0Rg > IwcA9mSMy7SelcWmKJeRJ0o4TbpdXO2HQ+FdcAHBDV1Lc9VGJYMq+rySjfmYJ9dh > b0YOva7xqqYo8lk68S3OGyCZpmFXCHzcfSv+kLujXkCgCblb6lLmyal5eSG6qXaf > vqfRojb/qZgtrltZpiQ839JMDHJGSJ67SBNwQY89P3xIXsszL7j1ec5eVg5iw3hS > 2m/t5I5rTMk72wOF39XMDIXAdHnIN+BZpY9ykxPRn9Y2Wzgn9S0/Uw5WghkXOfDJ > k6wHJ8+MTMQKzJovhpzIIIyH3My63JdqSBDqsYt3n6sQI5EsvK/5aA== > =/ZNr > -----END PGP SIGNATURE----- > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > > -- Leonardo Santagada (http://www.lomohomes.com/retype) N?o se preocupe com o que os outros v?o fazer. O melhor jeito de prever o futuro ? inventa-lo. - Alan Kay From alexandre.fayolle at logilab.fr Thu Nov 30 13:54:01 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Thu, 30 Nov 2006 13:54:01 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <2f2e5f950611300436r764da43bq4f39a4d7046d6d23@mail.gmail.com> References: <20061130095638.GC24889@crater.logilab.fr> <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> <20061130122904.GE24889@crater.logilab.fr> <2f2e5f950611300436r764da43bq4f39a4d7046d6d23@mail.gmail.com> Message-ID: <20061130125401.GF24889@crater.logilab.fr> (please, don't add me to the recipients when you reply on-list. I'm subscribed. Thanks) On Thu, Nov 30, 2006 at 10:36:41AM -0200, Leonardo Santagada wrote: > At least If I remember correctly what samuele told me is that if you > really want to use llvm you should be using the cvs trunk version, > this might fix some bugs for you (or open new ones :-)) Well, the Debian package uses a cvs snapshot from 2006-10-21, and apparently, llvm does not support AMD64. The support is "hacked back in" by the package maintainer, so the problem may well be in the toolchain. (and maybe it's not that usefull trying to get that backend to work on AMD64 until llvm supports the arch officially) -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061130/625e41f4/attachment.pgp From jacob at strakt.com Thu Nov 30 15:38:48 2006 From: jacob at strakt.com (Jacob =?iso-8859-1?q?Hall=E9n?=) Date: Thu, 30 Nov 2006 15:38:48 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <20061130122904.GE24889@crater.logilab.fr> References: <20061130095638.GC24889@crater.logilab.fr> <8D17E8A2-A0A4-471B-B911-60145C89E5DB@vanrietpaap.nl> <20061130122904.GE24889@crater.logilab.fr> Message-ID: <200611301538.48988.jacob@strakt.com> On Thursday 30 November 2006 13:29, Alexandre Fayolle wrote: > On Thu, Nov 30, 2006 at 12:42:48PM +0100, Eric van Riet Paap wrote: > > Hi Alexandre, > > > > Thank you for trying this out! > > > > Do the llvm tests pass? > > Unfortunately, no. See http://codespeak.net/~afayolle/summary.html for > details. > > The installed llvm version is 1.8b. llvm-gcc --version says > llvm-gcc (GCC) 3.4-llvm 20051104 (LLVM 1.7cvs) > > > pypy-llvm has not been properly tested on 64 bit systems! Some > > efforts were made but compatibilty was never maintained because of a > > lack of such a machine. > > If you could give me an account on your computer I could make sure > > that at least proper checks are in place. > > I am afraid this is going to be difficult. The machine is inside > logilab's LAN, and only available through a VPN. However I'll be happy > to help solving the issue (trying to patch the code if pointed in some > initial direction, or applying patches and reporting back) I have an amd64 machine running a pure 64 bit debian system (etch). I can make an account available. Jacob From arigo at tunes.org Thu Nov 30 18:58:10 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 30 Nov 2006 18:58:10 +0100 Subject: [pypy-dev] llvm backend issue on AMD64 In-Reply-To: <20061130095638.GC24889@crater.logilab.fr> References: <20061130095638.GC24889@crater.logilab.fr> Message-ID: <20061130175810.GA13616@code0.codespeak.net> Hi Alex, On Thu, Nov 30, 2006 at 10:56:38AM +0100, Alexandre Fayolle wrote: > While running translations with various options on an AMD64 We don't actively support 64-bit machines right now. There are failing tests. They reflect known problems. They are things to fix first before we should try to debug the resulting inconsistencies. No deep problem, but still... (We decided to focus on 32-bit machines for simple lack of time.) A bientot, Armin From mwh at python.net Thu Nov 30 20:15:10 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 30 Nov 2006 20:15:10 +0100 Subject: [pypy-dev] Some new first thoughts about pypy References: <000501c7142a$54c66aa0$6400a8c0@EDWARDOFFICE> Message-ID: <87slg0hhkx.fsf@starship.python.net> "Edward K. Ream" writes: > The pypy tools promise to move programming (not just python > programming) in new directions. They may show a way to realize an > ancient dream of mine, viz., to eliminate the gc for some (most?, > all?) objects. For details, see: > > http://webpages.charter.net/edreamleo/whitepapers.html#allocating-storage-using-lifetimes Um, I think this technique is used reasonably often in the implementation of compilers, is it not? Even Python 2.5 uses it... In the context of PyPy it's easy to think of a program where the use of this technique is less clear: an interpreter. > Of course, nothing at all may come of this; it's just something to > keep in the back of my mind. Your (gentle) comments are welcome. It is certainly the case that memory management is of utmost important to the performance of pypy (looking at kcachegrind for, oo, half a second will tell you that) but it is less clear how to do anything about that. PyPy already performs an analysis or two designed to remove allocations, and a few more have been written but are not used because they didn't help or are buggy. They are all local to a function so far though, I think. Googling around I found this paper: http://www.cs.cmu.edu/afs/cs/academic/class/15745-s06/web/handouts/lattner-pldi05.pdf -- would someone like to read it for me and tell me if it applies to PyPy? :-) The logical extension of the pool allocation idea is a generational garbage collector, and I'd dearly like to have one of those for PyPy but practically speaking you need a copying collector for one to be worth it and we don't support that yet for grotty reasons. Cheers, mwh -- Ability to type on a computer terminal is no guarantee of sanity, intelligence, or common sense. -- Gene Spafford's Axiom #2 of Usenet From arigo at tunes.org Fri Dec 1 15:52:37 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 1 Dec 2006 15:52:37 +0100 Subject: [pypy-dev] A new idea? In-Reply-To: <9c0bb8a00611291503i59b2e570o2d8d828d12d31e05@mail.gmail.com> References: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> <20061129213747.GA26408@code0.codespeak.net> <9c0bb8a00611291503i59b2e570o2d8d828d12d31e05@mail.gmail.com> Message-ID: <20061201145237.GA7669@code0.codespeak.net> Hi Paul, On Wed, Nov 29, 2006 at 06:03:24PM -0500, Paul deGrandis wrote: > I'm interested in general interesting applications of emerging technology > like Pypy. > Pypy has so many amazing features and aspects that there are a pool of > interesting applications to create. Alternatively, there are also boring > applications > that when done with PyPy, offer an elegant solution. RuntimeError: infinite recursion :-) Sorry for the bad joke - I asked if you had some examples in mind about applications that would benefit from PyPy's features, and the answer is quite self-recursive. Thanks for the background, though :-) Do you have anything more precise in mind about which PyPy features you'd like to see used in which applications? A bientot, Armin From paul.degrandis at gmail.com Fri Dec 1 21:17:00 2006 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Fri, 1 Dec 2006 15:17:00 -0500 Subject: [pypy-dev] A new idea? In-Reply-To: <20061201145237.GA7669@code0.codespeak.net> References: <9c0bb8a00611050946o713a970di4ef8b1165d78255d@mail.gmail.com> <20061129213747.GA26408@code0.codespeak.net> <9c0bb8a00611291503i59b2e570o2d8d828d12d31e05@mail.gmail.com> <20061201145237.GA7669@code0.codespeak.net> Message-ID: <9c0bb8a00612011217o2d4aec8bs76f887a04320670e@mail.gmail.com> Hi Armin, On 12/1/06, Armin Rigo wrote: > > > RuntimeError: infinite recursion :-) haha. I'm so sorry. I really wanted to leave it up to the core PyPy developer's for their input. I also wanted to generate "tutorials" for the applications I developed. This way we could build up a wealth of documentation that showed practical examples of how to apply the techonology in PyPy. I think compiler, JIT, proxies, lazily computed objects, stackless features all have interesting applications. I thought the original idea submitted was a good one and will work towards that unless someone has a better idea. Also, I see you're involved in TUNES. I love that project, a lot. I also really liked parts of the Slate language. Paul Sorry for the bad joke - I asked if you had some examples in mind about > applications that would benefit from PyPy's features, and the answer is > quite self-recursive. Thanks for the background, though :-) > > Do you have anything more precise in mind about which PyPy features you'd > like to see used in which applications? > > > A bientot, > > Armin > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061201/3d907496/attachment.htm From arigo at tunes.org Mon Dec 4 12:05:08 2006 From: arigo at tunes.org (Armin Rigo) Date: Mon, 4 Dec 2006 12:05:08 +0100 Subject: [pypy-dev] test_logicobjspace Message-ID: <20061204110507.GA30618@code0.codespeak.net> Hi, There are these 3 failures in test_logicobjspace that have been around for, possibly, a few months now... Anybody feeling like something needs to be done about it? AppTest_LogicFutures.test_lazy_producer_consummer AppTest_LogicFutures.test_stacklet AppTest_LogicFutures.test_wait_two Thanks! Armin From aurelien.campeas at logilab.fr Mon Dec 4 12:48:12 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Mon, 4 Dec 2006 12:48:12 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <20061204110507.GA30618@code0.codespeak.net> References: <20061204110507.GA30618@code0.codespeak.net> Message-ID: <20061204114812.GB7093@crater.logilab.fr> On Mon, Dec 04, 2006 at 12:05:08PM +0100, Armin Rigo wrote: > Hi, > > There are these 3 failures in test_logicobjspace that have been around > for, possibly, a few months now... Anybody feeling like something needs > to be done about it? > > AppTest_LogicFutures.test_lazy_producer_consummer > AppTest_LogicFutures.test_stacklet > AppTest_LogicFutures.test_wait_two > > > Thanks! > > Armin Last time I had a look, they worked perfectly fine. I believe Anders also checked positively that they worked on his machine. We both tried on a fresh check-out. They pass. Locally. So something might be happening on snake. I don't know. This is clearly out of scope for me right now. Feel free to do something about it. Well, I don't think simply disabling them would be satisfying. Aur?lien. From cfbolz at gmx.de Mon Dec 4 12:52:52 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 04 Dec 2006 12:52:52 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <20061204114812.GB7093@crater.logilab.fr> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> Message-ID: <45740C14.2050606@gmx.de> Aur?lien Camp?as wrote: > On Mon, Dec 04, 2006 at 12:05:08PM +0100, Armin Rigo wrote: >> Hi, >> >> There are these 3 failures in test_logicobjspace that have been around >> for, possibly, a few months now... Anybody feeling like something needs >> to be done about it? >> >> AppTest_LogicFutures.test_lazy_producer_consummer >> AppTest_LogicFutures.test_stacklet >> AppTest_LogicFutures.test_wait_two >> >> >> Thanks! >> >> Armin > > Last time I had a look, they worked perfectly fine. I believe Anders > also checked positively that they worked on his machine. We both tried > on a fresh check-out. They pass. Locally. > > So something might be happening on snake. I don't know. This is > clearly out of scope for me right now. Feel free to do something about > it. Well, I don't think simply disabling them would be satisfying. They fail quite consistently for me too, on all the various machines I am developing on. I would actually prefer it if they are fixed, since you never know if something you changed caused the problem. If nobody can work on this currently, they should be skipped and fixed later. Cheers, Carl Friedrich From aurelien.campeas at logilab.fr Mon Dec 4 13:00:37 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Mon, 4 Dec 2006 13:00:37 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <45740C14.2050606@gmx.de> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> <45740C14.2050606@gmx.de> Message-ID: <20061204120034.GC7093@crater.logilab.fr> On Mon, Dec 04, 2006 at 12:52:52PM +0100, Carl Friedrich Bolz wrote: > Aur?lien Camp?as wrote: > > On Mon, Dec 04, 2006 at 12:05:08PM +0100, Armin Rigo wrote: > >> Hi, > >> > >> There are these 3 failures in test_logicobjspace that have been around > >> for, possibly, a few months now... Anybody feeling like something needs > >> to be done about it? > >> > >> AppTest_LogicFutures.test_lazy_producer_consummer > >> AppTest_LogicFutures.test_stacklet > >> AppTest_LogicFutures.test_wait_two > >> > >> > >> Thanks! > >> > >> Armin > > > > Last time I had a look, they worked perfectly fine. I believe Anders > > also checked positively that they worked on his machine. We both tried > > on a fresh check-out. They pass. Locally. > > > > So something might be happening on snake. I don't know. This is > > clearly out of scope for me right now. Feel free to do something about > > it. Well, I don't think simply disabling them would be satisfying. > > They fail quite consistently for me too, on all the various machines I > am developing on. I would actually prefer it if they are fixed, since > you never know if something you changed caused the problem. If > nobody I tested them on a freshly checked-out pypy. > can work on this currently, they should be skipped and fixed later. > How do you invoke the tests ? I do, for instance (working dir being objspace/test) : ~/devel/pypy-dist/py/bin/py.test test_logicobjspace.py -s -k test_stacklet Does that trigger a failure on the machines you develop on ? > Cheers, > > Carl Friedrich > From arigo at tunes.org Mon Dec 4 13:02:54 2006 From: arigo at tunes.org (Armin Rigo) Date: Mon, 4 Dec 2006 13:02:54 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <20061204114812.GB7093@crater.logilab.fr> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> Message-ID: <20061204120254.GA4310@code0.codespeak.net> Hi Aurelien, On Mon, Dec 04, 2006 at 12:48:12PM +0100, Aur?lien Camp?as wrote: > So something might be happening on snake. I don't know. I've seen the same 3 failures on various machines, though admittedly not on tuatara, our PowerPC server. It fails on pypy2, for example, so you can investigate over there if needed (ssh codespeak.net; ssh pypy2). A bientot, Armin From aurelien.campeas at logilab.fr Mon Dec 4 13:08:36 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Mon, 4 Dec 2006 13:08:36 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <20061204120254.GA4310@code0.codespeak.net> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> <20061204120254.GA4310@code0.codespeak.net> Message-ID: <20061204120836.GD7093@crater.logilab.fr> On Mon, Dec 04, 2006 at 01:02:54PM +0100, Armin Rigo wrote: > Hi Aurelien, > > On Mon, Dec 04, 2006 at 12:48:12PM +0100, Aur?lien Camp?as wrote: > > So something might be happening on snake. I don't know. > > I've seen the same 3 failures on various machines, though admittedly not > on tuatara, our PowerPC server. It fails on pypy2, for example, so you > can investigate over there if needed (ssh codespeak.net; ssh pypy2). they fail indeed i'll have a look after lunch > > > A bientot, > > Armin > From scott+pypy-dev at scottdial.com Mon Dec 4 15:01:56 2006 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Mon, 04 Dec 2006 09:01:56 -0500 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <20061204120254.GA4310@code0.codespeak.net> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> <20061204120254.GA4310@code0.codespeak.net> Message-ID: <45742A54.7020001@scottdial.com> Armin Rigo wrote: > On Mon, Dec 04, 2006 at 12:48:12PM +0100, Aur?lien Camp?as wrote: >> So something might be happening on snake. I don't know. > > I've seen the same 3 failures on various machines, though admittedly not > on tuatara, our PowerPC server. It fails on pypy2, for example, so you > can investigate over there if needed (ssh codespeak.net; ssh pypy2). > Also for consideration is that the test has consistently passed on my win32 tester. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From aurelien.campeas at logilab.fr Mon Dec 4 15:05:30 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Mon, 4 Dec 2006 15:05:30 +0100 Subject: [pypy-dev] test_logicobjspace In-Reply-To: <45742A54.7020001@scottdial.com> References: <20061204110507.GA30618@code0.codespeak.net> <20061204114812.GB7093@crater.logilab.fr> <20061204120254.GA4310@code0.codespeak.net> <45742A54.7020001@scottdial.com> Message-ID: <20061204140530.GE7093@crater.logilab.fr> On Mon, Dec 04, 2006 at 09:01:56AM -0500, Scott Dial wrote: > Armin Rigo wrote: > >On Mon, Dec 04, 2006 at 12:48:12PM +0100, Aur?lien Camp?as wrote: > >>So something might be happening on snake. I don't know. > > > >I've seen the same 3 failures on various machines, though admittedly not > >on tuatara, our PowerPC server. It fails on pypy2, for example, so you > >can investigate over there if needed (ssh codespeak.net; ssh pypy2). > > > > Also for consideration is that the test has consistently passed on my > win32 tester. It was a dict ordering issue: the test depended on somedict.values()[0] always giving the right thing ... > > -Scott > > -- > Scott Dial > scott at scottdial.com > scodial at cs.indiana.edu > From arigo at tunes.org Tue Dec 5 18:10:05 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 5 Dec 2006 18:10:05 +0100 Subject: [pypy-dev] Leysin sprint: 8-14th January 2007 Message-ID: <20061205171003.GA10831@code0.codespeak.net> ===================================================================== PyPy Leysin Winter Sports Sprint (8-14th January 2007) ===================================================================== .. image:: http://www.ermina.ch/002.JPG The next PyPy sprint will be in Leysin, Switzerland, for the fourth time. This sprint will be the final public sprint of our EU-funded period, and a kick-off for the final work on the upcoming PyPy 1.0 release (scheduled for mid-February). The sprint is the last chance for students looking for a "summer" job with PyPy this winter! If you have a proposal and would like to work with us in the mountains please send it in before 15th December to "pypy-tb at codespeak.net" and cross-read this page: http://codespeak.net/pypy/dist/pypy/doc/summer-of-pypy.html ------------------------------ Goals and topics of the sprint ------------------------------ * Like previous winter, the main side goal is to have fun in winter sports :-) We can take a couple of days off for ski; at this time of year, ski days end before 4pm, which still leaves plenty of time to recover (er, I mean hack). * Progress on the JIT compiler, which we are just starting to scale to the whole of PyPy. `[1]`_ * Polish the code and documentation of the py lib, and eventually release it. * Work on prototypes that use the new features that PyPy enables: distribution `[2]`_ (based on transparent proxying `[3]`_), security `[4]`_, persistence `[5]`_, etc. `[6]`_. .. _[1]: http://codespeak.net/pypy/dist/pypy/doc/jit.html .. _[2]: http://codespeak.net/svn/pypy/dist/pypy/lib/distributed/ .. _[3]: http://codespeak.net/pypy/dist/pypy/doc/proxy.html .. _[4]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#security .. _[5]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html#persistence .. _[6]: http://codespeak.net/pypy/dist/pypy/doc/project-ideas.html ----------------------- Location & Accomodation ----------------------- Leysin, Switzerland, "same place as before". Let me refresh your memory: both the sprint venue and the lodging will be in a very spacious pair of chalets built specifically for bed & breakfast: http://www.ermina.ch/. The place has a baseline ADSL Internet connexion (600Kb) with wireless installed. You can of course arrange your own lodging anywhere (so long as you are in Leysin, you cannot be more than a 15 minute walk away from the sprint venue), but I definitely recommend lodging there too -- you won't find a better view anywhere else (though you probably won't get much worse ones easily, either :-) I made pre-reservations in the Chalet, so please *confirm* quickly that you are coming so that we can adjust the reservations as appropriate. The rate so far has been 60 CHF a night all included in 2-person rooms, with breakfast. There are larger rooms too (less expensive, possibly more available too) and the possibility to get a single room if you really want to. Please register by svn: http://codespeak.net/svn/pypy/extradoc/sprintinfo/leysin-winter-2007/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint You need a Swiss-to-(insert country here) power adapter. There will be some Swiss-to-EU adapters around - bring a EU-format power strip if you have one. ----------- Exact times ----------- Officially, 8th-14th January 2007. Both dates are flexible, you can arrive or leave earlier or later, though it is recommended to arrive on the 7th (if many people arrive on the 6th we need to check for accomodation availability first). We will give introductions and tutorials on the 8th, in the morning if everybody is there, or in the afternoon otherwise. From stephan.diehl at gmx.net Tue Dec 5 19:12:34 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Tue, 05 Dec 2006 19:12:34 +0100 Subject: [pypy-dev] coroutine interface for CPython Message-ID: <4575B692.9090206@gmx.net> Hi all, I've just hacked together a coroutine interface for CPython. Basicly, I've stripped out all references to pypy from christians interp_coroutine.py module. Amazingly, even pypy's stackless module works with this coroutine implementation. At the moment, all of this can be found here: https://codespeak.net/viewvc/user/stephan/hacks/coroutine/ Should I put this somewhere into the pypy repository? And if yes, where exactly? Cheers Stephan From alexandre.fayolle at logilab.fr Tue Dec 5 20:29:03 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Tue, 5 Dec 2006 20:29:03 +0100 Subject: [pypy-dev] problem with build tool Message-ID: <20061205192903.GE20280@crater.logilab.fr> Hi, I will probably have no network access tomorrow, so I post this as a reminder, and also because the error takes time to reproduce. When finishing a build, the client process crashes with the following exception: [translation:info] created: ./pypy-c Traceback (most recent call last): File "bin/client", line 65, in ? zip.writestr(fpath.relto(udir), fpath.read()) File "/home/alf/dev/pypy/dist/py/path/common.py", line 312, in read f = self.open(mode) File "/home/alf/dev/pypy/dist/py/path/local/local.py", line 200, in open return self._callex(open, self.strpath, mode) File "/home/alf/dev/pypy/dist/py/path/common.py", line 229, in _callex raise cls, value py.error.ENOENT: [No such file or directory]: file('/tmp/usession-16/.lock', 'rb') Exception exceptions.AttributeError: "'ChannelWrapper' object has no attribute 'tell'" in > ignored The error is a bit mysterious to me, so if people in the assistance have an idea of what is happening but no time to fix it, their opinion is very welcome. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061205/04526b72/attachment.pgp From arigo at tunes.org Tue Dec 5 20:35:28 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 5 Dec 2006 20:35:28 +0100 Subject: [pypy-dev] problem with build tool In-Reply-To: <20061205192903.GE20280@crater.logilab.fr> References: <20061205192903.GE20280@crater.logilab.fr> Message-ID: <20061205193528.GA28030@code0.codespeak.net> Hi, On Tue, Dec 05, 2006 at 08:29:03PM +0100, Alexandre Fayolle wrote: > py.error.ENOENT: [No such file or directory]: > file('/tmp/usession-16/.lock', 'rb') This .lock file is a symlink whose content is the pid of the process, so uncareful places (as here) find it with listdir() but crash when opening it. Not sure why the build tool tries to zip it all, instead of just the 'testing_1' subdirectory. Not sure why it didn't crash before, either. A bientot, Armin From alexandre.fayolle at logilab.fr Wed Dec 6 15:42:04 2006 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Wed, 6 Dec 2006 15:42:04 +0100 Subject: [pypy-dev] problem with build tool In-Reply-To: <20061205193528.GA28030@code0.codespeak.net> References: <20061205192903.GE20280@crater.logilab.fr> <20061205193528.GA28030@code0.codespeak.net> Message-ID: <20061206144204.GA15969@crater.logilab.fr> On Tue, Dec 05, 2006 at 08:35:28PM +0100, Armin Rigo wrote: > Hi, > > On Tue, Dec 05, 2006 at 08:29:03PM +0100, Alexandre Fayolle wrote: > > py.error.ENOENT: [No such file or directory]: > > file('/tmp/usession-16/.lock', 'rb') > > This .lock file is a symlink whose content is the pid of the process, so > uncareful places (as here) find it with listdir() but crash when opening > it. > > Not sure why the build tool tries to zip it all, instead of just the > 'testing_1' subdirectory. Not sure why it didn't crash before, either. Well, before changes I checked in yesterday, it crashed before reaching this line, so this is why ;-) I fixed this locally in the train this morning, and will check in once I've been able to run a successful translation, which I cannot do reasonably while running on battery. -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20061206/f8b2ddd1/attachment.pgp From arigo at tunes.org Fri Dec 8 03:19:50 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 8 Dec 2006 03:19:50 +0100 Subject: [pypy-dev] The first JITing pypy-c! Message-ID: <20061208021946.GA1900@code0.codespeak.net> Hi all, Two days ago, we got our first JITing version of pypy-c! This was mostly thanks to Arre, who compiled a version even though we knew it would segfault due to missing support for C structures with non-int-sized fields. Writing to a 1-byte-sized Bool field would overwrite the next 3 bytes of memory with zeroes... Nevertheless, the result managed to successfully run one function (and indeed segfault on many other functions). Our first JIT-run function! A recursive factorial. Today the field size problem is fixed. Playing around seems to show that it's harder to provoke a segfault now. The generated machine code is completely incredible, in size and complexity, but the following example runs. It could even be said to run faster with the JIT (8.2 seconds versus 11.4 seconds) but that's unfair, as all normal optimizations are turned off in this example (a regular pypy-c runs this example in 2.8 seconds). It still shows that our JIT already gives an improvement over completely-unoptimized C code, which is already some kind of success! def f(n): while n > 0: n -= 2 return n Many other examples give an UnboundLocalError, due probably to some minor bug somewhere either in the JIT transformation or in the back-end (along the lines of a == compiled as a !=). If you want to try for yourself: - check out or switch to the branch http://codespeak.net/svn/pypy/branch/jit-real-world - run "translate.py /path/to/pypy/jit/goal/targetjit.py" (that's the usual translate.py from translator/goal) - you get uncompiled C sources for now; copy it safely away from /tmp/usession-yourname/testing_1/ before it gets deleted, and compile it ("make" or "make debug"). - run "PYPYJITLOG=log ./testing_1" import pypyjit; pypyjit.enable(f.func_code) f(7000000) # see f above - the above PYPYJITLOG env var causes a file called 'log' to be produced, containing the generated assembler code. It can be viewed in a flowgraph-like fashion with pypy/jit/codegen/i386/viewcode.py. Don't ask me yet to describe the result, nor where the while loop is :-) If you are familiar with i386 assembler, you'll laugh at the obviously bad code, too. That's where your help would be appreciated! Making the backend produce more reasonable code, starting with some basic register allocation, is a mostly-independent project. The PPC backend, btw, has already got this kind of techniques (I wonder what speed-ups we get on PPC from a jitting pypy-c). Have fun, Armin From niko at alum.mit.edu Fri Dec 8 07:49:08 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 08 Dec 2006 07:49:08 +0100 Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: <20061208021946.GA1900@code0.codespeak.net> References: <20061208021946.GA1900@code0.codespeak.net> Message-ID: <45790AE4.3030506@alum.mit.edu> > Two days ago, we got our first JITing version of pypy-c! > That's very exciting! Congratulations. Niko From mwh at python.net Fri Dec 8 11:23:26 2006 From: mwh at python.net (Michael Hudson) Date: Fri, 08 Dec 2006 11:23:26 +0100 Subject: [pypy-dev] The first JITing pypy-c! References: <20061208021946.GA1900@code0.codespeak.net> Message-ID: <878xhifzz5.fsf@starship.python.net> Armin Rigo writes: > Hi all, > > Two days ago, we got our first JITing version of pypy-c! > > This was mostly thanks to Arre, who compiled a version even though we > knew it would segfault due to missing support for C structures with > non-int-sized fields. Writing to a 1-byte-sized Bool field would > overwrite the next 3 bytes of memory with zeroes... Nevertheless, the > result managed to successfully run one function (and indeed segfault on > many other functions). Our first JIT-run function! A recursive > factorial. Hooray! > - the above PYPYJITLOG env var causes a file called 'log' to be > produced, containing the generated assembler code. It can be viewed > in a flowgraph-like fashion with pypy/jit/codegen/i386/viewcode.py. > Don't ask me yet to describe the result, nor where the while loop is > :-) If you are familiar with i386 assembler, you'll laugh at the > obviously bad code, too. That's where your help would be appreciated! > Making the backend produce more reasonable code, starting with some > basic register allocation, is a mostly-independent project. The PPC > backend, btw, has already got this kind of techniques (I wonder what > speed-ups we get on PPC from a jitting pypy-c). Well, I guess so far it just plain doesn't work, but also the PPC register allocation is pretty horrible, especially around function calls. We've talked about the backend change needed to make this sensible (giving the list of live values to the block-ending builder methods), I guess it just has to be done at some point... Cheers, mwh -- You can lead an idiot to knowledge but you cannot make him think. You can, however, rectally insert the information, printed on stone tablets, using a sharpened poker. -- Nicolai -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From rxe at ukshells.co.uk Fri Dec 8 18:20:29 2006 From: rxe at ukshells.co.uk (Richard Emslie) Date: Fri, 8 Dec 2006 17:20:29 +0000 (GMT) Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: <20061208021946.GA1900@code0.codespeak.net> References: <20061208021946.GA1900@code0.codespeak.net> Message-ID: Wow - that's awesome! Congratulations to all involved. Cheers, Richard On Fri, 8 Dec 2006, Armin Rigo wrote: > Hi all, > > Two days ago, we got our first JITing version of pypy-c! > > This was mostly thanks to Arre, who compiled a version even though we > knew it would segfault due to missing support for C structures with > non-int-sized fields. Writing to a 1-byte-sized Bool field would > overwrite the next 3 bytes of memory with zeroes... Nevertheless, the > result managed to successfully run one function (and indeed segfault on > many other functions). Our first JIT-run function! A recursive > factorial. > > Today the field size problem is fixed. Playing around seems to show > that it's harder to provoke a segfault now. The generated machine code > is completely incredible, in size and complexity, but the following > example runs. It could even be said to run faster with the JIT (8.2 > seconds versus 11.4 seconds) but that's unfair, as all normal > optimizations are turned off in this example (a regular pypy-c runs this > example in 2.8 seconds). It still shows that our JIT already gives an > improvement over completely-unoptimized C code, which is already some > kind of success! > > def f(n): > while n > 0: > n -= 2 > return n > > Many other examples give an UnboundLocalError, due probably to some > minor bug somewhere either in the JIT transformation or in the back-end > (along the lines of a == compiled as a !=). > > If you want to try for yourself: > > - check out or switch to the branch > http://codespeak.net/svn/pypy/branch/jit-real-world > > - run "translate.py /path/to/pypy/jit/goal/targetjit.py" > (that's the usual translate.py from translator/goal) > > - you get uncompiled C sources for now; copy it safely away > from /tmp/usession-yourname/testing_1/ before it gets deleted, > and compile it ("make" or "make debug"). > > - run "PYPYJITLOG=log ./testing_1" > > import pypyjit; pypyjit.enable(f.func_code) > f(7000000) # see f above > > - the above PYPYJITLOG env var causes a file called 'log' to be > produced, containing the generated assembler code. It can be viewed > in a flowgraph-like fashion with pypy/jit/codegen/i386/viewcode.py. > Don't ask me yet to describe the result, nor where the while loop is > :-) If you are familiar with i386 assembler, you'll laugh at the > obviously bad code, too. That's where your help would be appreciated! > Making the backend produce more reasonable code, starting with some > basic register allocation, is a mostly-independent project. The PPC > backend, btw, has already got this kind of techniques (I wonder what > speed-ups we get on PPC from a jitting pypy-c). > > > Have fun, > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From arigo at tunes.org Sat Dec 9 03:03:33 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 9 Dec 2006 03:03:33 +0100 Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: <20061208021946.GA1900@code0.codespeak.net> References: <20061208021946.GA1900@code0.codespeak.net> Message-ID: <20061209020332.GA14456@code0.codespeak.net> Re-hi all, On Fri, Dec 08, 2006 at 03:19:50AM +0100, Armin Rigo wrote: > Many other examples give an UnboundLocalError Fixed now. Jitting execution seems to work as well as the normal one. We tried various things, including stuff not supported by Psyco (generators, nested scopes...), with success. Now it's only a matter to make it produce useful code, as opposed to just slow, big and incredible code... > If you want to try for yourself: > > - check out or switch to the branch > http://codespeak.net/svn/pypy/branch/jit-real-world The branch is now merged into the trunk! The rest of the instructions still apply. (The merging was a bit of fun, as usual; if someone thinks that he might have lost any changes, don't hesitate to blame us before checking in detail :-) A bientot, Armin. From niko at alum.mit.edu Sat Dec 9 12:17:21 2006 From: niko at alum.mit.edu (Niko Matsakis) Date: Sat, 9 Dec 2006 12:17:21 +0100 Subject: [pypy-dev] Problem with subversion? Message-ID: <45ACBD2E-D300-4285-B8F7-6FD0BA334092@alum.mit.edu> When I attempt to do a "svn up", I get the following error: ;svn up svnserve: error while loading shared libraries: libaprutil-0.so.0: cannot open shared object file: No such file or directory svn: Connection closed unexpectedly ; I get the error on multiple machines, both linux and OS/X, and I am able to work with other remote repositories, so I don't *think* it's my machine's fault... Niko From hpk at trillke.net Sat Dec 9 15:36:27 2006 From: hpk at trillke.net (holger krekel) Date: Sat, 9 Dec 2006 15:36:27 +0100 Subject: [pypy-dev] Problem with subversion? In-Reply-To: <45ACBD2E-D300-4285-B8F7-6FD0BA334092@alum.mit.edu> References: <45ACBD2E-D300-4285-B8F7-6FD0BA334092@alum.mit.edu> Message-ID: <20061209143626.GH11948@solar.trillke.net> Hi Niko, On Sat, Dec 09, 2006 at 12:17 +0100, Niko Matsakis wrote: > When I attempt to do a "svn up", I get the following error: > > ;svn up > svnserve: error while loading shared libraries: libaprutil-0.so.0: > cannot open shared object file: No such file or directory > svn: Connection closed unexpectedly > ; > > I get the error on multiple machines, both linux and OS/X, and I am > able to work with other remote repositories, so I don't *think* it's > my machine's fault... is the problem still there? Today there was an update of the apache server on the codespeak.net machine because the old one caused memory problems and had to be restarted daily. If you still have problems, please talk to Martin who did the upgrade (CCed in this mail). best, holger From arigo at tunes.org Sun Dec 10 01:38:02 2006 From: arigo at tunes.org (Armin Rigo) Date: Sun, 10 Dec 2006 01:38:02 +0100 Subject: [pypy-dev] RuPy conference Message-ID: <20061210003802.GA17355@code0.codespeak.net> Hi all, Should we try to get a PyPy talk at the upcoming RuPy conference in Poznan (Poland)? http://rupy.wmid.amu.edu.pl/ With luck it could trigger the start of a translation frontend for the restricted variant of Ruby that one of the Ruby-in-Ruby projects is based on, if we can convince people that this way leads to a full Ruby JIT :-) A bientot, Armin From fijal at genesilico.pl Sun Dec 10 11:24:58 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 10 Dec 2006 11:24:58 +0100 Subject: [pypy-dev] RuPy conference In-Reply-To: <20061210003802.GA17355@code0.codespeak.net> References: <20061210003802.GA17355@code0.codespeak.net> Message-ID: <457BE07A.4000904@genesilico.pl> Armin Rigo wrote: >Hi all, > >Should we try to get a PyPy talk at the upcoming RuPy conference in >Poznan (Poland)? > > Yes, it's definitely Poland ;-) About 3h train ride from my place. > http://rupy.wmid.amu.edu.pl/ > >With luck it could trigger the start of a translation frontend for the >restricted variant of Ruby that one of the Ruby-in-Ruby projects is >based on, if we can convince people that this way leads to a full Ruby >JIT :-) > > >A bientot, > >Armin >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > The date is quite convinient (it's just after EU funding period, right?), I can take care about hotels there and such. Cheers, Fijal From faassen at startifact.com Mon Dec 11 20:40:08 2006 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 11 Dec 2006 20:40:08 +0100 Subject: [pypy-dev] nightly builds and benchmarks page down Message-ID: Hi there, I've noticed for a while now that the following page is down: http://snake.cs.uni-duesseldorf.de/pypytest/benchmark.html It's listed here: http://codespeak.net/pypy/dist/pypy/doc/index.html I thought I'd post a message here in case that this isn't deliberate. :) Regards, Martijn From elmo13 at jippii.fi Mon Dec 11 22:01:44 2006 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Mon, 11 Dec 2006 23:01:44 +0200 Subject: [pypy-dev] nightly builds and benchmarks page down In-Reply-To: References: Message-ID: <457DC738.4070906@jippii.fi> Martijn Faassen wrote: > Hi there, > > I've noticed for a while now that the following page is down: > > http://snake.cs.uni-duesseldorf.de/pypytest/benchmark.html > > It's listed here: > > http://codespeak.net/pypy/dist/pypy/doc/index.html > > I thought I'd post a message here in case that this isn't deliberate. :) > > Regards, > > Martijn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > That particular server ("snake") has been acting weird lately, and no solution at the moment. So, not deliberate. Luckily, autotesting has been set up on a nother server currently: http://www.strakt.com/~pedronis/autotest/summary.html Elmo From fijal at genesilico.pl Tue Dec 12 12:06:47 2006 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Tue, 12 Dec 2006 12:06:47 +0100 Subject: [pypy-dev] nightly builds and benchmarks page down In-Reply-To: <457DC738.4070906@jippii.fi> References: <457DC738.4070906@jippii.fi> Message-ID: <457E8D47.1070604@genesilico.pl> Elmo M?ntynen wrote: >Martijn Faassen wrote: > > >>Hi there, >> >>I've noticed for a while now that the following page is down: >> >>http://snake.cs.uni-duesseldorf.de/pypytest/benchmark.html >> >>It's listed here: >> >>http://codespeak.net/pypy/dist/pypy/doc/index.html >> >>I thought I'd post a message here in case that this isn't deliberate. :) >> >>Regards, >> >>Martijn >> >>_______________________________________________ >>pypy-dev at codespeak.net >>http://codespeak.net/mailman/listinfo/pypy-dev >> >> >> >That particular server ("snake") has been acting weird lately, and no >solution at the moment. So, not deliberate. Luckily, autotesting has >been set up on a nother server currently: >http://www.strakt.com/~pedronis/autotest/summary.html > >Elmo >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev > > And benchmarks are on tuatara. http://tuatara.cs.uni-duesseldorf.de/benchmark.html From mwh at python.net Tue Dec 12 12:38:36 2006 From: mwh at python.net (Michael Hudson) Date: Tue, 12 Dec 2006 12:38:36 +0100 Subject: [pypy-dev] nightly builds and benchmarks page down References: <457DC738.4070906@jippii.fi> Message-ID: <87irghe43n.fsf@starship.python.net> Elmo M?ntynen writes: > Martijn Faassen wrote: >> Hi there, >> >> I've noticed for a while now that the following page is down: >> >> http://snake.cs.uni-duesseldorf.de/pypytest/benchmark.html >> >> It's listed here: >> >> http://codespeak.net/pypy/dist/pypy/doc/index.html >> >> I thought I'd post a message here in case that this isn't deliberate. :) >> >> Regards, >> >> Martijn >> >> _______________________________________________ >> pypy-dev at codespeak.net >> http://codespeak.net/mailman/listinfo/pypy-dev >> > That particular server ("snake") has been acting weird lately, and no > solution at the moment. So, not deliberate. Luckily, autotesting has > been set up on a nother server currently: > http://www.strakt.com/~pedronis/autotest/summary.html But if it's a benchmark.html you're looking for, a smaller set of benchmarks has been being run on 'tuatara' (a PowerMac G5 in the same server room as snake): http://tuatara.cs.uni-duesseldorf.de/benchmark.html Cheers, mwh -- When physicists speak of a TOE, they don't really mean a theory of *everything*. Taken literally, "Everything" covers a lot of ground, including biology, art, decoherence and the best way to barbecue ribs. -- John Baez, sci.physics.research From len-l at telus.net Thu Dec 14 00:50:21 2006 From: len-l at telus.net (Lenard Lindstrom) Date: Wed, 13 Dec 2006 15:50:21 -0800 Subject: [pypy-dev] Confusion with meaning of "Stackless" Message-ID: <4580213D.9801.19B2FC9@len-l.telus.net> Tasklets are nice, but I just wish to update my understanding of "Stackless". Does the old motto "A Python Implementation That Does Not Use The C Stack" [1] still apply? Is it relevant to PyPy? [1] http://stackless.com/index_old.htm Lenard Lindstrom From rxe at ukshells.co.uk Thu Dec 14 02:22:15 2006 From: rxe at ukshells.co.uk (Richard Emslie) Date: Thu, 14 Dec 2006 01:22:15 +0000 (GMT) Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: <20061209020332.GA14456@code0.codespeak.net> References: <20061208021946.GA1900@code0.codespeak.net> <20061209020332.GA14456@code0.codespeak.net> Message-ID: Hi, On Sat, 9 Dec 2006, Armin Rigo wrote: > Re-hi all, > > On Fri, Dec 08, 2006 at 03:19:50AM +0100, Armin Rigo wrote: > > Fixed now. Jitting execution seems to work as well as the normal one. > We tried various things, including stuff not supported by Psyco > (generators, nested scopes...), with success. Very cool! But do recursive intepreters work now? :-) Cheers, Richard From arigo at tunes.org Thu Dec 14 04:22:05 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 14 Dec 2006 04:22:05 +0100 Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: References: <20061208021946.GA1900@code0.codespeak.net> <20061209020332.GA14456@code0.codespeak.net> Message-ID: <20061214032205.GA28797@code0.codespeak.net> Hi Richard, On Thu, Dec 14, 2006 at 01:22:15AM +0000, Richard Emslie wrote: > > Fixed now. Jitting execution seems to work as well as the normal one. > > We tried various things, including stuff not supported by Psyco > > (generators, nested scopes...), with success. > > Very cool! But do recursive intepreters work now? :-) Yes, everything works :-) Except the bytecode trace hook. The machine code is really terrible at all points of view, but you can JIT whatever piece of Python code you like - generators, nested scopes, class: statement bodies, all these cases where Psyco give up. That's the point of the approach, really :-) We have a minor detail to solve before it can be tested on larger examples, though - it exhausts the 32-bit address space far too early (without actually consuming much of the reserved pages) and then we get a MemoryError. It's a back-end problem; Arre started working on that today. A bientot, Armin. From arigo at tunes.org Thu Dec 14 07:48:58 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 14 Dec 2006 07:48:58 +0100 Subject: [pypy-dev] Confusion with meaning of "Stackless" In-Reply-To: <4580213D.9801.19B2FC9@len-l.telus.net> References: <4580213D.9801.19B2FC9@len-l.telus.net> Message-ID: <20061214064857.GA14933@code0.codespeak.net> Hi Lenard, On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote: > Tasklets are nice, but I just wish to update my understanding of > "Stackless". Does the old motto "A Python Implementation That Does > Not Use The C Stack" [1] still apply? Is it relevant to PyPy? Yes and no. A bit confusingly, the usage of the term has evolved, along with the Stackless Python project itself. Now, "a Stackless feature" means a feature that cannot be implemented without some form of explicit stack control, either by being very careful about the C code (e.g. avoiding recursive calls) or by saving and restoring pieces of the C stack by a brute force 'memcpy' approach. In PyPy, we use a variant of the former approach: a "stackless build" of pypy-c is a compiled version of some C code that was generated with careful systematic tweaks. The motto of a Stackless PyPy is probably: "A Python implementation that uses the C Stack as a cache". Indeed, the C code supports saving its own stack of frames away into the heap, and restoring it from there, frame by frame. So the C stack is really just a cache for the heap. If the cache is full (stack overflow), we save some more frames into the heap to free some cache space. If a context switch occurs (coroutine switch) the cache is invalidated (entierely flushed to the heap) and the new context's frames are copied from the heap to the cache as needed (the function's frames are resumed one by one). A bientot, Armin. From mwh at python.net Thu Dec 14 10:11:36 2006 From: mwh at python.net (Michael Hudson) Date: Thu, 14 Dec 2006 10:11:36 +0100 Subject: [pypy-dev] Confusion with meaning of "Stackless" References: <4580213D.9801.19B2FC9@len-l.telus.net> <20061214064857.GA14933@code0.codespeak.net> Message-ID: <873b7i6dvb.fsf@starship.python.net> Armin Rigo writes: > Hi Lenard, > > On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote: >> Tasklets are nice, but I just wish to update my understanding of >> "Stackless". Does the old motto "A Python Implementation That Does >> Not Use The C Stack" [1] still apply? Is it relevant to PyPy? > > Yes and no. A bit confusingly, the usage of the term has evolved, along > with the Stackless Python project itself. Now, "a Stackless feature" > means a feature that cannot be implemented without some form of explicit > stack control, either by being very careful about the C code (e.g. > avoiding recursive calls) or by saving and restoring pieces of the C > stack by a brute force 'memcpy' approach. In PyPy, we use a variant of > the former approach: a "stackless build" of pypy-c is a compiled version > of some C code that was generated with careful systematic tweaks. > > The motto of a Stackless PyPy is probably: "A Python implementation that > uses the C Stack as a cache". Indeed, the C code supports saving its > own stack of frames away into the heap, and restoring it from there, > frame by frame. So the C stack is really just a cache for the heap. If > the cache is full (stack overflow), we save some more frames into the > heap to free some cache space. If a context switch occurs (coroutine > switch) the cache is invalidated (entierely flushed to the heap) and the > new context's frames are copied from the heap to the cache as needed > (the function's frames are resumed one by one). Do you feel like adding this explanation to the D07.1 report? :-) Cheers, mwh -- A VoIP server "powered entirely by stabbing, that I made out of this gun I had" -- from Twisted.Quotes From rxe at ukshells.co.uk Thu Dec 14 19:02:34 2006 From: rxe at ukshells.co.uk (Richard Emslie) Date: Thu, 14 Dec 2006 18:02:34 +0000 (GMT) Subject: [pypy-dev] The first JITing pypy-c! In-Reply-To: <20061214032205.GA28797@code0.codespeak.net> References: <20061208021946.GA1900@code0.codespeak.net> <20061209020332.GA14456@code0.codespeak.net> <20061214032205.GA28797@code0.codespeak.net> Message-ID: Hi Armin! Wow! So at risk of coming across really stupid, I am going to hazard a guess at what is going on. :-) As timeshifting is a translation time thing, we can handle mutiple or recursive merge points as a one off atomic operation before we run. So now when we run we have two modes of execution - 'compile mode' and 'normal mode'. (i hope at least this part is right!) So now we are running our timeshifted translated code, where the original rpython code defined a recursive interpreter, which in turn defined a recursive function with a mergepoint. When we hit a flexiswitch (the merge point?), we go from normal mode to compile mode and may promote / partially evaluate run time variables for each new instance of a run time variable (the promotion hinted ones). So what if a 'promotion' in compile mode triggers another promotion down the line (ie during the timeshifted residualizing code) and we go into a infinite loop of promoting? Ok - so more questions than statements - and a high probabilty that it made no sense to all - and maybe I should just wait for the docs! :-) Cheers, Richard On Thu, 14 Dec 2006, Armin Rigo wrote: > Hi Richard, > > On Thu, Dec 14, 2006 at 01:22:15AM +0000, Richard Emslie wrote: >>> Fixed now. Jitting execution seems to work as well as the normal one. >>> We tried various things, including stuff not supported by Psyco >>> (generators, nested scopes...), with success. >> >> Very cool! But do recursive intepreters work now? :-) > > Yes, everything works :-) Except the bytecode trace hook. The machine > code is really terrible at all points of view, but you can JIT whatever > piece of Python code you like - generators, nested scopes, class: > statement bodies, all these cases where Psyco give up. That's the point > of the approach, really :-) > > We have a minor detail to solve before it can be tested on larger > examples, though - it exhausts the 32-bit address space far too early > (without actually consuming much of the reserved pages) and then we get > a MemoryError. It's a back-end problem; Arre started working on that > today. > > > A bientot, > > Armin. > From len-l at telus.net Thu Dec 14 23:12:57 2006 From: len-l at telus.net (Lenard Lindstrom) Date: Thu, 14 Dec 2006 14:12:57 -0800 Subject: [pypy-dev] Confusion with meaning of "Stackless" In-Reply-To: <20061214064857.GA14933@code0.codespeak.net> References: <4580213D.9801.19B2FC9@len-l.telus.net> Message-ID: <45815BE9.10807.12AF266@len-l.telus.net> On 14 Dec 2006 at 7:48, Armin Rigo wrote: > Hi Lenard, > > On Wed, Dec 13, 2006 at 03:50:21PM -0800, Lenard Lindstrom wrote: > > Tasklets are nice, but I just wish to update my understanding of > > "Stackless". Does the old motto "A Python Implementation That Does > > Not Use The C Stack" [1] still apply? Is it relevant to PyPy? > > Yes and no. A bit confusingly, the usage of the term has evolved, along > with the Stackless Python project itself. Now, "a Stackless feature" > means a feature that cannot be implemented without some form of explicit > stack control, either by being very careful about the C code (e.g. > avoiding recursive calls) or by saving and restoring pieces of the C > stack by a brute force 'memcpy' approach. In PyPy, we use a variant of > the former approach: a "stackless build" of pypy-c is a compiled version > of some C code that was generated with careful systematic tweaks. > > The motto of a Stackless PyPy is probably: "A Python implementation that > uses the C Stack as a cache". Indeed, the C code supports saving its > own stack of frames away into the heap, and restoring it from there, > frame by frame. So the C stack is really just a cache for the heap. If > the cache is full (stack overflow), we save some more frames into the > heap to free some cache space. If a context switch occurs (coroutine > switch) the cache is invalidated (entierely flushed to the heap) and the > new context's frames are copied from the heap to the cache as needed > (the function's frames are resumed one by one). > Hi Armin, Thanks for the explanation. I find that "Stackless" is still associated with Scheme like continuations and unlimited recursion [1]. But yes, the PyPy documentation clearly states that "Stackless" means greenlets, coroutines and tasklets at the application level. [1] http://en.wikipedia.org/wiki/Stackless Lenard Lindstrom From arigo at tunes.org Fri Dec 15 10:17:28 2006 From: arigo at tunes.org (Armin Rigo) Date: Fri, 15 Dec 2006 10:17:28 +0100 Subject: [pypy-dev] [stephan.diehl@gmx.net: Re: [pypy-svn] r35720 - pypy/dist/pypy/lib] Message-ID: <20061215091728.GA25265@code0.codespeak.net> Hi all, With his permission... (Note that user/stephan/hacks/coroutines is by no way stable, according to Stephan.) Armin ----- Forwarded message from Stephan Diehl ----- Date: Thu, 14 Dec 2006 08:59:36 +0100 From: Stephan Diehl To: arigo at tunes.org Subject: Re: [pypy-svn] r35720 - pypy/dist/pypy/lib Hi Armin, arigo at codespeak.net wrote: >Author: arigo >Date: Thu Dec 14 07:31:17 2006 >New Revision: 35720 > >Added: > pypy/dist/pypy/lib/stackless.py > - copied unchanged from r35688, pypy/dist/pypy/lib/stackless.py > pypy/dist/pypy/lib/stackless_new.py > - copied unchanged from r35719, pypy/dist/pypy/lib/stackless.py >Removed: > pypy/dist/pypy/lib/stackless_old.py >Log: >Temporarily keep the old stackless.py. The new one seems to break all >tests depending on it (at least its own test_stackless, and >test_distributed). > >Stephan: I suspect that you only tested it on top of CPython with your >greenlet-based wrapper (could you check it in somewhere too, btw?). The >behavior of this wrapper probably differs from the behavior of the >module/_stackless coroutine in some way. Arghhh. Actually, I ran ../../../test_all.py test_stackless.py from within the pypy/module/_stackless/test directory and it passed everything. Otherwise I wouldn't have replaced the stuff. Sigh. Anyway. All the stuff can be found in http://codespeak.net/svn/user/stephan/hacks/coroutine/ (the new stackless is called stackless2 :-) The tests I have there (test_stackless2.py) are passing when running on top of CPython, CStackless and pypy-c. (The secret plan is of course, to have the stackless interface available to CPython users --- minus the pickling, of course) The coroutine module is just the (old) interp_coroutine minus the parts that depend on pypy. I suspect that it could be made much, much thinner. Whenever this (greenlet based) coroutine is ready, I'd like to place it in py.magic. I'm really sorry about the breakage. pypy is really a difficult beast :-) Cheers Stephan P.S.: Unfortunatelly, I don't have any time left for this week, but I'll continue next week, to hunt down the bugs and complete the interface (and write much, much more tests as most features of stackless are not really covert yet) > >_______________________________________________ >pypy-svn mailing list >pypy-svn at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-svn > ----- End forwarded message ----- From jcea at argo.es Fri Dec 15 12:48:03 2006 From: jcea at argo.es (Jesus Cea) Date: Fri, 15 Dec 2006 12:48:03 +0100 Subject: [pypy-dev] Interesting technology to look at: Strongtalk Message-ID: <45828B73.4050004@argo.es> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Since both smalltalk and python are highly dynamic language without static type information, efficient execution is a challenge. This smaltalk virtual machine efficiency is stunning. Maybe could give some ideas to pypy effort. http://www.strongtalk.org/ - -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea at argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ jabber / xmpp:jcea at jabber.org _/_/ _/_/ _/_/_/_/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQCVAwUBRYKLc5lgi5GaxT1NAQKaogQAiKTVixiPa5q/94XE6EN+fr+lJ8+lTB+j nkui/g4i0juLcyHzPY3hIcNxDSw5oDF/ljHQaLs1y8z2nwwQwI/TUAXWuxqCzmP8 h/t9BSKlywG4yieKToLXR4YEoDZvsW9640Crx28CNsJUP8fFhvjh6rvmuKSjbQmh iyiyR1MCmys= =seEN -----END PGP SIGNATURE----- From stephan.diehl at gmx.net Sat Dec 16 10:28:18 2006 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Sat, 16 Dec 2006 10:28:18 +0100 Subject: [pypy-dev] update on CPython coroutine Message-ID: <4583BC32.7070303@gmx.net> Hi all, after my (unsuccessful:-) attempt to place CPython runnable coroutines into py.magic (they might end up there just after the next release, we'll see), they live now in http://codespeak.net/svn/user/stephan/hacks/coroutine/coroutine.py In order to use them, on must place that file into the python path. I've fixed a couple of things, Armin pointed out to me. I'd be happy to get some feedback. Background: the pypy.lib.stackless_new implementation relies only on coroutines as a special requirement. If one had coroutines that were running on CPython, one could run have the stackless interface on CPython as well (minus the pickling, of course). It turned out that the coroutine interface is so similar to the greenlet one, that just a few lines of code are enough to provide the coroutine interface to greenlets. Have a nice weekend Stephan From david at ar.media.kyoto-u.ac.jp Mon Dec 18 04:32:45 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Mon, 18 Dec 2006 12:32:45 +0900 Subject: [pypy-dev] Newcommer in pypy, numpy user Message-ID: <45860BDD.4020004@ar.media.kyoto-u.ac.jp> Hi there, I am a recently convert of python, which I started using for my numerical computing needs (I am a PhD student in signal processing) to replace matlab. For those who do not know matlab, it is a big (and expensive) software which implements a 'language' optimized for linear algebra, and with time becomes one of the most used software for numerical computation. In my field of research (signal processing), matlab is almost a standard (by standard, I mean everybody knows it and uses it). I think python with numpy/scipy is better in almost any way if you are ready to invest some time. Now, concerning pypy. The main idea of numpy is to give an array class to python, so that most inner loops are not interpreted, but run in Cpython or through highly optimized fortran libraries. In those cases, python is fast enough for most cases. But there are some cases where this paradigm of using linear algebra to speed things up does not work really well (recursive algorithms); in those case, the loop + function call cost of python makes any implementation for non toy problems really slow. Right now, the only choice is to code the thing in C, with a big loss on the flexibility side. I was wondering if pypy has some solution/new approaches to this problem. For example, when using numpy, I would suspect that many functions calls are 'static', that is always expect the same type of arguments; also, for simple loop on integers, my understanding is that JIT compilation has some nice solution to give to have much better performances (matlab has a JIT compiler to make loop faster for interpreted code). It looks like at some point, there was some work done in pypy relatively to numpy arrays, but I didn't find any documentation on that. cheers, David From mwh at python.net Mon Dec 18 17:39:36 2006 From: mwh at python.net (Michael Hudson) Date: Mon, 18 Dec 2006 17:39:36 +0100 Subject: [pypy-dev] Newcommer in pypy, numpy user References: <45860BDD.4020004@ar.media.kyoto-u.ac.jp> Message-ID: <87ejqx17lj.fsf@starship.python.net> David Cournapeau writes: > Hi there, > > I am a recently convert of python, which I started using for my > numerical computing needs (I am a PhD student in signal processing) to > replace matlab. > > For those who do not know matlab, it is a big (and expensive) > software which implements a 'language' optimized for linear algebra, and > with time becomes one of the most used software for numerical > computation. In my field of research (signal processing), matlab is > almost a standard (by standard, I mean everybody knows it and uses it). > I think python with numpy/scipy is better in almost any way if you are > ready to invest some time. > > Now, concerning pypy. The main idea of numpy is to give an array > class to python, so that most inner loops are not interpreted, but run > in Cpython or through highly optimized fortran libraries. In those > cases, python is fast enough for most cases. But there are some cases > where this paradigm of using linear algebra to speed things up does not > work really well (recursive algorithms); in those case, the loop + > function call cost of python makes any implementation for non toy > problems really slow. Right now, the only choice is to code the thing in > C, with a big loss on the flexibility side. > > I was wondering if pypy has some solution/new approaches to this > problem. For example, when using numpy, I would suspect that many > functions calls are 'static', that is always expect the same type of > arguments; also, for simple loop on integers, my understanding is that > JIT compilation has some nice solution to give to have much better > performances (matlab has a JIT compiler to make loop faster for > interpreted code). It may be that the JIT is good for this sort of code. It's not yet though :-) > It looks like at some point, there was some work done in pypy > relatively to numpy arrays, but I didn't find any documentation on > that. As far as I'm aware this was something else: teaching PyPy's annotator to recognise code that uses Numeric arrays and the code generator how to compile this to equivalent code that manipulates arrays at a lower level. This could be seen as an alternative to rewriting your code in C. I'm not sure what the state of this code is, but I don't think it's very advanced. Cheers, mwh -- The Internet is full. Go away. -- http://www.disobey.com/devilshat/ds011101.htm From tismer at stackless.com Mon Dec 18 21:15:24 2006 From: tismer at stackless.com (Christian Tismer) Date: Mon, 18 Dec 2006 21:15:24 +0100 Subject: [pypy-dev] Confusion with meaning of "Stackless" In-Reply-To: <45815BE9.10807.12AF266@len-l.telus.net> References: <4580213D.9801.19B2FC9@len-l.telus.net> <45815BE9.10807.12AF266@len-l.telus.net> Message-ID: <4586F6DC.40903@stackless.com> Lenard Lindstrom wrote: > On 14 Dec 2006 at 7:48, Armin Rigo wrote: ... > Thanks for the explanation. I find that "Stackless" is still > associated with Scheme like continuations and unlimited recursion > [1]. But yes, the PyPy documentation clearly states that "Stackless" > means greenlets, coroutines and tasklets at the application level. Well, I should try to update that page. While it is still true, the basic machinery can support continuations, and infinite recursion was available in every Stackless. The "tamed" continuations which are greenlets, coroutines or tasklets seem to be what is much more acceptable for the users. ciao - chris -- Christian Tismer :^) tismerysoft GmbH : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9A : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 802 86 56 mobile +49 177 402 25 57 fax +49 30 80 90 57 05 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 where do you want to jump today? http://www.stackless.com/ From len-l at telus.net Tue Dec 19 01:15:48 2006 From: len-l at telus.net (Lenard Lindstrom) Date: Mon, 18 Dec 2006 16:15:48 -0800 Subject: [pypy-dev] Confusion with meaning of "Stackless" In-Reply-To: <4586F6DC.40903@stackless.com> References: <45815BE9.10807.12AF266@len-l.telus.net> Message-ID: <4586BEB4.13298.19981EC@len-l.telus.net> On 18 Dec 2006 at 21:15, Christian Tismer wrote: > Lenard Lindstrom wrote: > > On 14 Dec 2006 at 7:48, Armin Rigo wrote: > > ... > > Thanks for the explanation. I find that "Stackless" is still > > associated with Scheme like continuations and unlimited recursion > > [1]. But yes, the PyPy documentation clearly states that "Stackless" > > means greenlets, coroutines and tasklets at the application level. > > Well, I should try to update that page. > While it is still true, the basic machinery > can support continuations, and infinite recursion > was available in every Stackless. The "tamed" continuations > which are greenlets, coroutines or tasklets seem to be > what is much more acceptable for the users. > Taskets interest people. But so did infinite recursion. That it is still a part of Stackless Python is worth mentioning. And if Stackless PyPy supports it then infinite recursion should be documented as a user level feature along with tasklets and such. It does effect the choices a coder makes. As for continuations, I have only seen them advertised as a language feature when the language provides a call/cc or something. Tasklets, coroutines, generators and such are basically treated as independent features. Lenard Lindstrom From arigo at tunes.org Tue Dec 19 13:36:00 2006 From: arigo at tunes.org (Armin Rigo) Date: Tue, 19 Dec 2006 13:36:00 +0100 Subject: [pypy-dev] Confusion with meaning of "Stackless" In-Reply-To: <4586BEB4.13298.19981EC@len-l.telus.net> References: <45815BE9.10807.12AF266@len-l.telus.net> <4586BEB4.13298.19981EC@len-l.telus.net> Message-ID: <20061219123600.GA17671@code0.codespeak.net> Hi Lenard, On Mon, Dec 18, 2006 at 04:15:48PM -0800, Lenard Lindstrom wrote: > Taskets interest people. But so did infinite recursion. That it is > still a part of Stackless Python is worth mentioning. And if > Stackless PyPy supports it then infinite recursion should be > documented as a user level feature along with tasklets and such. It > does effect the choices a coder makes. Good point. I've added it to stackless.html. A bientot, Armin From aurelien.campeas at logilab.fr Tue Dec 19 18:50:46 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Tue, 19 Dec 2006 18:50:46 +0100 Subject: [pypy-dev] some questions Message-ID: <20061219175046.GI28906@crater.logilab.fr> Dear pypyers, I've resumed work on WP9, with the intent to finish it ASAP. Right now I face a little difficulty. I'd like to have your input (essentially Armin's input I guess) on this. Back when I was thinking about the requirements of WP9, esp. the non-deterministic aspects, I thought clonable coroutines/threads would do, fuzzily. Yet obviously these features have been provided and could still be used to achieve my goals, but that is not what I have decided to do since then... What I'd like to know if I have painted myself in a corner with the current design of the logic objectspace. Right now, a computation space is defined as a set of concurrent activities (thread group); in the case of constraint solving, there is one 'main' thread doing the choices, and an army of propagator threads trying to empty the domains. The idea is to associate a garbage collector pool to such a set. I wrote the pool handling code and the clone stuff following as closely as possible what's in modules/_stackless/interp_clonable and friends (most recent versions I think); without a clear understanding of how it works ... (see in objspace/cclp/types.py the Pool class, some client code in space.py and scheduler.py) Then, what happens now is that the cloning routine returns the original thread group. My question is the following : * is there a chance that a peer review and eventual correction/advice on that non-working code can yield a working solution in a reasonnable amount of time ? or * is it time for me to simplify considerably the computation space down to one single thread, so as to directly peruse the (working) coroutine cloning facility ? The later option would entail throwing away ten days of work, and spending a more days to reorganize and rewrite stuff, but can be done while retaining a reasonnable amount of functionnality. (Well, in truth I expect a quick code review by Armin to expose some blunders of mine... Just don't hit too hard my poor head.) Regards, Aur?lien. From david at ar.media.kyoto-u.ac.jp Wed Dec 20 03:50:37 2006 From: david at ar.media.kyoto-u.ac.jp (David Cournapeau) Date: Wed, 20 Dec 2006 11:50:37 +0900 Subject: [pypy-dev] Newcommer in pypy, numpy user In-Reply-To: <87ejqx17lj.fsf@starship.python.net> References: <45860BDD.4020004@ar.media.kyoto-u.ac.jp> <87ejqx17lj.fsf@starship.python.net> Message-ID: <4588A4FD.90009@ar.media.kyoto-u.ac.jp> Michael Hudson wrote: > David Cournapeau writes: > >> Hi there, >> >> I am a recently convert of python, which I started using for my >> numerical computing needs (I am a PhD student in signal processing) to >> replace matlab. >> >> For those who do not know matlab, it is a big (and expensive) >> software which implements a 'language' optimized for linear algebra, and >> with time becomes one of the most used software for numerical >> computation. In my field of research (signal processing), matlab is >> almost a standard (by standard, I mean everybody knows it and uses it). >> I think python with numpy/scipy is better in almost any way if you are >> ready to invest some time. >> >> Now, concerning pypy. The main idea of numpy is to give an array >> class to python, so that most inner loops are not interpreted, but run >> in Cpython or through highly optimized fortran libraries. In those >> cases, python is fast enough for most cases. But there are some cases >> where this paradigm of using linear algebra to speed things up does not >> work really well (recursive algorithms); in those case, the loop + >> function call cost of python makes any implementation for non toy >> problems really slow. Right now, the only choice is to code the thing in >> C, with a big loss on the flexibility side. >> >> I was wondering if pypy has some solution/new approaches to this >> problem. For example, when using numpy, I would suspect that many >> functions calls are 'static', that is always expect the same type of >> arguments; also, for simple loop on integers, my understanding is that >> JIT compilation has some nice solution to give to have much better >> performances (matlab has a JIT compiler to make loop faster for >> interpreted code). > > It may be that the JIT is good for this sort of code. It's not yet > though :-) I understand that pypy is still in (relative) infancy, but I have to confess I am more and more interested in all those concepts of JIT, etc... even if I don't know much outside the concept and the big picture. > > As far as I'm aware this was something else: teaching PyPy's annotator > to recognise code that uses Numeric arrays and the code generator how > to compile this to equivalent code that manipulates arrays at a lower > level. This could be seen as an alternative to rewriting your code in > C. I'm not sure what the state of this code is, but I don't think > it's very advanced. > I am not familiar with the pypy vocabulary yet, so let me rephrase to be sure I understand: the idea is to detect numpy arrays, and instead of using C extension for fast computation, it would generate automatically code in the target language (let's say C for C generation) ? So for example, if a and b are numpy arrays, a python expression b *= a would be translated in C by something like for(i = 0; i < b.size; ++i) {b[i] *= a[i];} ? Is anyone working on that ? cheers, David From mwh at python.net Wed Dec 20 11:12:03 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 20 Dec 2006 11:12:03 +0100 Subject: [pypy-dev] Newcommer in pypy, numpy user References: <45860BDD.4020004@ar.media.kyoto-u.ac.jp> <87ejqx17lj.fsf@starship.python.net> <4588A4FD.90009@ar.media.kyoto-u.ac.jp> Message-ID: <87wt4myiz0.fsf@starship.python.net> David Cournapeau writes: > Michael Hudson wrote: >> David Cournapeau writes: >>> I was wondering if pypy has some solution/new approaches to this >>> problem. For example, when using numpy, I would suspect that many >>> functions calls are 'static', that is always expect the same type of >>> arguments; also, for simple loop on integers, my understanding is that >>> JIT compilation has some nice solution to give to have much better >>> performances (matlab has a JIT compiler to make loop faster for >>> interpreted code). >> >> It may be that the JIT is good for this sort of code. It's not yet >> though :-) > I understand that pypy is still in (relative) infancy, but I have to > confess I am more and more interested in all those concepts of JIT, > etc... even if I don't know much outside the concept and the big picture. It's definitely interesting stuff :-) >> As far as I'm aware this was something else: teaching PyPy's annotator >> to recognise code that uses Numeric arrays and the code generator how >> to compile this to equivalent code that manipulates arrays at a lower >> level. This could be seen as an alternative to rewriting your code in >> C. I'm not sure what the state of this code is, but I don't think >> it's very advanced. >> > I am not familiar with the pypy vocabulary yet, so let me rephrase to be > sure I understand: the idea is to detect numpy arrays, and instead of > using C extension for fast computation, it would generate automatically > code in the target language (let's say C for C generation) ? > > So for example, if a and b are numpy arrays, a python expression b *= a > would be translated in C by something like for(i = 0; i < b.size; ++i) > {b[i] *= a[i];} ? Yes, I think that's a pretty good description. > Is anyone working on that ? Not that I know of. Cheers, mwh -- > say-hi-to-the-flying-pink-elephants-for-me-ly y'rs, No way, the flying pink elephants are carrying MACHINE GUNS! Aiiee!! Time for a kinder, gentler hallucinogen... -- Barry Warsaw & Greg Ward, python-dev From faassen at startifact.com Wed Dec 20 14:43:52 2006 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 20 Dec 2006 14:43:52 +0100 Subject: [pypy-dev] r35918 - pypy/dist/pypy/config In-Reply-To: <20061220132226.ED5D21007E@code0.codespeak.net> References: <20061220132226.ED5D21007E@code0.codespeak.net> Message-ID: mwh at codespeak.net wrote: > Author: mwh > Date: Wed Dec 20 14:22:23 2006 > New Revision: 35918 > > Modified: > pypy/dist/pypy/config/pypyoption.py > Log: > add a --faassen command line optimization that enables all optimizations. I saw that. :) Regards, Martijn From mwh at python.net Wed Dec 20 15:24:33 2006 From: mwh at python.net (Michael Hudson) Date: Wed, 20 Dec 2006 15:24:33 +0100 Subject: [pypy-dev] r35918 - pypy/dist/pypy/config References: <20061220132226.ED5D21007E@code0.codespeak.net> Message-ID: <87zm9iwspq.fsf@starship.python.net> Martijn Faassen writes: > mwh at codespeak.net wrote: >> Author: mwh >> Date: Wed Dec 20 14:22:23 2006 >> New Revision: 35918 >> >> Modified: >> pypy/dist/pypy/config/pypyoption.py >> Log: >> add a --faassen command line optimization that enables all optimizations. > > I saw that. :) Good :-) Cheers, mwh -- the highest calling of technical book writers is to destroy the sun -- from Twisted.Quotes From guido at merlinux.de Wed Dec 20 15:50:44 2006 From: guido at merlinux.de (Guido Wesdorp) Date: Wed, 20 Dec 2006 15:50:44 +0100 Subject: [pypy-dev] r35918 - pypy/dist/pypy/config In-Reply-To: References: <20061220132226.ED5D21007E@code0.codespeak.net> Message-ID: <45894DC4.8080405@merlinux.de> Martijn Faassen wrote: > mwh at codespeak.net wrote: > >> Author: mwh >> Date: Wed Dec 20 14:22:23 2006 >> New Revision: 35918 >> >> Modified: >> pypy/dist/pypy/config/pypyoption.py >> Log: >> add a --faassen command line optimization that enables all optimizations. >> > > I saw that. :) > > Hahaha! Cheers, Guido From cfbolz at gmx.de Wed Dec 20 15:51:51 2006 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 20 Dec 2006 15:51:51 +0100 Subject: [pypy-dev] r35918 - pypy/dist/pypy/config In-Reply-To: <45894DC4.8080405@merlinux.de> References: <20061220132226.ED5D21007E@code0.codespeak.net> <45894DC4.8080405@merlinux.de> Message-ID: <45894E07.3040509@gmx.de> Guido Wesdorp wrote: > Martijn Faassen wrote: >> mwh at codespeak.net wrote: >> >>> Author: mwh >>> Date: Wed Dec 20 14:22:23 2006 >>> New Revision: 35918 >>> >>> Modified: >>> pypy/dist/pypy/config/pypyoption.py >>> Log: >>> add a --faassen command line optimization that enables all optimizations. >>> >> I saw that. :) >> >> > Hahaha! > > Cheers, > > Guido Admit it, Guido, it's all your fault! :-) Carl Friedrich From simon at arrowtheory.com Wed Dec 20 17:08:24 2006 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 20 Dec 2006 08:08:24 -0800 Subject: [pypy-dev] Newcommer in pypy, numpy user In-Reply-To: <4588A4FD.90009@ar.media.kyoto-u.ac.jp> References: <45860BDD.4020004@ar.media.kyoto-u.ac.jp> <87ejqx17lj.fsf@starship.python.net> <4588A4FD.90009@ar.media.kyoto-u.ac.jp> Message-ID: <20061220080824.4cf6bd1d.simon@arrowtheory.com> On Wed, 20 Dec 2006 11:50:37 +0900 David Cournapeau wrote: > > So for example, if a and b are numpy arrays, a python expression b *= a > would be translated in C by something like for(i = 0; i < b.size; ++i) > {b[i] *= a[i];} ? Yes, but numpy already does this. The ultimate goal was to be able to collapse many such operations into an "optimal" (minimal) number of loops, eg. a = b + c + d => for(...) {a[i]=b[i]+c[i]+d[i]} which is much more cache friendly. Simon. From guido at merlinux.de Wed Dec 20 21:39:31 2006 From: guido at merlinux.de (Guido Wesdorp) Date: Wed, 20 Dec 2006 21:39:31 +0100 Subject: [pypy-dev] r35918 - pypy/dist/pypy/config In-Reply-To: <45894E07.3040509@gmx.de> References: <20061220132226.ED5D21007E@code0.codespeak.net> <45894DC4.8080405@merlinux.de> <45894E07.3040509@gmx.de> Message-ID: <45899F83.1000403@merlinux.de> Carl Friedrich Bolz wrote: > Admit it, Guido, it's all your fault! > > Fault? It's an honour, isn't it? :) Cheers, Guido From arigo at tunes.org Thu Dec 21 17:44:00 2006 From: arigo at tunes.org (Armin Rigo) Date: Thu, 21 Dec 2006 17:44:00 +0100 Subject: [pypy-dev] some questions In-Reply-To: <20061219175046.GI28906@crater.logilab.fr> References: <20061219175046.GI28906@crater.logilab.fr> Message-ID: <20061221164400.GA8604@code0.codespeak.net> Hi Aurelien, On Tue, Dec 19, 2006 at 06:50:46PM +0100, Aur?lien Camp?as wrote: > * is there a chance that a peer review and eventual correction/advice > on that non-working code can yield a working solution in a reasonnable > amount of time ? No. Getting the original coroutine cloning code to really work correctly took us quite some time, and certainly more than 10 days, including a final night following data structures at the C level with ddd. (Btw, that's a gdb front-end that I really recommend for Debugging and Displaying Data :-) It looks innocent enough but it is really subtle code, so you should try to build on top of that and not reinvent it from scratch. Note also that if you need sets of several coroutines that are all cloned together, it's possible to put each set in a single InterpClonableCoroutine that acts as a "big" thread; and it would itself contain several regular Coroutines for its own local purposes. This works nicely if in each InterpClonableCoroutine you first create a new CoState() instance, and use it as the 'state' argument of all the local Coroutines. This local CoState has a current and a main that are local too. If you clone one of the "big" InterpClonableCoroutine, everything will be cloned - the local CoState and its Coroutines. Hum, actually for now that's theory only. The local Coroutines have a __del__ and objects with a __del__ are never cloned. I guess that my point is that we could make this work with minimal efforts (e.g. with a variety of Coroutines that doesn't need a __del__). > The later option would entail throwing away ten days of work, and > spending a more days to reorganize and rewrite stuff, but can be done > while retaining a reasonnable amount of functionnality. Well, I would have given you the same answer 10 days ago, or even a few months ago, if only I had known. I should point out that we ("the technical board") tried to ask Logilab in general to tell us what plans they had, and we tried quite a few times now, so I fail to feel bad about telling you now to throw away just 10 days of work, sorry. A bientot, Armin From aurelien.campeas at logilab.fr Thu Dec 21 18:13:25 2006 From: aurelien.campeas at logilab.fr (=?iso-8859-1?Q?Aur=E9lien_Camp=E9as?=) Date: Thu, 21 Dec 2006 18:13:25 +0100 Subject: [pypy-dev] some questions In-Reply-To: <20061221164400.GA8604@code0.codespeak.net> References: <20061219175046.GI28906@crater.logilab.fr> <20061221164400.GA8604@code0.codespeak.net> Message-ID: <20061221171325.GB20558@crater.logilab.fr> On Thu, Dec 21, 2006 at 05:44:00PM +0100, Armin Rigo wrote: > Hi Aurelien, > > On Tue, Dec 19, 2006 at 06:50:46PM +0100, Aur?lien Camp?as wrote: > > * is there a chance that a peer review and eventual correction/advice > > on that non-working code can yield a working solution in a reasonnable > > amount of time ? > > No. Getting the original coroutine cloning code to really work > correctly took us quite some time, and certainly more than 10 days, > including a final night following data structures at the C level with > ddd. (Btw, that's a gdb front-end that I really recommend for Debugging > and Displaying Data :-) It looks innocent enough but it is really > subtle code, so you should try to build on top of that and not reinvent > it from scratch. I never meant to reinvent this from scratch. You also said to me recently that the local pooling was not especially tied to coroutines, which matched the intuition I had from reading the interp_clonable stuff. > > Note also that if you need sets of several coroutines that are all > cloned together, it's possible to put each set in a single > InterpClonableCoroutine that acts as a "big" thread; and it would itself > contain several regular Coroutines for its own local purposes. This > works nicely if in each InterpClonableCoroutine you first create a new > CoState() instance, and use it as the 'state' argument of all the local > Coroutines. This local CoState has a current and a main that are local > too. If you clone one of the "big" InterpClonableCoroutine, everything > will be cloned - the local CoState and its Coroutines. Some time ago I tried that road but abandonned it because it yielded non-working builds. But at that time I missed the "this works nicely if ..." part. Thanks for taking the time to talk about it. > > Hum, actually for now that's theory only. The local Coroutines have a > __del__ and objects with a __del__ are never cloned. I guess that my > point is that we could make this work with minimal efforts (e.g. with a > variety of Coroutines that doesn't need a __del__). I don't think in my usage of Coroutines I need a __del__. So maybe there is a tiny hope. > > > The later option would entail throwing away ten days of work, and > > spending a more days to reorganize and rewrite stuff, but can be done > > while retaining a reasonnable amount of functionnality. > > Well, I would have given you the same answer 10 days ago, or even a few > months ago, if only I had known. I should point out that we ("the > technical board") tried to ask Logilab in general to tell us what plans > they had, and we tried quite a few times now, Logilab's plan wrt WP9 have always been clear I think, but whatever. Talking about logilab relationships with the PyPy group is for another day. > so I fail to feel bad > about telling you now to throw away just 10 days of work, sorry. Well, sorry if you interpreted my poor wording about days of work being thrown away as an attempt to culpabilize you. That was not my intent. Moreover I highly respect what you have done for the cloning facility and everything in general the core PyPy developpers have achieved so far. Thanks, Aur?lien. From arigo at tunes.org Wed Dec 27 11:44:34 2006 From: arigo at tunes.org (Armin Rigo) Date: Wed, 27 Dec 2006 11:44:34 +0100 Subject: [pypy-dev] Machines at uni-duesseldorf Message-ID: <20061227104434.GA10138@code0.codespeak.net> Hi all, Here's an update on the computers that we have for PyPy at the HHU. Snake is back, but nothing wrong was found with its hardware, which means that it'll probably continue to hang randomly from time to time... Maybe we'll reinstall the machine from scratch. For all I know it could be corrupted system files. However, there are now two other machines. Just like snake they are machines whose performances can be a bit strange - hyperthreaded Xeon cores. Each core has roughly snake-like performance, but each machine has got 4 cores. The software setup is: SuSE Linux base, which I'm hiding as much as possible behind a Gentoo Linux installation in the /gentoo directory. When you log in, you reach a chrooted environment, so you should not see that at all unless you dig in special files. I've copied some accounts from codespeak, so you must use the same public key to log in (no password logins). Just ask me if you want to use the machines but I forgot to copy your account over. To login, you must use port 922 instead of 22. The latter would reach the SSH daemon of the outer SuSE installation (where you don't have an account). I recommend adding the following lines to your file ~/.ssh/config: Host wyvern HostName wyvern.cs.uni-duesseldorf.de Port 922 Host cobra HostName cobra.cs.uni-duesseldorf.de Port 922 Then you can log in with 'ssh wyvern' or 'ssh cobra'. Note that 'wyvern' is the fastest machine so feel free to prefer it. A great way to use them is for distributed testing. Put in your file ~/conftest.py: disthosts = ['wyvern'] * 4 + ['cobra'] * 4 Then you can do 'py.test --session=R'. Try also the --runbrowser option. A bientot, Armin From arigo at tunes.org Sat Dec 30 16:05:49 2006 From: arigo at tunes.org (Armin Rigo) Date: Sat, 30 Dec 2006 16:05:49 +0100 Subject: [pypy-dev] Automated tests Message-ID: <20061230150546.GA17431@code0.codespeak.net> Hi all, The automated tests are running on wyvern nightly: http://wyvern.cs.uni-duesseldorf.de/pypytest/summary.html As the tests are parallelized it takes on the order of two hours to run them all, so we could consider running them more than once per day. Something else I'd like to consider is a tool that checks new failures shown by these test runs, finding the exact revision that seemed to broke it, and actively signals the problem somewhere - maybe as a notice by elbowtone on #pypy and as an e-mail to e.g. the author of the revision? What do you think about it? Longer term, we could also have a way to ask for a complete test run on wyvern, so that we can check within two hours if we broke something after a delicate check-in, or just after we fixed a couple of problems. If we manage 0-failure runs at least every few days we could then have an URL to which we copy the trunk whenever all tests pass, so that "outside" people have the option to follow with 'svn up' an URL that is at least fully tested instead of the bleeding edge trunk. A bientot, Armin From santagada at gmail.com Sat Dec 30 17:39:01 2006 From: santagada at gmail.com (Leonardo Santagada) Date: Sat, 30 Dec 2006 14:39:01 -0200 Subject: [pypy-dev] Automated tests In-Reply-To: <20061230150546.GA17431@code0.codespeak.net> References: <20061230150546.GA17431@code0.codespeak.net> Message-ID: <24D9047B-1393-44E9-927C-1AA786EC1CC8@gmail.com> Em 30/12/2006, ?s 13:05, Armin Rigo escreveu: > Hi all, > > The automated tests are running on wyvern nightly: > > http://wyvern.cs.uni-duesseldorf.de/pypytest/summary.html > > As the tests are parallelized it takes on the order of two hours to > run > them all, so we could consider running them more than once per day. > > Something else I'd like to consider is a tool that checks new failures > shown by these test runs, finding the exact revision that seemed to > broke it, and actively signals the problem somewhere - maybe as a > notice > by elbowtone on #pypy and as an e-mail to e.g. the author of the > revision? What do you think about it? > Does buildbot do something like this or you will have to implement all yourself? > Longer term, we could also have a way to ask for a complete test > run on > wyvern, so that we can check within two hours if we broke something > after a delicate check-in, or just after we fixed a couple of > problems. > If we manage 0-failure runs at least every few days we could then have > an URL to which we copy the trunk whenever all tests pass, so that > "outside" people have the option to follow with 'svn up' an URL > that is > at least fully tested instead of the bleeding edge trunk. > We could just have a moving tag named "latest_tested" and people could be following it (or a branch, I really don't know that much about svn) > > A bientot, > > Armin > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev Leonardo Santagada santagada at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20061230/8eaff1e7/attachment-0001.htm