From niko at alum.mit.edu Sun Apr 1 07:48:39 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Sun, 1 Apr 2007 07:48:39 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <460D916E.8070008@gmail.com> References: <460D61A7.2060908@gmail.com> <460D916E.8070008@gmail.com> Message-ID: > The last time I checked what java's Hasttable offers and I saw you > can't pass to it custom hashing and equality functions, but maybe > there is a simple way to do it that I don't know. No, there isn't, but it shouldn't be too hard to cook up some kind of Hashtable substitute that uses small wrapper classes to handle that. I think that's what you did for C#, right? Niko From anto.cuni at gmail.com Sun Apr 1 13:46:19 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sun, 01 Apr 2007 13:46:19 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: References: <460D61A7.2060908@gmail.com> <460D916E.8070008@gmail.com> Message-ID: <460F9B8B.7010106@gmail.com> Niko Matsakis wrote: >> The last time I checked what java's Hasttable offers and I saw you >> can't pass to it custom hashing and equality functions, but maybe >> there is a simple way to do it that I don't know. > > No, there isn't, but it shouldn't be too hard to cook up some kind of > Hashtable substitute that uses small wrapper classes to handle that. I > think that's what you did for C#, right? No, for .NET it was simpler because the standard Dictionary type also accepts an optional class that implements the custom functions, so all I need to do is to create a class for each unique pair of equality and hashing functions (see cli/comparer.py). ciao Anto From lkcl at lkcl.net Mon Apr 2 13:58:09 2007 From: lkcl at lkcl.net (Luke Kenneth Casson Leighton) Date: Mon, 2 Apr 2007 11:58:09 +0000 (UTC) Subject: [pypy-dev] Paul Fernhout wants edit-and-continue in python Message-ID: <7379457735lkcl@lkcl.net> integration with leo - http://leo.sf.net - would be a good start. that would give you the GUI part, and context-sensitive execution. you _might_ have to do something similar to what gcc currently do, which is to hook in to eevvvveerryyything via __blahblah__ functions, for memory access eevveryyyything, in the "profiling". but that shouldn't be too hard (but would result in a performance hit - oh well, can't have everything). execution itself (from leo) could be performed via "yield". ok. you can obtain a list of all variables (global etc.) that a particular function modifies, right? so, before you go into any function, right, you record the previous "state". in fact, you might be able to record "state" on a per-line basis (bit extreme i know) and basically have an "undo" stack yeh. it's very simple. look up how "undo" is implemented, in editors, and then treat all variables like that. then, you can keep a record (using Leo) of which lines were modified, and use that to calculate which bits need JIT recompilation and how far back the "undo" stack you need to roll. in _fact_, and this is going to _really_ blow you away, Leo probably already _has_ an "Undo" capability - and so if you mentally turn things round, by treating "execute 1 function" as an "edit" operation under Leo, then you get what you want, for free!!! ooooo :) of course, if any variables are "volatile" - e.g. in an OS, like cleese, then you're shafted - but that's another story. and, of course, if anything calls out to _non_ python code, then you're equally shafted. in both cases, you can simply throw an exception saying "sorry, you need to run this program from scratch, dude". you stand a better chance of doing this in pypy than the python developers do. they consider it "impossible", because it's too radical a rethink, because they have been trapped, by design, in c-code. l. p.s. leo is in python, in case you hadn't guessed. From paul.degrandis at gmail.com Tue Apr 3 18:45:22 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Tue, 3 Apr 2007 12:45:22 -0400 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <460F9B8B.7010106@gmail.com> References: <460D61A7.2060908@gmail.com> <460D916E.8070008@gmail.com> <460F9B8B.7010106@gmail.com> Message-ID: <9c0bb8a00704030945u16636af9k700172fa3379c831@mail.gmail.com> Hey guys, (and everyone else) I was working on an autonomic computing project using JDK6, and in the middle decided to switch up to hacking on PyPy and seeing if I could get a semi-functioning JSR-223 bindings by the end of the week. I svn uped, and wanted know if this release has any of the work the at you guys did? If not, let me know, and i'll get working so we can get a JVM translated PyPy Paul On 4/1/07, Antonio Cuni wrote: > > Niko Matsakis wrote: > >> The last time I checked what java's Hasttable offers and I saw you > >> can't pass to it custom hashing and equality functions, but maybe > >> there is a simple way to do it that I don't know. > > > > No, there isn't, but it shouldn't be too hard to cook up some kind of > > Hashtable substitute that uses small wrapper classes to handle that. I > > think that's what you did for C#, right? > > No, for .NET it was simpler because the standard Dictionary type also > accepts an optional class that implements the custom functions, so all I > need to do is to create a class for each unique pair of equality and > hashing functions (see cli/comparer.py). > > ciao Anto > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20070403/bd51419c/attachment.htm From niko at alum.mit.edu Tue Apr 3 20:06:21 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Tue, 3 Apr 2007 20:06:21 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <460F9B8B.7010106@gmail.com> References: <460D61A7.2060908@gmail.com> <460D916E.8070008@gmail.com> <460F9B8B.7010106@gmail.com> Message-ID: <1157E100-0D8F-4AF4-B03A-AF117756E6AA@alum.mit.edu> > No, for .NET it was simpler because the standard Dictionary type > also accepts an optional class that implements the custom > functions, so all I need to do is to create a class for each unique > pair of equality and hashing functions (see cli/comparer.py). So, on a whim I implemented a "RDict" class for the JVM that does the necessary adaptation. (It consists almost entirely of generic type declarations and no actual code ;) I haven't integrated it into the Python code yet, looking to see where best to do that. Are most of the tests for custom dicts in test_objectmodel? Also, why are they called "rdicts" in some places, and "custom dicts" in others? Or is there a distinction I am missing? Niko From niko at alum.mit.edu Wed Apr 4 09:54:21 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 4 Apr 2007 09:54:21 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <460D61A7.2060908@gmail.com> References: <460D61A7.2060908@gmail.com> Message-ID: <6E4AD99C-F907-43AC-9DB7-81FFD68BCB81@alum.mit.edu> > Some weeks ago I also tried to translate pypy-jvm; it seems that > the two most important missing features are r_dict and weakrefs. Ok, I implemented r_dicts now and checked it in. Not too much work, actually, ended up fitting fairly naturally into the existing code. I guess I'll look at weakrefs next, though no promises as to when that will be. :) Niko From anto.cuni at gmail.com Wed Apr 4 14:39:21 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 04 Apr 2007 14:39:21 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <6E4AD99C-F907-43AC-9DB7-81FFD68BCB81@alum.mit.edu> References: <460D61A7.2060908@gmail.com> <6E4AD99C-F907-43AC-9DB7-81FFD68BCB81@alum.mit.edu> Message-ID: <46139C79.4070403@gmail.com> Niko Matsakis wrote: >> Some weeks ago I also tried to translate pypy-jvm; it seems that the >> two most important missing features are r_dict and weakrefs. > > Ok, I implemented r_dicts now and checked it in. Not too much work, > actually, ended up fitting fairly naturally into the existing code. I Hi Niko! This is very cool! :-) Most of r_dict tests are in test_objectmodel, though you may want to have also a look at rpython/test/test_rconstantdict. About the distinction between r_dict and "custom dict": do you find any place where they are used interchangeably? I would say that r_dict referes so the rpython-level type (objectmodel.r_dict), while "custom dict" should refer to the low level type used by the rtyper (ootype.CustomDict). Also, probably CustomDict would be a better name than RDict for your java class, I guess. > guess I'll look at weakrefs next, though no promises as to when that > will be. :) Adding weakrefs to gencli was very simple: I just needed to map lltypesystem.llmemory.WeakGcAddress to 'System.WeakReference', add the straightforward support for constants to cli/constant.py and add the also straightforward 'cast_ptr_to_weakadr' and 'cast_weakadr_to_ptr' operations in opcodes.py. I don't know for jvm, but I guess it would not be much more complicate. ciao Anto From niko at alum.mit.edu Wed Apr 4 15:05:05 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Wed, 4 Apr 2007 15:05:05 +0200 Subject: [pypy-dev] Towards pypy-jvm In-Reply-To: <46139C79.4070403@gmail.com> References: <460D61A7.2060908@gmail.com> <6E4AD99C-F907-43AC-9DB7-81FFD68BCB81@alum.mit.edu> <46139C79.4070403@gmail.com> Message-ID: <2B86C471-9C18-4C9B-A247-9C086590ED06@alum.mit.edu> > Most of r_dict tests are in test_objectmodel, though you may want > to have also a look at rpython/test/test_rconstantdict. Ah, ok. > About the distinction between r_dict and "custom dict": do you find > any place where they are used interchangeably? I guess not. I just wasn't clear if they should be usable interchangeably or not. As it happens, I renamed my Java class to CustomDict anyhow, because it was an implementation of the ootype.CustomDict, so it seemed right. > Adding weakrefs to gencli was very simple: I just needed to map > lltypesystem.llmemory.WeakGcAddress to 'System.WeakReference', add > the straightforward support for constants to cli/constant.py and > add the also straightforward 'cast_ptr_to_weakadr' and > 'cast_weakadr_to_ptr' operations in opcodes.py. > I don't know for jvm, but I guess it would not be much more > complicate. Cool, I'll check it out. Niko From grobinson at goombah.com Thu Apr 5 22:03:25 2007 From: grobinson at goombah.com (Gary Robinson) Date: Thu, 5 Apr 2007 16:03:25 -0400 Subject: [pypy-dev] Releasing the GIL in CPython extensions? Message-ID: <20070405160325173009.ceb4f62f@goombah.com> Hello, In PyPy's facilities for writing a CPython extension, would it be conceivable to have an option to release the CPython GIL when the extension is entered and grab it back upon exit? That would enable multiprocessing and could be extremely useful when the programmer knows he's not going to modifying shared items during the execution of the extension. Thanks, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson at goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net From simon at arrowtheory.com Thu Apr 5 22:16:22 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 5 Apr 2007 13:16:22 -0700 Subject: [pypy-dev] Releasing the GIL in CPython extensions? In-Reply-To: <20070405160325173009.ceb4f62f@goombah.com> References: <20070405160325173009.ceb4f62f@goombah.com> Message-ID: <20070405131622.90c68e4f.simon@arrowtheory.com> On Thu, 5 Apr 2007 16:03:25 -0400 Gary Robinson wrote: > > Hello, > > In PyPy's facilities for writing a CPython extension, would it be conceivable to have an option to release the CPython GIL when the extension is entered and grab it back upon exit? It's easy enough to call the PyGIL_ functions manually from rpython.. Simon. From cfbolz at gmx.de Thu Apr 5 22:24:21 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 05 Apr 2007 22:24:21 +0200 Subject: [pypy-dev] Releasing the GIL in CPython extensions? In-Reply-To: <20070405131622.90c68e4f.simon@arrowtheory.com> References: <20070405160325173009.ceb4f62f@goombah.com> <20070405131622.90c68e4f.simon@arrowtheory.com> Message-ID: <46155AF5.6010406@gmx.de> Simon Burton wrote: > On Thu, 5 Apr 2007 16:03:25 -0400 > Gary Robinson wrote: > >> Hello, >> >> In PyPy's facilities for writing a CPython extension, would it be conceivable to have an option to release the CPython GIL when the extension is entered and grab it back upon exit? > > It's easy enough to call the PyGIL_ functions manually from rpython.. That might well be, but that does not release PyPy's GIL when the module is used together with PyPy. Cheers, Carl Friedrich From scott+pypy-dev at scottdial.com Sat Apr 7 00:47:25 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 06 Apr 2007 18:47:25 -0400 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <4603B1E0.1080200@scottdial.com> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> Message-ID: <4616CDFD.2090005@scottdial.com> Scott Dial wrote: > Maciek Fijalkowski wrote: >>> There is a bug somewhere in the translator.js.test suite >>> >> All places should not do this. Hmm... Could you provide more detailed >> info? Thanks for spotting. > > http://scottdial.com/pypy-dev/translator.js.diff > I'm pinging this because it's still messed up and I even provided the fix :-p Thanks, -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From simon at arrowtheory.com Sat Apr 7 20:56:59 2007 From: simon at arrowtheory.com (Simon Burton) Date: Sat, 7 Apr 2007 11:56:59 -0700 Subject: [pypy-dev] translate.py --jit targetpypystandalone: broken? Message-ID: <20070407115659.6354d8d5.simon@arrowtheory.com> this is not working for me: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 287, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 711, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/simon/site-packages/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 260, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/driver.py", line 365, in task_hintannotate_lltype [translation:ERROR] {OriginFlags(): True}) [translation:ERROR] File "/home/simon/local/pypy/pypy/jit/hintannotator/annotator.py", line 62, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, input_args_hs) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 192, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 248, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 477, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 537, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 724, in consider_op [translation:ERROR] raise Exception,"unknown op: %r" % op [translation:ERROR] Exception: unknown op: v6855 = cast_ptr_to_int(i_25) [translation:ERROR] Processing block: [translation:ERROR] block at -1 is a Block at some unknown location [translation:ERROR] containing the following operations: [translation:ERROR] v6856 = cast_pointer(i_25) [translation:ERROR] v6855 = cast_ptr_to_int(i_25) [translation:ERROR] v6857 = cast_int_to_uint(v6855) [translation:ERROR] v6858 = getfield(v6856, ('typeptr')) [translation:ERROR] v6859 = getfield(v6858, ('name')) [translation:ERROR] v6860 = getarraysize(v6859) [translation:ERROR] v6861 = int_sub(v6860, (1)) [translation:ERROR] v6862 = direct_call((<* fn mallocstr__Signed>), v6861) [translation:ERROR] v6863 = getfield((<* struct ExcData { exc_type=..., exc_value=... }>), ('exc_type')) [translation:ERROR] v6864 = ptr_iszero(v6863) [translation:ERROR] --end-- [translation] start debugger... > /home/simon/site-packages/pypy/annotation/annrpython.py(724)consider_op() -> raise Exception,"unknown op: %r" % op $ svn info Path: . URL: https://codespeak.net/svn/pypy/dist/pypy Repository Root: https://codespeak.net/svn Repository UUID: fd0d7bf2-dfb6-0310-8d31-b7ecfe96aada Revision: 41953 Node Kind: directory Schedule: normal Last Changed Author: hpk Last Changed Rev: 41916 Last Changed Date: 2007-04-05 09:29:13 -0700 (Thu, 05 Apr 2007) Properties Last Updated: 2006-10-08 12:39:47 -0700 (Sun, 08 Oct 2006) Simon. From simon at arrowtheory.com Mon Apr 9 19:24:23 2007 From: simon at arrowtheory.com (Simon Burton) Date: Mon, 9 Apr 2007 10:24:23 -0700 Subject: [pypy-dev] translate.py --jit targetpypystandalone: broken? In-Reply-To: <20070407115659.6354d8d5.simon@arrowtheory.com> References: <20070407115659.6354d8d5.simon@arrowtheory.com> Message-ID: <20070409102423.908417ec.simon@arrowtheory.com> On another machine: [translation:ERROR] Error: [translation:ERROR] Traceback (most recent call last): [translation:ERROR] File "./translate.py", line 287, in main [translation:ERROR] drv.proceed(goals) [translation:ERROR] File "/home/simon/site-packages/pypy/translator/driver.py", line 711, in proceed [translation:ERROR] return self._execute(goals, task_skip = self._maybe_skip()) [translation:ERROR] File "/home/simon/site-packages/pypy/translator/tool/taskengine.py", line 112, in _execute [translation:ERROR] res = self._do(goal, taskcallable, *args, **kwds) [translation:ERROR] File "/home/simon/site-packages/pypy/translator/driver.py", line 260, in _do [translation:ERROR] res = func() [translation:ERROR] File "/home/simon/site-packages/pypy/translator/driver.py", line 279, in task_annotate [translation:ERROR] s = annotator.build_types(self.entry_point, self.inputtypes) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 101, in build_types [translation:ERROR] return self.build_graph_types(flowgraph, inputcells, complete_now=complete_now) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 192, in build_graph_types [translation:ERROR] self.complete() [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 248, in complete [translation:ERROR] self.processblock(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 477, in processblock [translation:ERROR] self.flowin(graph, block) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 537, in flowin [translation:ERROR] self.consider_op(block.operations[i]) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 744, in consider_op [translation:ERROR] raise_nicer_exception(op, str(graph)) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/annrpython.py", line 741, in consider_op [translation:ERROR] resultcell = consider_meth(*argcells) [translation:ERROR] File "", line 3, in consider_op_getattr [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/unaryop.py", line 519, in getattr [translation:ERROR] attrdef = ins.classdef.find_attribute(attr) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 218, in find_attribute [translation:ERROR] return self.locate_attribute(attr).attrs[attr] [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 213, in locate_attribute [translation:ERROR] self.generalize_attr(attr) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 322, in generalize_attr [translation:ERROR] self._generalize_attr(attr, s_value) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 303, in _generalize_attr [translation:ERROR] newattr.add_constant_source(origin_classdef, source) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 79, in add_constant_source[translation:ERROR] s_value = source.s_get_value(classdef, self.name) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 435, in s_get_value [translation:ERROR] s_value = self.bookkeeper.immutablevalue(v) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 440, in immutablevalue [translation:ERROR] self.see_mutable(x) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 543, in see_mutable [translation:ERROR] clsdef.add_source_for_attribute(attr, source) # can trigger reflowing [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 183, in add_source_for_attribute [translation:ERROR] attrdef.add_constant_source(self, source) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 79, in add_constant_source[translation:ERROR] s_value = source.s_get_value(classdef, self.name) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 435, in s_get_value [translation:ERROR] s_value = self.bookkeeper.immutablevalue(v) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 368, in immutablevalue [translation:ERROR] result.dictdef.generalize_value(self.immutablevalue(ev)) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 440, in immutablevalue [translation:ERROR] self.see_mutable(x) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 543, in see_mutable [translation:ERROR] clsdef.add_source_for_attribute(attr, source) # can trigger reflowing [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 183, in add_source_for_attribute [translation:ERROR] attrdef.add_constant_source(self, source) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 79, in add_constant_source[translation:ERROR] s_value = source.s_get_value(classdef, self.name) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/classdef.py", line 435, in s_get_value [translation:ERROR] s_value = self.bookkeeper.immutablevalue(v) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 368, in immutablevalue [translation:ERROR] result.dictdef.generalize_value(self.immutablevalue(ev)) [translation:ERROR] File "/home/simon/site-packages/pypy/annotation/bookkeeper.py", line 432, in immutablevalue [translation:ERROR] elif hasattr(x, '_freeze_') and x._freeze_(): [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/mixedmodule.py", line 80, in _freeze_ [translation:ERROR] self.getdict() [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/mixedmodule.py", line 74, in getdict [translation:ERROR] w_value = self.get(name) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/mixedmodule.py", line 34, in get [translation:ERROR] w_value = self.getdictvalue_w(space, name) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/baseobjspace.py", line 24, in getdictvalue_w[translation:ERROR] return self.getdictvalue(space, space.wrap(attr)) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/mixedmodule.py", line 54, in getdictvalue [translation:ERROR] w_value = loader(space) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/mixedmodule.py", line 203, in afileloader [translation:ERROR] return app.wget(space, attrname) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/gateway.py", line 769, in wget [translation:ERROR] w_globals = self.getwdict(space) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/gateway.py", line 760, in getwdict [translation:ERROR] return space.fromcache(ApplevelCache).getorbuild(self) [translation:ERROR] File "/home/simon/site-packages/pypy/tool/cache.py", line 43, in getorbuild [translation:ERROR] result = self._build(key) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/baseobjspace.py", line 145, in _build [translation:ERROR] return self.build(key) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/gateway.py", line 820, in build [translation:ERROR] return PyPyCacheDir.build_applevelinterp_dict(app, self.space) [translation:ERROR] File "/home/simon/site-packages/pypy/interpreter/gateway.py", line 874, in build_applevelinterp_dict [translation:ERROR] self.code, self.filename, self.modname) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1525, in translate_as_module [translation:ERROR] gen.gen_source(tmpname, file=_file) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1010, in gen_source [translation:ERROR] self.gen_source_temp() [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1091, in gen_source_temp [translation:ERROR] self.gen_rpyfunction(func) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1173, in gen_rpyfunction [translation:ERROR] body = list(self.rpyfunction_body(graph, localscope)) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1391, in rpyfunction_body [translation:ERROR] for line in render_block(block): [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 1313, in render_block [translation:ERROR] for line in self.oper(op, localscope).split("\n"): [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 249, in oper [translation:ERROR] exv = self.expr(v, localscope) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 216, in expr [translation:ERROR] debug=('Constant in the graph of', self.currentfunc)) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 407, in nameof [translation:ERROR] name = meth(obj) [translation:ERROR] File "/home/simon/local/pypy/pypy/translator/geninterplevel.py", line 914, in nameof_type [translation:ERROR] self.currentfunc.__name__ or "*no function at all*")) [translation:ERROR] AssertionError': built-in class not found in typename_mapping while compiling _compile [translation:ERROR] .. v4273 = getattr(self_312, ('w_dict')) [translation:ERROR] .. '(pypy.interpreter.mixedmodule:43)MixedModule.getdictvalue' [translation:ERROR] Processing block: [translation:ERROR] block at 3 is a [translation:ERROR] in (pypy.interpreter.mixedmodule:43)MixedModule.getdictvalue [translation:ERROR] containing the following operations: [translation:ERROR] v4274 = getattr(space_0, ('finditem')) [translation:ERROR] v4273 = getattr(self_312, ('w_dict')) [translation:ERROR] v4275 = simple_call(v4274, v4273, w_name_0) [translation:ERROR] v4276 = getattr(self_312, ('lazy')) [translation:ERROR] v4277 = is_true(v4276) [translation:ERROR] --end-- From micahel at gmail.com Mon Apr 9 19:44:45 2007 From: micahel at gmail.com (Michael Hudson) Date: Mon, 9 Apr 2007 18:44:45 +0100 Subject: [pypy-dev] translate.py --jit targetpypystandalone: broken? In-Reply-To: <20070409102423.908417ec.simon@arrowtheory.com> References: <20070407115659.6354d8d5.simon@arrowtheory.com> <20070409102423.908417ec.simon@arrowtheory.com> Message-ID: On 09/04/07, Simon Burton wrote: > > On another machine: > I think the first problem was caused by Armin's change to add the id of objects to their repr, and it should be shallow enough to fix if you know your way around the hintannotator and the backends and so on to add support for cast_ptr_to_int (should be easy, it's basically a no-op at this level after all). The second one looks like one of those random annotation problems we all love so much :( Cheers, mwh From arigo at tunes.org Tue Apr 10 18:18:34 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 Apr 2007 18:18:34 +0200 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <4616CDFD.2090005@scottdial.com> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> Message-ID: <20070410161834.GA21559@code0.codespeak.net> Hi Scott, On Fri, Apr 06, 2007 at 06:47:25PM -0400, Scott Dial wrote: > I'm pinging this because it's still messed up and I even provided the > fix :-p Tentatively checked in, though it's not my area of expertise :-) Armin From arigo at tunes.org Tue Apr 10 18:46:05 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 10 Apr 2007 18:46:05 +0200 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <20070410161834.GA21559@code0.codespeak.net> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> <20070410161834.GA21559@code0.codespeak.net> Message-ID: <20070410164605.GA22993@code0.codespeak.net> Hi Scott, On Tue, Apr 10, 2007 at 06:18:34PM +0200, Armin Rigo wrote: > On Fri, Apr 06, 2007 at 06:47:25PM -0400, Scott Dial wrote: > > I'm pinging this because it's still messed up and I even provided the > > fix :-p > > Tentatively checked in, though it's not my area of expertise :-) Uh, I just realized that many tests fail after the use_pdb argument is removed, because they try to pass one anyway. Could you provide a more complete patch? Thanks! A bientot, Armin From scott+pypy-dev at scottdial.com Tue Apr 10 21:22:27 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Tue, 10 Apr 2007 15:22:27 -0400 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <20070410164605.GA22993@code0.codespeak.net> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> <20070410161834.GA21559@code0.codespeak.net> <20070410164605.GA22993@code0.codespeak.net> Message-ID: <461BE3F3.3090203@scottdial.com> Armin Rigo wrote: >> Tentatively checked in, though it's not my area of expertise :-) > > Uh, I just realized that many tests fail after the use_pdb argument is > removed, because they try to pass one anyway. Could you provide a more > complete patch? Thanks! > Agh, sorry. I didn't do a thorough search for "use_pdb" before. I think I saw the problem in main and updated test_main to match and (poorly) thought that would be the end of that. This explains why Maciek probably has been sitting on it. After having looked at the other places where "use_pdb" is passsed, there are other instances of functions that take a "use_pdb" optional argument which would have to be dealt with as well. I'm not familiar enough with all the ways those functions are called to say how this should be dealt with. With that said, I'm not feeling like it is my place or that I am qualified to be making the kind of sweeping changes that it would require. And for the time being, I'm not being annoyed by this because the underlying bugs that brought up the pygame windows has been fixed. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From fijal at genesilico.pl Wed Apr 11 11:26:34 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Wed, 11 Apr 2007 11:26:34 +0200 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <461BE3F3.3090203@scottdial.com> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> <20070410161834.GA21559@code0.codespeak.net> <20070410164605.GA22993@code0.codespeak.net> <461BE3F3.3090203@scottdial.com> Message-ID: <461CA9CA.4000800@genesilico.pl> Scott Dial wrote: > Armin Rigo wrote: > >>> Tentatively checked in, though it's not my area of expertise :-) >>> >> Uh, I just realized that many tests fail after the use_pdb argument is >> removed, because they try to pass one anyway. Could you provide a more >> complete patch? Thanks! >> >> > > Agh, sorry. I didn't do a thorough search for "use_pdb" before. I think > I saw the problem in main and updated test_main to match and (poorly) > thought that would be the end of that. This explains why Maciek probably > has been sitting on it. > > After having looked at the other places where "use_pdb" is passsed, > there are other instances of functions that take a "use_pdb" optional > argument which would have to be dealt with as well. I'm not familiar > enough with all the ways those functions are called to say how this > should be dealt with. > > With that said, I'm not feeling like it is my place or that I am > qualified to be making the kind of sweeping changes that it would > require. And for the time being, I'm not being annoyed by this because > the underlying bugs that brought up the pygame windows has been fixed. > > -Scott > > Sorry, I was a bit out. Will try to take a look at it in few days. I've tried applying your patch, but this didn't work out, so I've dropped the issue, my fault :-) From grobinson at goombah.com Wed Apr 11 14:07:47 2007 From: grobinson at goombah.com (Gary Robinson) Date: Wed, 11 Apr 2007 08:07:47 -0400 Subject: [pypy-dev] Releasing the GIL in CPython extensions? Message-ID: <20070411080747666981.df029092@goombah.com> >> It's easy enough to call the PyGIL_ functions manually from rpython.. > That might well be, but that does not release PyPy's GIL when the module > is used together with PyPy. OK, but overall it sounds like, if one were to write an CPython extension using RPython programming style so that the critical parts are translated to C, one CAN release the CPython GIL and thereby take advantage of multiprocessors for threading. My understanding from the second comment above is that this extension would not have the GIL-releasing ability when the module is used with PyPy. The docs say that the extension-writing ability is beta and that "bugs and rough edges are likely." Are there any estimates as to when it might be fairly stable? Thanks, Gary -- Gary Robinson CTO Emergent Music, LLC grobinson at goombah.com 207-942-3463 Company: http://www.goombah.com Blog: http://www.garyrobinson.net From paul.degrandis at gmail.com Thu Apr 12 13:59:42 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Thu, 12 Apr 2007 07:59:42 -0400 Subject: [pypy-dev] SoC Acceptance! Message-ID: <9c0bb8a00704120459s30b9b30fw33b02bf49696b96f@mail.gmail.com> Hello everyone! I'm on board for Google Summer of Code (and long after). To refresh everyone's memory I'll be pushing the JVM backend forward and creating JSR-223 bindings to allow you to script PyPy in Java applications. Niko polished off dictionaries, leaving weakrefs to be worked on. Antonio is my mentor, whom I'm very happy and honored to work under. I'm really excited to take this portion of PyPy on. I'll be using my work on the project to advance an autonomic computing project I'm working on and collecting empirical data on PyPy's affect on productivity in a research environment. Results from all my work will be published in research papers. I'll be on IRC and AIM all day tomorrow. Regards, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20070412/25d68335/attachment.htm From tjreedy at udel.edu Thu Apr 12 18:29:55 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Apr 2007 12:29:55 -0400 Subject: [pypy-dev] Forward: Dyla 07 Message-ID: Posted on c.l.p., but possibly more relevant here: ------------- Dear colleges, You might want to consider Dyla'07 as a good venue to present your work and your favourite programming language. Regards, Alexandre ************************************************************************** Call for Papers Dyla 2007: 3rd Workshop on Dynamic Languages and Applications July 31, 2007, Berlin (Collocated with ECOOP 2007) http://dyla2007.unibe.ch ************************************************************************** Objective ========= The goal of this workshop is to act as a forum where we can discuss new advances in the conception, implementation and application of object-oriented languages that radically diverge from the statically typed class-based reflectionless doctrine. The goal of the workshop is to discuss new as well as older "forgotten" languages and features in this context. Topics of interest include, but are certainly not limited to: - agents, actors, active object, distribution, concurrency and mobility - delegation, prototypes, mixins - first-class closures, continuations, environments - reflection and meta-programming - (dynamic) aspects for dynamic languages - higher-order objects & messages - ... other exotic dynamic features which you would categorize as OO - multi-paradigm & static/dynamic-marriages - (concurrent/distributed/mobile/aspect) virtual machines - optimisation of dynamic languages - automated reasoning about dynamic languages - "regular" syntactic schemes (cf. S-expressions, Smalltalk, Self) - Smalltalk, Python, Ruby, Scheme, Lisp, Self, ABCL, Prolog, ... - ... any topic relevant in applying and/or supporting dynamic languages. We solicit high-quality submissions on research results and/or experience. Submitted papers must be unpublished and not submitted for publication elsewhere. Submissions should not exceed 10 pages, LNCS format (www.springer.com/lncs). Submission ========== Prospective attendees are requested to submit a position paper or an essay (max 10 pages, references included) on a topic relevant to the workshop to Alexandre Bergel (Alexandre.Bergel at cs.tcd.ie). Submissions are demanded to be in .pdf format and should arrive before May 13, 2007. A selection of the best papers will be made, which will require an extension for an inclusion in a special issue in Electronic Communications of the EASST (eceasst.cs.tu-berlin.de). For this purpose, a new deadline will be set after the workshop. Moreover, Springer publishes a Workshop-Reader (as in the case of previous ECOOPs) which appears after the Conference and which contains Workshop-Reports (written by the organizers) and not the position papers submitted by the participants. Important dates =============== Submission due: May 13, 2007 Notification of Authors: May 31, 2007 ECOOP'07 Early Registration Date: June 15th. Workshop: July 31, 2007 Organisers ========== Alexandre Bergel Wolfgang De Meuter St?phane Ducasse Oscar Nierstrasz Roel Wuyts Program committee ================= Alexandre Bergel (LERO & Trinity College Dublin, Ireland) Johan Brichau (Universit? catholique de Louvain, Belgium) Pascal Costanza (Vrije Universiteit Brussel, Belgium) Wolfgang De Meuter (Vrije Universiteit Brussel, Belgium) St?phane Ducasse (University of Annecy, France) Erik Ernst (University of Aarhus, Denmark) Robert Hirschfeld (Hasso-Plattner-Institut, University of Potsdam, Germany) Oscar Nierstrasz (University of Bern, Switzerland) Matthew Flatt (University of Utah, USA) Dave Thomas (Bedarra Research Labs, Canada) Laurence Tratt (King's College London, UK) Roel Wuyts (IMEC & Universit? Libre de Bruxelles, Belgium) From holger at merlinux.de Thu Apr 12 19:43:16 2007 From: holger at merlinux.de (holger krekel) Date: Thu, 12 Apr 2007 19:43:16 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: References: Message-ID: <20070412174316.GE18225@solar.trillke> Hi Terry, thanks a lot for the pointer! it's indeed highly interesting and as it's also geographically close to my usual living place i'd like to co-work on submitting a PyPy paper - i think we have more than enough fitting material. holger On Thu, Apr 12, 2007 at 12:29 -0400, Terry Reedy wrote: > Posted on c.l.p., but possibly more relevant here: > ------------- > Dear colleges, > > You might want to consider Dyla'07 as a good venue to present your > work and your favourite programming language. > > Regards, > Alexandre > > > ************************************************************************** > Call for Papers > Dyla 2007: 3rd Workshop on Dynamic Languages and Applications > July 31, 2007, Berlin (Collocated with ECOOP 2007) > > http://dyla2007.unibe.ch > ************************************************************************** > > Objective > ========= > > The goal of this workshop is to act as a forum where we can discuss > new advances in the conception, implementation and application of > object-oriented languages that radically diverge from the statically > typed class-based reflectionless doctrine. The goal of the workshop is > to discuss new as well as older "forgotten" languages and features in > this context. Topics of interest include, but are certainly not > limited to: > > - agents, actors, active object, distribution, concurrency and > mobility > - delegation, prototypes, mixins > - first-class closures, continuations, environments > - reflection and meta-programming > - (dynamic) aspects for dynamic languages > - higher-order objects & messages > - ... other exotic dynamic features which you would categorize as > OO > - multi-paradigm & static/dynamic-marriages > - (concurrent/distributed/mobile/aspect) virtual machines > - optimisation of dynamic languages > - automated reasoning about dynamic languages > - "regular" syntactic schemes (cf. S-expressions, Smalltalk, Self) > - Smalltalk, Python, Ruby, Scheme, Lisp, Self, ABCL, Prolog, ... > - ... any topic relevant in applying and/or supporting dynamic > languages. > > We solicit high-quality submissions on research results > and/or experience. Submitted papers must be unpublished > and not submitted for publication elsewhere. Submissions > should not exceed 10 pages, LNCS format (www.springer.com/lncs). > > > Submission > ========== > > Prospective attendees are requested to submit a position paper or an > essay (max 10 pages, references included) on a topic relevant to the > workshop to Alexandre Bergel (Alexandre.Bergel at cs.tcd.ie). Submissions > are demanded to be in .pdf format and should arrive before May 13, > 2007. > > A selection of the best papers will be made, which will require an > extension for an inclusion in a special issue in Electronic > Communications of the EASST (eceasst.cs.tu-berlin.de). For this > purpose, a new deadline will be set after the workshop. > > Moreover, Springer publishes a Workshop-Reader (as in the case of > previous ECOOPs) which appears after the Conference and which contains > Workshop-Reports (written by the organizers) and not the position > papers submitted by the participants. > > > Important dates > =============== > > Submission due: May 13, 2007 > Notification of Authors: May 31, 2007 > ECOOP'07 Early Registration Date: June 15th. > Workshop: July 31, 2007 > > > Organisers > ========== > > Alexandre Bergel > Wolfgang De Meuter > St?phane Ducasse > Oscar Nierstrasz > Roel Wuyts > > > Program committee > ================= > > Alexandre Bergel (LERO & Trinity College Dublin, Ireland) > Johan Brichau (Universit? catholique de Louvain, Belgium) > Pascal Costanza (Vrije Universiteit Brussel, Belgium) > Wolfgang De Meuter (Vrije Universiteit Brussel, Belgium) > St?phane Ducasse (University of Annecy, France) > Erik Ernst (University of Aarhus, Denmark) > Robert Hirschfeld (Hasso-Plattner-Institut, University of > Potsdam, Germany) > Oscar Nierstrasz (University of Bern, Switzerland) > Matthew Flatt (University of Utah, USA) > Dave Thomas (Bedarra Research Labs, Canada) > Laurence Tratt (King's College London, UK) > Roel Wuyts (IMEC & Universit? Libre de Bruxelles, > Belgium) > > > > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From anto.cuni at gmail.com Fri Apr 13 11:10:08 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 13 Apr 2007 11:10:08 +0200 Subject: [pypy-dev] SoC Acceptance! In-Reply-To: <9c0bb8a00704120459s30b9b30fw33b02bf49696b96f@mail.gmail.com> References: <9c0bb8a00704120459s30b9b30fw33b02bf49696b96f@mail.gmail.com> Message-ID: <461F48F0.9020105@gmail.com> Paul deGrandis wrote: > Hello everyone! hi Paul! > I'm on board for Google Summer of Code (and long after). To refresh > everyone's memory I'll be pushing the JVM backend forward and creating > JSR-223 bindings to allow you to script PyPy in Java applications. Congratulations! I'm very happy that your application has been accepted, I'm sure I'll be able to do a great job. > Niko polished off dictionaries, leaving weakrefs to be worked on. > Antonio is my mentor, whom I'm very happy and honored to work under. I think that before writing code it would be better to have a good understanding of pypy's architecture. How much do you know of PyPy? Next, you should get an account on codespeak to be able to check your code in; have you already got such an account? > I'm really excited to take this portion of PyPy on. I'll be using my > work on the project to advance an autonomic computing project I'm > working on and collecting empirical data on PyPy's affect on > productivity in a research environment. Results from all my work will > be published in research papers. > > I'll be on IRC and AIM all day tomorrow. Today I'll be on IRC until about 16:00 UTC+2. My nick is - surprisingly enough - antocuni :-). ciao Anto From cfbolz at gmx.de Fri Apr 13 13:22:59 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 13 Apr 2007 13:22:59 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: <20070412174316.GE18225@solar.trillke> References: <20070412174316.GE18225@solar.trillke> Message-ID: <461F6813.1020001@gmx.de> holger krekel wrote: > thanks a lot for the pointer! it's indeed highly > interesting and as it's also geographically close > to my usual living place i'd like to co-work on > submitting a PyPy paper - i think we have more > than enough fitting material. fwiw, I was considering to submit my Prolog paper (or a variation of it) there - but we can do some common planning, not sure whether we should submit two PyPy papers. Cheers, Carl Friedrich From arigo at tunes.org Sat Apr 14 11:05:53 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 14 Apr 2007 11:05:53 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: <461F6813.1020001@gmx.de> References: <20070412174316.GE18225@solar.trillke> <461F6813.1020001@gmx.de> Message-ID: <20070414090553.GA20210@code0.codespeak.net> Hi Carl, On Fri, Apr 13, 2007 at 01:22:59PM +0200, Carl Friedrich Bolz wrote: > fwiw, I was considering to submit my Prolog paper (or a variation of it) > there - but we can do some common planning, not sure whether we should > submit two PyPy papers. I think that it's a good place to submit a "core" PyPy paper, and I also think that your Prolog paper is independent enough to not count as another PyPy paper. In fact I'd imagine that if we were "academic enough", at this point, several of us should try to publish several papers independently on several aspects of PyPy... A bientot, Armin. From tjreedy at udel.edu Sun Apr 15 01:21:56 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Apr 2007 19:21:56 -0400 Subject: [pypy-dev] Forward: Dyla 07 References: <20070412174316.GE18225@solar.trillke><461F6813.1020001@gmx.de> <20070414090553.GA20210@code0.codespeak.net> Message-ID: "Armin Rigo" wrote in message news:20070414090553.GA20210 at code0.codespeak.net... | Hi Carl, | | On Fri, Apr 13, 2007 at 01:22:59PM +0200, Carl Friedrich Bolz wrote: | > fwiw, I was considering to submit my Prolog paper (or a variation of it) | > there - but we can do some common planning, not sure whether we should | > submit two PyPy papers. | | I think that it's a good place to submit a "core" PyPy paper, and I also | think that your Prolog paper is independent enough to not count as | another PyPy paper. In fact I'd imagine that if we were "academic | enough", at this point, several of us should try to publish several | papers independently on several aspects of PyPy... For instance, if there is anything innovative in the JIT work, then that might make a nice paper in itself. TJR From amauryfa at gmail.com Mon Apr 16 00:31:32 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Mon, 16 Apr 2007 00:31:32 +0200 Subject: [pypy-dev] Socket module on windows Message-ID: Hello, I finally managed to port the socket module to windows. It is probably incomplete, but at least it translates, and I prove it: > pypy.exe debug: WARNING: library path not found, using compiled-in sys.path Python 2.4.1 (pypy 1.0.0 build 41908) on win32 Type "help", "copyright", "credits" or "license" for more information. >>>> import socket >>>> s = socket.socket() >>>> s.connect(('localhost', 80)) >>>> s.send('GET\n\n') 5 >>>> s.recv(100) '>>> I can send a (rough) patch file to anybody who wants to test it... Since my svn account is still active, I will carefully submit my changes step by step, hoping not to break anything: I do not have access to a Unix machine, so I will closely follow the results of the nightly tests. Of course your comments are most welcome... there were some rough parts and I am sure I took the wrong way sometimes. Cheers, -- Amaury Forgeot d'Arc From fijal at genesilico.pl Mon Apr 16 13:35:57 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 16 Apr 2007 13:35:57 +0200 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <4616CDFD.2090005@scottdial.com> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> Message-ID: <46235F9D.4050600@genesilico.pl> Scott Dial wrote: > Scott Dial wrote: >> Maciek Fijalkowski wrote: >>>> There is a bug somewhere in the translator.js.test suite >>>> >>> All places should not do this. Hmm... Could you provide more >>> detailed info? Thanks for spotting. >> >> http://scottdial.com/pypy-dev/translator.js.diff >> > > I'm pinging this because it's still messed up and I even provided the > fix :-p > > Thanks, > -Scott > Ok, I'm coming back to issue. I didn't find on windows status page any failures related to that :-( Could you point me to some? Also seems that examples doesn't work on windows. Bad. From scott+pypy-dev at scottdial.com Tue Apr 17 03:24:22 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Mon, 16 Apr 2007 21:24:22 -0400 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <46235F9D.4050600@genesilico.pl> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> <46235F9D.4050600@genesilico.pl> Message-ID: <462421C6.6010601@scottdial.com> Maciek Fijalkowski wrote: > Ok, I'm coming back to issue. I didn't find on windows status page any > failures related to that :-( > > Could you point me to some? > Also seems that examples doesn't work on windows. Bad. > I've not tried to actually run the examples myself, so I hadn't realized that. With respect to failures, the underlying failures that would cause the need for PDB to run have been corrected. I think it would be silly to introduce an intentionally erroring test case for this purpose, so you'll have to trust me on the logic being broken. The problem as I see it is that configuration parameters are not being passed around in a consistent way. And the logic of rpython2javascript() is to use whatever "use_pdb" says ignoring the "jsconfig" entirely. It would perhaps make enough sense to say that they both have to be True for the debugger to be run. However, I think there is a larger issue here with the reason "use_pdb" needs to exist at all. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From fijal at genesilico.pl Tue Apr 17 09:54:44 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Tue, 17 Apr 2007 09:54:44 +0200 Subject: [pypy-dev] translator.js.test invokes pygame (AKA, yet more annoyances from Scott) In-Reply-To: <462421C6.6010601@scottdial.com> References: <4603A19C.9090908@scottdial.com> <4603A508.9010004@genesilico.pl> <4603B1E0.1080200@scottdial.com> <4616CDFD.2090005@scottdial.com> <46235F9D.4050600@genesilico.pl> <462421C6.6010601@scottdial.com> Message-ID: <46247D44.3020108@genesilico.pl> Scott Dial wrote: > Maciek Fijalkowski wrote: >> Ok, I'm coming back to issue. I didn't find on windows status page >> any failures related to that :-( >> >> Could you point me to some? >> Also seems that examples doesn't work on windows. Bad. >> > > I've not tried to actually run the examples myself, so I hadn't > realized that. With respect to failures, the underlying failures that > would cause the need for PDB to run have been corrected. I think it > would be silly to introduce an intentionally erroring test case for > this purpose, so you'll have to trust me on the logic being broken. > > The problem as I see it is that configuration parameters are not being > passed around in a consistent way. And the logic of > rpython2javascript() is to use whatever "use_pdb" says ignoring the > "jsconfig" entirely. It would perhaps make enough sense to say that > they both have to be True for the debugger to be run. However, I think > there is a larger issue here with the reason "use_pdb" needs to exist > at all. > > -Scott > Ah, I think I got it (finally!). Ok, the idea was to have use_pdb override value in config, to be sure that in some places this config is not used, even though it defines use_pdb=False. I'll try to fix the logic than to use jsconfig.use_pdb in first place and override it. Thanks, Fijal From cfbolz at gmx.de Wed Apr 18 09:57:09 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 18 Apr 2007 09:57:09 +0200 Subject: [pypy-dev] "A Prolog Interpreter in Python" -- bachelor thesis submitted Message-ID: <4625CF55.6000403@gmx.de> Hi all! Yesterday I (finally) submitted my bachelor thesis about the implementation of a Prolog interpreter in Python. If you are interested in looking at it, you can find it here: http://codespeak.net/pypy/extradoc/paper/prolog-in-python.pdf Cheers, Carl Friedrich From radix at twistedmatrix.com Wed Apr 18 10:37:01 2007 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Wed, 18 Apr 2007 09:37:01 +0100 Subject: [pypy-dev] "A Prolog Interpreter in Python" -- bachelor thesis submitted In-Reply-To: <4625CF55.6000403@gmx.de> References: <4625CF55.6000403@gmx.de> Message-ID: <60ed19d40704180137j3f5fa0eu1aa77befa19d14bc@mail.gmail.com> On 4/18/07, Carl Friedrich Bolz wrote: > Yesterday I (finally) submitted my bachelor thesis about the > implementation of a Prolog interpreter in Python. Congratulations! I hope it's well-accepted. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ From jgustak at gmail.com Wed Apr 18 14:21:45 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Wed, 18 Apr 2007 14:21:45 +0200 Subject: [pypy-dev] scheme interpreter Message-ID: I think, it's about time for me to say hello! I am in the game for Google Summer of Code. I will be working on Scheme interpreter/front-end during this summer. Armin Rigo is my mentor. Right now I am doing more research on the subject and getting familiar with PyPy code base. The main issue right now is: How to start with PyPy? In short time we will have discussion on some basic ideas about the interpreter itself. We will probably use #pypy-scheme on IRC, to not to disturb the regular discussions on PyPy development. It is really great opportunity for me to work with you. Best regards, Jakub Gustak From fijal at genesilico.pl Wed Apr 18 18:59:40 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Wed, 18 Apr 2007 18:59:40 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: Message-ID: <46264E7C.1010006@genesilico.pl> Jakub Gustak wrote: > I think, it's about time for me to say hello! > > I am in the game for Google Summer of Code. I will be working on > Scheme interpreter/front-end during this summer. Armin Rigo is my > mentor. > > Right now I am doing more research on the subject and getting familiar > with PyPy code base. > > The main issue right now is: How to start with PyPy? > http://codespeak.net/pypy/dist/pypy/doc/getting-started.html is a nice place to start. > In short time we will have discussion on some basic ideas about the > interpreter itself. > > We will probably use #pypy-scheme on IRC, to not to disturb the > regular discussions on PyPy development. > I think that using #pypy is a very good place also because there is not too much discussions there right now. I think it makes sense if you sit there and have some overview what's going on. Cheers, fijal From micahel at gmail.com Wed Apr 18 19:08:31 2007 From: micahel at gmail.com (Michael Hudson) Date: Wed, 18 Apr 2007 18:08:31 +0100 Subject: [pypy-dev] scheme interpreter In-Reply-To: <46264E7C.1010006@genesilico.pl> References: <46264E7C.1010006@genesilico.pl> Message-ID: On 18/04/07, Maciek Fijalkowski wrote: > Jakub Gustak wrote: > > We will probably use #pypy-scheme on IRC, to not to disturb the > > regular discussions on PyPy development. > > > I think that using #pypy is a very good place also because there is not > too much discussions there right now. I think it makes sense if you sit > there and have some overview what's going on. I agree. Don't be shy! Cheers, mwh From paul.degrandis at gmail.com Wed Apr 18 20:03:40 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Wed, 18 Apr 2007 14:03:40 -0400 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> Message-ID: <9c0bb8a00704181103u1a3255d6k2d4218c9aab37de7@mail.gmail.com> As fellow SoCer, I'm going to suggest #pypy as well. Everyone is super helpful and real friendly. I've followed the project for over a year, and never used the IRC channel, I regret that now. Paul On 4/18/07, Michael Hudson wrote: > > On 18/04/07, Maciek Fijalkowski wrote: > > Jakub Gustak wrote: > > > We will probably use #pypy-scheme on IRC, to not to disturb the > > > regular discussions on PyPy development. > > > > > I think that using #pypy is a very good place also because there is not > > too much discussions there right now. I think it makes sense if you sit > > there and have some overview what's going on. > > I agree. Don't be shy! > > Cheers, > mwh > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20070418/4165312b/attachment.htm From lac at openend.se Wed Apr 18 20:41:19 2007 From: lac at openend.se (Laura Creighton) Date: Wed, 18 Apr 2007 20:41:19 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: Message from "Paul deGrandis" of "Wed, 18 Apr 2007 14:03:40 EDT." <9c0bb8a00704181103u1a3255d6k2d4218c9aab37de7@mail.gmail.com> References: <46264E7C.1010006@genesilico.pl> <9c0bb8a00704181103u1a3255d6k2d4218c9aab37de7@mail.gmail.com> Message-ID: <200704181841.l3IIfJNS001582@theraft.openend.se> I regularly read the logs of #pypy for when I cannot be around. I learn a lot this way. If the interesting discussion starts happening someplace else, I won't learn as much, which I would consider a shame. Laura From lac at openend.se Fri Apr 20 13:10:10 2007 From: lac at openend.se (Laura Creighton) Date: Fri, 20 Apr 2007 13:10:10 +0200 Subject: [pypy-dev] SciPy conference August 14-18 at Caltech Message-ID: <200704201110.l3KBAAjR031705@theraft.openend.se> Anybody here going? Even if we don't make a pypy presentation, we should probably make a poster/hand out. This conference is a likely source of the kind of people who do the sort of algorithmic computing that we can expect to best optimise. It would bt nice to get some speedup figures from them. Laura From jgustak at gmail.com Fri Apr 20 13:10:29 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 20 Apr 2007 13:10:29 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: <46264E7C.1010006@genesilico.pl> References: <46264E7C.1010006@genesilico.pl> Message-ID: > > Right now I am doing more research on the subject and getting familiar > > with PyPy code base. > > > > The main issue right now is: How to start with PyPy? > > > http://codespeak.net/pypy/dist/pypy/doc/getting-started.html > is a nice place to start. I have already went through that, and many more documentation. Now I am looking for some guidance how to start writing the interpreter itself, which will fit into the PyPy, and won't repeat effort you have already made. I am studying javascript interpreter and some parts of pyrolog. I can see that a lot of work is done by the pypy.rlib.parsing.ebnfparse. I will focus on that right now. Any links, documentation or literature to help me understand this part? Best regards, Jakub Gustak From santagada at gmail.com Fri Apr 20 14:01:05 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 20 Apr 2007 09:01:05 -0300 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> Message-ID: Em 20/04/2007, ?s 08:10, Jakub Gustak escreveu: > I have already went through that, and many more documentation. Now I > am looking for some guidance how to start writing the interpreter > itself, which will fit into the PyPy, and won't repeat effort you have > already made. Basically a pypy interpreter is a python program that can be translated using pypy (which means, it uses rpython and only rpython routines). You can start defining a vm for the language in python trying to remember the limitations of rpython and after a little people can help you to get it rpythonic enough to be translated. to do the translation you will need something like pypy/translator/goal/ targetjsstandalone.py which you can mostly copy (We copied someone elses goal). You can also think about the parser for scheme, as I am doing right now for javascript, and for that you can use rlib/parsing but that is up to you. > > I am studying javascript interpreter and some parts of pyrolog. > I can see that a lot of work is done by the > pypy.rlib.parsing.ebnfparse. > I will focus on that right now. Any links, documentation or literature > to help me understand this part? > pypy.rlib.parsing is the parser generator we are using... it is not that important for the beginning of your work now. Good luck, and you can catch me at irc (my nickname is santagada) > Best regards, > Jakub Gustak > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev -- Leonardo Santagada santagada at gmail.com From cfbolz at gmx.de Fri Apr 20 14:10:59 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 20 Apr 2007 12:10:59 +0000 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> Message-ID: <348899050704200510s6ce9a925q85dfbc6a7ef4ba51@mail.gmail.com> Hi Jakub! Welcome on board! I will try to co-mentor you a bit during the summer. 2007/4/20, Jakub Gustak : > > > Right now I am doing more research on the subject and getting familiar > > > with PyPy code base. > > > > > > The main issue right now is: How to start with PyPy? > > > > > http://codespeak.net/pypy/dist/pypy/doc/getting-started.html > > is a nice place to start. > > I have already went through that, and many more documentation. Now I > am looking for some guidance how to start writing the interpreter > itself, which will fit into the PyPy, and won't repeat effort you have > already made. To get started with development, you should get a codespeak account. Send a mail to Michael Hudson ( micahel at gmail.com ) with your name, the user name you want and your public ssh key. You should agree on using the MIT license for your code, which is what the rest of PyPy uses. Also you should try to follow the coding guide: http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html The Scheme interpreter should probably be put into the pypy/lang/scheme directory. In general you should write test before you start coding using the py.test testing framework: http://codespeak.net/py/dist/test.html You should check in fairly often so that we can see what you're working on and how you are doing and help you if you run into problems. > I am studying javascript interpreter and some parts of pyrolog. > I can see that a lot of work is done by the pypy.rlib.parsing.ebnfparse. > I will focus on that right now. Any links, documentation or literature > to help me understand this part? There are some docs about the parser generator here: http://codespeak.net/pypy/dist/pypy/doc/rlib.html If you have any questions about it you can ask me (on IRC or here), I wrote most of it. It is most certainly not finished, if you run into problems we can think about how to fix it. I guess by default Scheme is not that hard to parse (no clue what happens if macros and other evil things enter the picture). Cheers, Carl Friedrich From cfbolz at gmx.de Fri Apr 20 14:17:44 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 20 Apr 2007 12:17:44 +0000 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> Message-ID: <348899050704200517t79ad7bcai63ab4a586367d3f1@mail.gmail.com> 2007/4/20, Leonardo Santagada : > > Em 20/04/2007, ?s 08:10, Jakub Gustak escreveu: > > I have already went through that, and many more documentation. Now I > > am looking for some guidance how to start writing the interpreter > > itself, which will fit into the PyPy, and won't repeat effort you have > > already made. > > Basically a pypy interpreter is a python program that can be > translated using pypy (which means, it uses rpython and only rpython > routines). You can start defining a vm for the language in python > trying to remember the limitations of rpython and after a little > people can help you to get it rpythonic enough to be translated. to > do the translation you will need something like pypy/translator/goal/ > targetjsstandalone.py which you can mostly copy (We copied someone > elses goal). You can also think about the parser for scheme, as I am > doing right now for javascript, and for that you can use rlib/parsing > but that is up to you. I think you should not try very hard to to get the interpreter to be RPython in the beginning. Just try to use not too many highly dynamic features and try to concentrate on getting the interpreter do the right thing. > > I am studying javascript interpreter and some parts of pyrolog. > > I can see that a lot of work is done by the > > pypy.rlib.parsing.ebnfparse. > > I will focus on that right now. Any links, documentation or literature > > to help me understand this part? > > > pypy.rlib.parsing is the parser generator we are using... it is not > that important for the beginning of your work now. For scheme it should be easy enough to get a simple but usable parser. Cheers, Carl Friedrich From fijal at genesilico.pl Fri Apr 20 15:47:32 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 20 Apr 2007 15:47:32 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> Message-ID: <4628C474.4060403@genesilico.pl> Jakub Gustak wrote: >> > Right now I am doing more research on the subject and getting familiar >> > with PyPy code base. >> > >> > The main issue right now is: How to start with PyPy? >> > >> http://codespeak.net/pypy/dist/pypy/doc/getting-started.html >> is a nice place to start. > > I have already went through that, and many more documentation. Now I > am looking for some guidance how to start writing the interpreter > itself, which will fit into the PyPy, and won't repeat effort you have > already made. > > I am studying javascript interpreter and some parts of pyrolog. > I can see that a lot of work is done by the pypy.rlib.parsing.ebnfparse. > I will focus on that right now. Any links, documentation or literature > to help me understand this part? > > Best regards, > Jakub Gustak > > :ms We can try to organize some kind of mini-sprint in Poland if you like to introduce you. As far as I know you're living in Wroclaw, I'm passing through Wroclaw once in a while, so we can arrange one-two days long coding session if you find it suitable for you. Cheers, fijal From jgustak at gmail.com Fri Apr 20 17:47:29 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 20 Apr 2007 17:47:29 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: <4628C474.4060403@genesilico.pl> References: <46264E7C.1010006@genesilico.pl> <4628C474.4060403@genesilico.pl> Message-ID: > We can try to organize some kind of mini-sprint in Poland if you like to > introduce you. As far as I know you're living in Wroclaw, I'm passing > through Wroclaw once in a while, so we can arrange one-two days long > coding session if you find it suitable for you. Well that sounds fun! I (my association) can organize a place to work at my university with network and some desks. We definitively can do this. And by the way, my association is organizing "The 4th Linux Session". It is planned on 26th of May. It is, despite the name, generally about open software, etc. I am planning to give an introduction talk about PyPy. But maybe one of you would like to take over the talk, as long as I am a newbie. Cheers, Jakub Gustak From florian.schulze at gmx.net Fri Apr 20 16:46:06 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Fri, 20 Apr 2007 16:46:06 +0200 Subject: [pypy-dev] Javascript parser Message-ID: Hi! I read that some people here may work on a javascript parser. I'm also working on one, so I would be really interested in collaboration. Especially with people who have more experience with compiler and interpreter needs. I only have the need up to an AST and generating javascript from that again. I just need some transformations which may get more complicated though. Just contact me, Florian Schulze From lalo.martins at gmail.com Sun Apr 22 08:43:06 2007 From: lalo.martins at gmail.com (Lalo Martins) Date: Sun, 22 Apr 2007 06:43:06 +0000 (UTC) Subject: [pypy-dev] Embedding pypy Message-ID: Here's an interesting question... how hard would it be to write a C API for embedding the pypy VM in a C or C+ + application? I have an application for which I'm working on Python scripting, but for many reasons it would be interesting to use pypy instead. best, Lalo Martins -- So many of our dreams at first seem impossible, then they seem improbable, and then, when we summon the will, they soon become inevitable. ----- personal: http://lalo.hystericalraisins.net/ technical: http://www.hystericalraisins.net/ GNU: never give up freedom http://www.gnu.org/ From lac at openend.se Sun Apr 22 10:07:02 2007 From: lac at openend.se (Laura Creighton) Date: Sun, 22 Apr 2007 10:07:02 +0200 Subject: [pypy-dev] Should we decide to have another seminar in Gbg ... Message-ID: <200704220807.l3M872SH006577@theraft.openend.se> we should remember to invite this person: http://www.cs.lth.se/home/Calle_Lejdfors/ posted here, so we don't lose it. (Plus you might want to read his papers.) Laura From micahel at gmail.com Sun Apr 22 11:57:40 2007 From: micahel at gmail.com (Michael Hudson) Date: Sun, 22 Apr 2007 10:57:40 +0100 Subject: [pypy-dev] Embedding pypy In-Reply-To: References: Message-ID: On 22/04/07, Lalo Martins wrote: > Here's an interesting question... > > how hard would it be to write a C API for embedding the pypy VM in a C or C+ > + application? I have an application for which I'm working on Python > scripting, but for many reasons it would be interesting to use pypy instead. Well, to do it at all would require attacking a small pile of details: teaching the tool chain to build a library rather than an application, supporting multiple entry points (though I guess this bit is somewhat solved by the extcompiler and the js backend, in different ways), ... nothing too impossible though. The fun part would be the integration between the application and the interpreter. If you're happy with just passing strings back and forth (equivalent to Py_RunSimpleString or whatever it's called) that's probably quite easy, but also not very interesting. To provide app-specific modules to Python, to allow Python to call your app which then calls Python again... I dunno, but this sort of thing would probably be quite a lot of work. Cheers, mwh From mauriceling at gmail.com Sun Apr 22 13:07:08 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Sun, 22 Apr 2007 21:07:08 +1000 Subject: [pypy-dev] Who's developing language interpreters in PyPy Message-ID: <462B41DC.4090802@acm.org> Hi, I've been reading up on pypy and is really interested in the idea of implementing X language interpreters in pypy, where X can be any known programming language. Do we have a knowledge of who is developing or interested in which X-to-pypy interpreter so there's no competition nor duplication of work? Personally I have some interests in a few languages that I know, such as R and pascal. Cheers maurice From cfbolz at gmx.de Sun Apr 22 15:16:52 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 22 Apr 2007 15:16:52 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462B41DC.4090802@acm.org> References: <462B41DC.4090802@acm.org> Message-ID: <348899050704220616g36248a72ua7453479a68dc596@mail.gmail.com> Hi Maurice! 2007/4/22, Maurice Ling : > I've been reading up on pypy and is really interested in the idea of > implementing X language interpreters in pypy, where X can be any known > programming language. > > Do we have a knowledge of who is developing or interested in which > X-to-pypy interpreter so there's no competition nor duplication of work? Short overview: The following interpreters are in the PyPy repository currently (or will be, in the case of Scheme): - Python (various people) - Prolog (me) - JavaScript (Leonardo Santagada) - Scheme (Jakub Gustak, not yet started) In addition, Christopher Armstrong is working on "SafeLisp": https://code.launchpad.net/~subol-hackers/subol/safelisp-rpython > Personally I have some interests in a few languages that I know, such as > R and pascal. I don't really know either of them very well, but I imagine that the problem with R would be that you would have to implement all sorts of numerical functions (which is a lot of effort) or interface to existing C libraries (which is not much fun, since the rctypes are a mess, currently). Cheers, Carl Friedrich From santagada at gmail.com Sun Apr 22 16:04:56 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Sun, 22 Apr 2007 11:04:56 -0300 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462B41DC.4090802@acm.org> References: <462B41DC.4090802@acm.org> Message-ID: <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> Em 22/04/2007, ?s 08:07, Maurice Ling escreveu: > Hi, > > I've been reading up on pypy and is really interested in the idea of > implementing X language interpreters in pypy, where X can be any known > programming language. > Yep it is... but probably more interesting to interpreters... > Do we have a knowledge of who is developing or interested in which > X-to-pypy interpreter so there's no competition nor duplication of > work? > > Personally I have some interests in a few languages that I know, > such as > R and pascal. > Pascal should be easier to do, but you have to tell us exactly what you want. If you are planning to do a pascal compiler it would be the first compiler in pypy. If you are thinking on a pascal interpreter this could be quite easy to do but then you need to have some problem with current pascal compilers that you want to fix. I think this could be used on teaching compilers, as understanding a pascal compiler written in python would be much more comprehensible than a C or pascal one. -- Leonardo Santagada santagada at gmail.com From mauriceling at gmail.com Mon Apr 23 02:24:40 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Mon, 23 Apr 2007 10:24:40 +1000 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> References: <462B41DC.4090802@acm.org> <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> Message-ID: <462BFCC8.60100@acm.org> Thanks Leonardo >> >> Personally I have some interests in a few languages that I know, >> such as >> R and pascal. >> > Pascal should be easier to do, but you have to tell us exactly what > you want. If you are planning to do a pascal compiler it would be the > first compiler in pypy. If you are thinking on a pascal interpreter > this could be quite easy to do but then you need to have some problem > with current pascal compilers that you want to fix. I think this > could be used on teaching compilers, as understanding a pascal > compiler written in python would be much more comprehensible than a C > or pascal one. > I don't actually use Pascal these days but it's just the first programming language I learnt. So it's more like a childhood dish kinda thing, just refuse to let it go completely although I do agree with you that a Pascal compiler written in Python will be more comprehensible than one written in C or Pascal. And it may just seems oxymoronic to think about implementing a C compiler in PyPy. My current langauge kit is Python/C/Java/R/SQL/UML. So I presume there's a bit more interest in R than in pascal and implementing a R interpreter in python will reduce the need for RPy in many cases (I think). Otherwise, I may go on a total tangent and try working on modeling/simulation languages like Modelica/SBML combination or even a UML simulator in RPython, which may be more useful to my future employment. So yes, I'm kinda polling interests here... Any suggestions? Thanks Cheers maurice From santagada at gmail.com Mon Apr 23 02:41:44 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Sun, 22 Apr 2007 21:41:44 -0300 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462BFCC8.60100@acm.org> References: <462B41DC.4090802@acm.org> <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> <462BFCC8.60100@acm.org> Message-ID: Em 22/04/2007, ?s 21:24, Maurice Ling escreveu: > Thanks Leonardo You're welcome > So yes, I'm kinda polling interests here... Any suggestions? If you make an R interpreter using pypy we would probably still need RPy to use R inside python. What you could do is implement Numeric (I don't know how it is called this days, maybe numpy or numarray) in pypy, that probably helps to make python able to do the same kind of work as R is used to today... and maybe even faster :) Just remember implementing Numeric on pypy is going to be kind of hard... in exchange you will probably get your hack skillz up to a demi god level :) Good luck, -- Leonardo Santagada santagada at gmail.com From faassen at startifact.com Mon Apr 23 11:04:26 2007 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 23 Apr 2007 11:04:26 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: References: <462B41DC.4090802@acm.org> <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> <462BFCC8.60100@acm.org> Message-ID: Leonardo Santagada wrote: > Em 22/04/2007, ?s 21:24, Maurice Ling escreveu: > >> Thanks Leonardo > You're welcome > >> So yes, I'm kinda polling interests here... Any suggestions? > > If you make an R interpreter using pypy we would probably still need > RPy to use R inside python. What you could do is implement Numeric (I > don't know how it is called this days, maybe numpy or numarray) in > pypy, Numpy appears to be the official merger between Numeric and Numarray. In general you'd hope PyPy will eventually grow ways to reuse important bits of C-based Python infrastructure for the implementation of non-Python language. Or can it already do so through rctypes? It sounds a bit odd to use Numeric for the implementation of R that way, but it'd make sense to be able to do so, right? Naturally PyPy wouldn't be able to use its magical optimization abilities on those bits, though. Regards, Martijn From arigo at tunes.org Mon Apr 23 17:56:54 2007 From: arigo at tunes.org (Armin Rigo) Date: Mon, 23 Apr 2007 17:56:54 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462B41DC.4090802@acm.org> References: <462B41DC.4090802@acm.org> Message-ID: <20070423155654.GA20273@code0.codespeak.net> Hi Maurice, On Sun, Apr 22, 2007 at 09:07:08PM +1000, Maurice Ling wrote: > I've been reading up on pypy and is really interested in the idea of > implementing X language interpreters in pypy, where X can be any known > programming language. In general, the more interesting languages to try to implement with PyPy are dynamic languages. Of course it's possible, and maybe even nice, to write an interpreter for Pascal. It's just that there is a bias towards dynamic languages, both in what RPython supports and in what the translation toolchain can do with it. Well, you can always try to build a dynamically extensible Pascal environment with automatic memory management :-) Which reminds me to mention that a Smalltalk environment would be a nice project in PyPy. A bientot, Armin. From fijal at genesilico.pl Mon Apr 23 20:04:56 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 23 Apr 2007 20:04:56 +0200 Subject: [pypy-dev] scheme interpreter In-Reply-To: References: <46264E7C.1010006@genesilico.pl> <4628C474.4060403@genesilico.pl> Message-ID: <462CF548.9020902@genesilico.pl> Jakub Gustak wrote: >> We can try to organize some kind of mini-sprint in Poland if you like to >> introduce you. As far as I know you're living in Wroclaw, I'm passing >> through Wroclaw once in a while, so we can arrange one-two days long >> coding session if you find it suitable for you. > > Well that sounds fun! > > I (my association) can organize a place to work at my university with > network and some desks. We definitively can do this. > > And by the way, my association is organizing "The 4th Linux Session". > It is planned on 26th of May. It is, despite the name, generally about > open software, etc. I am planning to give an introduction talk about > PyPy. But maybe one of you would like to take over the talk, as long > as I am a newbie. > > Cheers, > Jakub Gustak > > :ms I don't think that actually giving PyPy talk to people who don't know python makes sense. Actually PyPy is not very good first python. Instead, a general python talk would be more productive (and way better understood). From tjreedy at udel.edu Mon Apr 23 21:13:36 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 23 Apr 2007 15:13:36 -0400 Subject: [pypy-dev] Who's developing language interpreters in PyPy References: <462B41DC.4090802@acm.org> <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> <462BFCC8.60100@acm.org> Message-ID: "Martijn Faassen" wrote in message news:f0hsqn$d4b$1 at sea.gmane.org... |Numpy appears to be the official merger between Numeric and Numarray. *is*, insofar as anything is 'official'. |It sounds a bit odd to use Numeric for the implementation of R that way, |but it'd make sense to be able to do so, right? 'R' is both a (statistics/data analisis) language and an interpreter system. (I have not used it but read some some time ago.) The latter, I presume, is implemented in some combination of C and Fortran, as are NumPy and SciPy. I would not be surprised if the R system used some of the same libraries (BLAS, LinPack, FFTPack, etc). So, if the array concepts are compatible, it make perfect sense to me to base an alternate implemention of R the language on NumPy and the R functions that are already in SciPy. The motivation would either be a) proof of concept of PyPy as a general platform for implementing dynamic languages or b) an implementation with value-added for users (free (if R is not yet), open-source (ditto), access to other Python facilities). | Naturally PyPy wouldn't be able to use its magical optimization abilities on those bits Many to most of 'those bits' have been heavily optimized already, which is why to use them. Terry Jan Reedy From simon at arrowtheory.com Mon Apr 23 21:40:11 2007 From: simon at arrowtheory.com (Simon Burton) Date: Mon, 23 Apr 2007 12:40:11 -0700 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: References: <462B41DC.4090802@acm.org> <9BC16641-1752-43D7-A0DB-68EB08CC4BA6@gmail.com> <462BFCC8.60100@acm.org> Message-ID: <20070423124011.a0c2caa2.simon@arrowtheory.com> On Mon, 23 Apr 2007 15:13:36 -0400 "Terry Reedy" wrote: > > | Naturally PyPy wouldn't be able to use its magical optimization abilities > on those bits > > Many to most of 'those bits' have been heavily optimized already, which is > why to use them. > > Terry Jan Reedy Yes, but how about this idea: that we would like to take vector expressions, such as the sum of three arrays, X+Y+Z, and instead of creating a temporary array to house X+Y (and then summing with Z), we sum the elements from all three arrays at once. This kind of calculation is very typical in numpy/R, where temporary arrays are created, transversed then destroyed alot, and the CPU cache overheats, and people become generally unhappy. Last year I had a look at teaching the rtyper to understand numpy expressions (see pypy.rpython.numpy), with the idea of heading in this more intelligent direction. This year i discovered the pypy JIT and had a brain orgasm; it would be very interesting to use (+develop) the JIT towards these cache friendly ideas also. It really makes no sense to reimplement R, i think, UNLESS you can interface with the present R libraries. The big strength of R is it's many many libraries for statistics etc. I don't think it's much of an interesting language in and of itself. Simon. From santagada at gmail.com Tue Apr 24 02:16:19 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 23 Apr 2007 21:16:19 -0300 Subject: [pypy-dev] New Javascript parser in the works Message-ID: I started development of the javascript parser, based on the pythongrammar.txt file on parsing test dir. The thing is still in its infancy, but I just wanted to say that I am working on it, and would love any help. It is living in users/santagada/newjsparser for now... will move to js when it is at least kind of working. About the parsing lib I have some questions (should be asking cfbolz but I don't seem to find him on irc lately): * ParsingError should print something like lineno: self.lineno+1 collum: self.collum+1 (I know there is a pretty error function, but still) * IGNORE and friends, how do I make then work? * I don't know why when it recognizes something (or I think it should) it just gives me list index out of bounds (I will check what is happening, but if someone know why it would be better). About the grammar: * based on python grammar, so automatic semicolon insertion is OFF. You have optional semicolons at the end of lines though * No regex literals (ahh someday maybe, i'm not really interested in them) * Lot's of work to be done still :) -- Leonardo Santagada santagada at gmail.com From mauriceling at gmail.com Tue Apr 24 02:47:32 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Tue, 24 Apr 2007 10:47:32 +1000 Subject: [pypy-dev] Motivation for re-implementing R (GNU S) in PyPy (perhaps?) Message-ID: <462D53A4.9000708@acm.org> Well, R is free and open sourced. It is based on S and Splus, which the developers decided to close-source after a while (after achieving success). R is also known as GNU S for that matter. IMHO, R itself is a simple language and a relatively simple platform. It is a language that manipulates data but philosophically, it is not a statistical or mathematical language. The R core team seems to adhere to the view that R is a programming language where (by chance and intent) a lot of statistical tools are built on. Hence, most of the statistical libraries for R are written in R. I believe that the entire Bioconductor is written in R. Hence, the motivation for considering implementing or re-implementing R in PyPy is "natively" to bring in the strength of R (huge amounts of statistical libraries and data sets) into Python. I have to admit that my skills in this area (langauage interpreters/compilers) is rather limited to undergraduate level but I'm willing to learn more. So I'm currently reading codes on language interpreters for esoteric languages like BrainFuck simply because they are tiny but provides the relevant concepts. So, anyone like to support the idea of (re-)implementing R on PyPy? I'll need some "emotional" strength to go on this route. Also willing to hear other radical options... Another thing of interest to me is implementing modeling/simulation languages in PyPy as it will help in my future career route but my knowledge of even the simulation languages are rather limited. Please advice and comment. Cheers maurice From santagada at gmail.com Tue Apr 24 04:36:00 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Mon, 23 Apr 2007 23:36:00 -0300 Subject: [pypy-dev] Motivation for re-implementing R (GNU S) in PyPy (perhaps?) In-Reply-To: <462D53A4.9000708@acm.org> References: <462D53A4.9000708@acm.org> Message-ID: <2A1FF07E-73B3-48CA-AB9D-1BBE9B8CEDCF@gmail.com> Em 23/04/2007, ?s 21:47, Maurice Ling escreveu: > Well, R is free and open sourced. It is based on S and Splus, which > the > developers decided to close-source after a while (after achieving > success). R is also known as GNU S for that matter. > > IMHO, R itself is a simple language and a relatively simple > platform. It > is a language that manipulates data but philosophically, it is not a > statistical or mathematical language. The R core team seems to > adhere to > the view that R is a programming language where (by chance and > intent) a > lot of statistical tools are built on. Hence, most of the statistical > libraries for R are written in R. I believe that the entire > Bioconductor > is written in R. > > Hence, the motivation for considering implementing or re- > implementing R > in PyPy is "natively" to bring in the strength of R (huge amounts of > statistical libraries and data sets) into Python. So, if the language is open source, quite simple and the only advantage would be to have huge amounts of libraries... Why not just make a code conversion from R to Numpy + Python, a full featured language, used in a lot more places. Then you can get the libraries and also a bigger comunity. Just remember that making an interpreter of R using pypy will not bring anything to python, as they would be diferent interpreters that just happen to be inplemented on the same platform. It would keep the same distance as R as it is today (implemented in C I suppose) and Python (also made in C if you consider CPython). So I would probably say, do the Numpy work on pypy than make a translator R2Py so you can run your favorite R library on top of your optimized Numpy implementation. > > I have to admit that my skills in this area (langauage > interpreters/compilers) is rather limited to undergraduate level > but I'm > willing to learn more. So I'm currently reading codes on language > interpreters for esoteric languages like BrainFuck simply because they > are tiny but provides the relevant concepts. Mine to, but there is a lot of really brainy people on the project... and they really help (specially if you could go to some sprint) > So, anyone like to support the idea of (re-)implementing R on PyPy? > I'll > need some "emotional" strength to go on this route. Also willing to > hear > other radical options... > > Another thing of interest to me is implementing modeling/simulation > languages in PyPy as it will help in my future career route but my > knowledge of even the simulation languages are rather limited. Making Numpy work as we are saying (making it really optimized and cache friendly) is probably going to help your career even more. > Please advice and comment. > My two cents. -- Leonardo Santagada santagada at gmail.com From mauriceling at gmail.com Tue Apr 24 05:48:26 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Tue, 24 Apr 2007 13:48:26 +1000 Subject: [pypy-dev] Motivation for re-implementing R (GNU S) in PyPy (perhaps?) In-Reply-To: <2A1FF07E-73B3-48CA-AB9D-1BBE9B8CEDCF@gmail.com> References: <462D53A4.9000708@acm.org> <2A1FF07E-73B3-48CA-AB9D-1BBE9B8CEDCF@gmail.com> Message-ID: <462D7E0A.2090209@acm.org> >> Well, R is free and open sourced. It is based on S and Splus, which the >> developers decided to close-source after a while (after achieving >> success). R is also known as GNU S for that matter. >> >> IMHO, R itself is a simple language and a relatively simple >> platform. It >> is a language that manipulates data but philosophically, it is not a >> statistical or mathematical language. The R core team seems to >> adhere to >> the view that R is a programming language where (by chance and >> intent) a >> lot of statistical tools are built on. Hence, most of the statistical >> libraries for R are written in R. I believe that the entire >> Bioconductor >> is written in R. >> >> Hence, the motivation for considering implementing or re- implementing R >> in PyPy is "natively" to bring in the strength of R (huge amounts of >> statistical libraries and data sets) into Python. > > > So, if the language is open source, quite simple and the only > advantage would be to have huge amounts of libraries... Why not just > make a code conversion from R to Numpy + Python, a full featured > language, used in a lot more places. Then you can get the libraries > and also a bigger comunity. This could be a nice idea - make R codes run on Numpy + Python. My line of thought is that since there is an implementation of Numpy (I do not know how complete it is) in PyPy already (pypy/rpython/numpy), implementing a means to get R codes to run in RPython will be just as equally good, since RPython are essentially complete Python programs. Kinda killing 2 birds with a stone, or so. > > Just remember that making an interpreter of R using pypy will not > bring anything to python, as they would be diferent interpreters that > just happen to be inplemented on the same platform. It would keep the > same distance as R as it is today (implemented in C I suppose) and > Python (also made in C if you consider CPython). So I would probably > say, do the Numpy work on pypy than make a translator R2Py so you can > run your favorite R library on top of your optimized Numpy > implementation. I think this line of argument can be made against every language interpreter implemented in PyPy, essentially everything can be interfaced at C. IMHO, what PyPy offers is the language-optimization-platform modularization. This may be loafy goal but sure worth the effort. I am not sure what making an interpreter of X (where X can be any language, such as javascript or prolog) using PyPy will really bring anything to Python than implementing X interpreter directly on Python itself (without using PyPy). Please enlighten me on this. Cheers maurice From rschmitt at medcom-online.de Tue Apr 24 08:52:30 2007 From: rschmitt at medcom-online.de (Robert Schmitt) Date: Tue, 24 Apr 2007 08:52:30 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy Message-ID: <462DA92E.2010103@medcom-online.de> > Well, you can always try to build a dynamically extensible Pascal > environment with automatic memory management :-) Pascal? A nice university language for its time, but the type system makes practical use an absurd attempt (cf http://www.cs.virginia.edu/~evans/cs655-S00/readings/bwk-on-pascal.html) > Which reminds me to mention that a Smalltalk environment would be a nice > project in PyPy. What about a C/C++ interpreter/compiler, there would be some real benefit if people could transpile C code to python. Plus, there's an existing C interpreter project out there, maybe there is a way to reuse some parts or at least some ideas of it. (This is probably too big a project for one person, but if there's any interest in this I'd be happy to join :-). From cfbolz at gmx.de Tue Apr 24 10:58:19 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 24 Apr 2007 10:58:19 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462DA92E.2010103@medcom-online.de> References: <462DA92E.2010103@medcom-online.de> Message-ID: <462DC6AB.8070400@gmx.de> Robert Schmitt wrote: > What about a C/C++ interpreter/compiler, there would be some real > benefit if people could transpile C code to python. To be honest, I don't see how a C-to-Python-compiler a) relates to PyPy and b) why it would be useful. What real benefit do you see there? Interfacing to C/C++ code is not that hard in CPython, and PyPy is getting there too. Such a compiler would also be quite hard, since C/C++ allow all sorts of crazy unsafe things, so you would end up doing something like described in the paper "Complete Translation of Unsafe Native Code to Safe Bytecode" by Brian Alliet and Adam Megacz. > Plus, there's an > existing C interpreter project out there, maybe there is a way to > reuse some parts or at least some ideas of it. You mean CINT? Quite crashy, in my experience. Which does not make it a less impressive (but very very crazy) thing :-) From cfbolz at gmx.de Tue Apr 24 10:58:38 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 24 Apr 2007 10:58:38 +0200 Subject: [pypy-dev] New Javascript parser in the works In-Reply-To: References: Message-ID: <462DC6BE.3050207@gmx.de> Hi Leonardo! Leonardo Santagada wrote: > I started development of the javascript parser, based on the > pythongrammar.txt file on parsing test dir. This is the point that does not really make sense to me already. You should base your grammar on the official (Mozilla) one, otherwise you will end up chasing and understanding strange corner cases. Also it makes updating the grammar much easier for future versions of the language. If you don't want to (or can't, for some reason) use the official grammar directly, you should use at least parts of it. > The thing is still in its > infancy, but I just wanted to say that I am working on it, and would > love any help. It is living in users/santagada/newjsparser for now... > will move to js when it is at least kind of working. I looked quickly at the directory, my comment is that you should really write tests. Otherwise you will get regressions when you change the grammar. As you know, PyPy is very strict about untested code, and an untested grammar is not really a good plan either. Also, the tests would have told me what you expect to work, in the current state it is completely worthless to me since I have no clue what should work and what not. > About the parsing lib I have some questions (should be asking cfbolz > but I don't seem to find him on irc lately): Sorry, I am enjoying my free time too much currently and don't want to be online consistently. Mail is fine for such a technical discussion anyway (and it has the additional benefit that I can answer you during this boring lecture I am sitting in now). > * ParsingError should print something like lineno: self.lineno+1 > collum: self.collum+1 (I know there is a pretty error function, but > still) There is no class called ParsingErro and ParseError (which exists) does not have a lineno attribute, so what exactly do you mean? If it is only a cosmetic change, just fix it yourself, I will look at it. Note that the line and column numbers of the parse errors are guesses only, there is no way to pinpoint the exact error location with PEGs. > * IGNORE and friends, how do I make then work? What friends? IGNORE itself is a bit documented, so read the docs (and the tests). The idea is that IGNORE is a regular expression of things that should be ignored in the input (probably comments and whitespace). > * I don't know why when it recognizes something (or I think it > should) it just gives me list index out of bounds (I will check what > is happening, but if someone know why it would be better). Check in a failing test (skipped or not, it's your directory anyway), otherwise I have no clue what you are talking about. > About the grammar: > * based on python grammar, so automatic semicolon insertion is OFF. > You have optional semicolons at the end of lines though > * No regex literals (ahh someday maybe, i'm not really interested in > them) Regex literals will only be possible after refactoring the parser generator, I fear. > * Lot's of work to be done still :) Yip. Cheers, Carl Friedrich From rschmitt at medcom-online.de Tue Apr 24 11:29:59 2007 From: rschmitt at medcom-online.de (Robert Schmitt) Date: Tue, 24 Apr 2007 11:29:59 +0200 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462DC6AB.8070400@gmx.de> References: <462DA92E.2010103@medcom-online.de> <462DC6AB.8070400@gmx.de> Message-ID: <462DCE17.1010005@medcom-online.de> Carl Friedrich Bolz schrieb: > To be honest, I don't see how a C-to-Python-compiler a) relates to PyPy > and b) why it would be useful. What real benefit do you see there? Such a module would not have to be part of pypy, but it could make quite good use of the machinery already in place. > Interfacing to C/C++ code is not that hard in CPython, and PyPy is > getting there too. Such a compiler would also be quite hard, since C/C++ > allow all sorts of crazy unsafe things, so you would end up doing > something like described in the paper "Complete Translation of Unsafe > Native Code to Safe Bytecode" by Brian Alliet and Adam Megacz. There certainly is a way to access python from C/C++ and vice versa, so that is not my concern. But going one step further would be really beneficial: Getting legacy C code to a platform where it can be used, test-infected, and refactored, that would be cool. On the other hand, I totally agree with you that it would be quite a crazy endeavor - then again, what's life without a challenge :-) Taking the approach of the paper would probably be easier than translating the source, but I don't see a benefit unless you use the symbolic intermediate data generated by the compiler so you can generate readable code in the end - or do I misunderstand the pypy architecture, can't you generate rpython output with it? From arigo at tunes.org Tue Apr 24 20:47:01 2007 From: arigo at tunes.org (arigo at tunes.org) Date: Tue, 24 Apr 2007 11:47:01 -0700 Subject: [pypy-dev] EuroPython 2007 - call for refereed papers Message-ID: <20070424184701.GA2267@bespin.org> Hi all, He're a reminder to submit a talk at EuroPython! Like each year, we have both the regular conference (see call at http://indico.cern.ch/conferenceCFA.py?confId=13919) and a somewhat separated Refereed Papers section. Here is the call for the latter. The deadline for both is the 18th of May. ======================================================================== EuroPython 2007 Vilnius, Lithuania 9-11 July Call for Refereed Papers http://www.europython.org/ ======================================================================== EuroPython is the only conference in the Python world that has a properly prestigious peer-reviewed forum for presenting technical and scientific papers. Such papers, with advanced and highly innovative contents, can equally well stem from academic research or industrial research. We think this is an important function for EuroPython, so we are even making some grants available to help people with travel costs. We will be happy to consider papers in subject areas including, but not necessarily limited to, the following: * Python language and implementations * Python modules (in the broadest sense) * Python extensions * Interoperation between Python and other languages / subsystems * Scientific applications of Python * Python in Education * Games * Agile Methodologies and Testing * Social Skills We are looking for Python-related scientific and technical papers of advanced, highly innovative content that present the results of original research (be it of the academic or "industrial research" kind), with proper attention to "state of the art" and previous relevant literature/results (whether such relevant previous literature is itself directly related to Python or not). We do not intend to let the specific subject area block a paper's acceptance, as long as the paper satisfies other requirements: innovative, Python-related, reflecting original research, with proper attention to previous literature. Abstracts ========= Please submit abstracts of no more than 200 words to the refereeing committee. You can send submissions no later than 18 May 2007. We shall inform you whether your paper has been selected and announce the conference schedule on the 25 May 2007. For all details regarding the submission of abstracts, please see the EuroPython website (http://www.europython.org). WARNING: Independently of their topic, all abstracts must be submitted *in the Refereed Papers track* in order to be considered by the refereeing committee! If your abstract is accepted, you must submit your corresponding paper before 29 June 2006. Last-minute changes will be accepted until the start of the conference. You should submit the paper as a PDF file, in A4 format, complete, "stand-alone", and readable on any standards-compliant PDF reader (basically, the paper must include all fonts and figures it uses, rather than using external pointers to them; by default, most PDF-preparation programs typically produce such valid "stand-alone" PDF documents). There are no strict typesetting rules. Refereeing ========== The refereeing committee, selected by Armin Rigo, will examine all abstracts and papers. The committee may consult external experts as it deems fit. Referees may suggest or require certain changes and editing in submissions, and make acceptance conditional on such changes being performed. We expect all papers to reflect the abstract as approved and reserve the right, at our discretion, to reject a paper, despite having accepted the corresponding abstract, if the paper does not substantially correspond to the approved abstract. Presentation ============ The paper must be presented at EuroPython by one or more of the authors. Presentation time will be between half an hour and an hour, including time for questions and answers, depending on each paper's details, and also on the total number of talks approved for presentation. Proceedings =========== We will publish the conference's proceedings in purely electronic form. By presenting a paper, authors agree to give the EuroPython conference non-exclusive rights to publish the paper in electronic forms (including, but not limited to, partial and total publication on web sites and/or such media as CDROM and DVD-ROM), and warrant that the papers are not infringing on the rights of any third parties. Authors retain all other intellectual property rights on their submitted abstracts and papers excepting only this non-exclusive license. Subsidised travel ================= We have funds available to subsidise travel costs for some presenters who would otherwise not be able to attend EuroPython. When submitting your abstract, please indicate if you would need such a subsidy as a precondition of being able to come and present your paper. (Yes, this possibility does exist even if you are coming from outside of Europe. Papers from people in New Zealand who can only come if their travel is subsidised, for example, would be just fine with us...). -+- Armin Rigo From simon at arrowtheory.com Wed Apr 25 00:45:20 2007 From: simon at arrowtheory.com (Simon Burton) Date: Tue, 24 Apr 2007 15:45:20 -0700 Subject: [pypy-dev] Motivation for re-implementing R (GNU S) in PyPy (perhaps?) In-Reply-To: <462D7E0A.2090209@acm.org> References: <462D53A4.9000708@acm.org> <2A1FF07E-73B3-48CA-AB9D-1BBE9B8CEDCF@gmail.com> <462D7E0A.2090209@acm.org> Message-ID: <20070424154520.32faf1ca.simon@arrowtheory.com> On Tue, 24 Apr 2007 13:48:26 +1000 Maurice Ling wrote: > > This could be a nice idea - make R codes run on Numpy + Python. My line > of thought is that since there is an implementation of Numpy (I do not > know how complete it is) in PyPy already (pypy/rpython/numpy), In terms of completeness, we barely scratched the surface. Simon. From simon at arrowtheory.com Wed Apr 25 00:50:29 2007 From: simon at arrowtheory.com (Simon Burton) Date: Tue, 24 Apr 2007 15:50:29 -0700 Subject: [pypy-dev] Who's developing language interpreters in PyPy In-Reply-To: <462DC6AB.8070400@gmx.de> References: <462DA92E.2010103@medcom-online.de> <462DC6AB.8070400@gmx.de> Message-ID: <20070424155029.e785a214.simon@arrowtheory.com> On Tue, 24 Apr 2007 10:58:19 +0200 Carl Friedrich Bolz wrote: > > Robert Schmitt wrote: > > What about a C/C++ interpreter/compiler, there would be some real > > benefit if people could transpile C code to python. > > To be honest, I don't see how a C-to-Python-compiler a) relates to PyPy > and b) why it would be useful. What real benefit do you see there? Being able to run optimisations that include external C libraries. This is what the llvm guys can do with the gcc front end, i believe. hmm, perhaps it then makes sense to build a llvm frontend :) Simon. From santagada at gmail.com Wed Apr 25 05:22:24 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 25 Apr 2007 00:22:24 -0300 Subject: [pypy-dev] New Javascript parser in the works In-Reply-To: <462DC6BE.3050207@gmx.de> References: <462DC6BE.3050207@gmx.de> Message-ID: <8F8CD746-5748-45C9-BCEA-A22776438E87@gmail.com> Em 24/04/2007, ?s 05:58, Carl Friedrich Bolz escreveu: > Hi Leonardo! > > Leonardo Santagada wrote: >> I started development of the javascript parser, based on the >> pythongrammar.txt file on parsing test dir. > > This is the point that does not really make sense to me already. You > should base your grammar on the official (Mozilla) one, otherwise you > will end up chasing and understanding strange corner cases. Also it > makes updating the grammar much easier for future versions of the > language. If you don't want to (or can't, for some reason) use the > official grammar directly, you should use at least parts of it. I've tried to start, converted the number literals, but somehow they are not working, can you try it and help me to understand how do I find and fix this kind of bugs (i'm a fast learner, antonio showed me how to find then when translating rpython code and now I can find and fix bugs miself). > >> The thing is still in its >> infancy, but I just wanted to say that I am working on it, and would >> love any help. It is living in users/santagada/newjsparser for now... >> will move to js when it is at least kind of working. > > I looked quickly at the directory, my comment is that you should > really > write tests. Otherwise you will get regressions when you change the > grammar. As you know, PyPy is very strict about untested code, and an > untested grammar is not really a good plan either. Also, the tests > would > have told me what you expect to work, in the current state it is > completely worthless to me since I have no clue what should work and > what not. > Done man, now I will be writing tons of tests, I just need to get the first ones to work. They are in test_parser.py >> About the parsing lib I have some questions (should be asking cfbolz >> but I don't seem to find him on irc lately): > > Sorry, I am enjoying my free time too much currently and don't want to > be online consistently. Mail is fine for such a technical discussion > anyway (and it has the additional benefit that I can answer you during > this boring lecture I am sitting in now). perfect. > >> * ParsingError should print something like lineno: self.lineno+1 >> collum: self.collum+1 (I know there is a pretty error function, but >> still) > > There is no class called ParsingErro and ParseError (which exists) > does > not have a lineno attribute, so what exactly do you mean? If it is > only > a cosmetic change, just fix it yourself, I will look at it. Note that > the line and column numbers of the parse errors are guesses only, > there > is no way to pinpoint the exact error location with PEGs. > It has a sourcepos and that has a lineno... but it is all working ok on test_parse.py >> * IGNORE and friends, how do I make then work? > > What friends? IGNORE itself is a bit documented, so read the docs (and > the tests). The idea is that IGNORE is a regular expression of things > that should be ignored in the input (probably comments and > whitespace). > Read the documentation, I think that there is not much point in separating pure regexes from the other actions (at least if I understand correctly in real PEGs there isn't such separations). But I am living with then, and man great docs! >> * I don't know why when it recognizes something (or I think it >> should) it just gives me list index out of bounds (I will check what >> is happening, but if someone know why it would be better). > > Check in a failing test (skipped or not, it's your directory anyway), > otherwise I have no clue what you are talking about. > There is some failing tests now >> About the grammar: >> * based on python grammar, so automatic semicolon insertion is OFF. >> You have optional semicolons at the end of lines though >> * No regex literals (ahh someday maybe, i'm not really interested in >> them) > > Regex literals will only be possible after refactoring the parser > generator, I fear. > I don't like then anyway so thats ok. >> * Lot's of work to be done still :) > > Yip. > > Cheers, > > Carl Friedrich > Thanks and thanks in advance, -- Leonardo Santagada santagada at gmail.com From cfbolz at gmx.de Fri Apr 27 21:36:25 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 27 Apr 2007 21:36:25 +0200 Subject: [pypy-dev] New Javascript parser in the works In-Reply-To: <8F8CD746-5748-45C9-BCEA-A22776438E87@gmail.com> References: <462DC6BE.3050207@gmx.de> <8F8CD746-5748-45C9-BCEA-A22776438E87@gmail.com> Message-ID: <463250B9.4080607@gmx.de> Hi Leonardo! Leonardo Santagada wrote: > Done man, now I will be writing tons of tests, I just need to get the > first ones to work. They are in test_parser.py I just looked at the current state of affairs, and while the grammar seems to be getting more complete, there is still only a couple of tests, and those that are there are failing. Ok, it's your time but I can't say I like this approach. Just looking at the grammar made me note the following problem: Something like the following is wrong: additiveexpression : multiplicativeexpression "+" additiveexpression | multiplicativeexpression "-" additiveexpression | multiplicativeexpression ; The problem is that something like "5 - 3 - 3" with this grammar will give you a parse tree that essentially looks like the following: additiveexpression /|\ / | \ 5 - additiveexpression /|\ / | \ 3 - 3 And evaluating this tree gives 5, instead of the correct -1. Unfortunately solving this is not trivial. In theory the grammar rule should look like this: additiveexpression : additiveexpression "+" multiplicativeexpression | additiveexpression "-" multiplicativeexpression | multiplicativeexpression ; which does not work, since then you have a left recursion, which is not supported. In theory there could be a nice grammar transformer that removes the left recursion for you (which is simple and always possible) together with a nice tree transformer that makes the tree look correctly (which is not so simple). Until then, you can use the following workaround: additiveexpression : multiplicativeexpression "+" >additiveexpression< | multiplicativeexpression "- >additiveexpression< | ; which will lead to a wrong tree originally, but the tranformer will tranform the tree into this: additiveexpression / | | | \ / | | | \ 5 - 3 - 3 which is not as nice, since additiveexpression has more children than only three but at least it's easy to find out what is correct. You have to make sure then that when you transform the tree into the nodes you use for interpretation that you do the right thing then. Oh, another small thing: would you mind not using tabs in the grammar? I know it is not a python file, but somehow the number of spaces per tab is different between your and my editor, so it makes it a bit annoying for me to look at. Do you have local changes? Otherwise I could do a tab-removing checkin. Cheers, Carl Friedrich From cfbolz at gmx.de Fri Apr 27 22:11:27 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 27 Apr 2007 22:11:27 +0200 Subject: [pypy-dev] New Javascript parser in the works In-Reply-To: <463250B9.4080607@gmx.de> References: <462DC6BE.3050207@gmx.de> <8F8CD746-5748-45C9-BCEA-A22776438E87@gmail.com> <463250B9.4080607@gmx.de> Message-ID: <463258EF.5030303@gmx.de> Carl Friedrich Bolz wrote: > Just looking at the grammar made me note the following problem: [snip] hm, now that I found this problem, which grammar are you using exactly? The one at http://www.mozilla.org/js/language/js20/formal/parser-grammar.html gives the rules for addition correctly: AdditiveExpression ==> MultiplicativeExpression | AdditiveExpression + MultiplicativeExpression | AdditiveExpression - MultiplicativeExpression or did you just rewrite it incorrectly? Cheers, Carl Friedrich From santagada at gmail.com Sat Apr 28 02:50:38 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Fri, 27 Apr 2007 21:50:38 -0300 Subject: [pypy-dev] Fwd: New Javascript parser in the works References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> Message-ID: Forgot to send to pypy-dev. And now I commited code that was supposed to do what I wanted with the tests in revision 42384. But the code is ugly, my visitor somehow don't visit everyone it should and I am to sleepy to code. Am I doing things the wrong way or doing what I wanted with the tests that hard? Iniciar mensagem reenviada: > De: Leonardo Santagada > Data: 27 de abril de 2007 19h24min15s GMT-03:00 > Para: Carl Friedrich Bolz > Assunto: Re: [pypy-dev] New Javascript parser in the works > > > Em 27/04/2007, ?s 17:11, Carl Friedrich Bolz escreveu: > >> Carl Friedrich Bolz wrote: >>> Just looking at the grammar made me note the following problem: >> [snip] >> >> hm, now that I found this problem, which grammar are you using >> exactly? The one at >> > the one in the standard, as you sugestd to me... it is at: > http://www.ecma-international.org/publications/standards/Ecma-262.htm > >> http://www.mozilla.org/js/language/js20/formal/parser-grammar.html >> >> gives the rules for addition correctly: >> >> AdditiveExpression ==> >> MultiplicativeExpression >> | AdditiveExpression + MultiplicativeExpression >> | AdditiveExpression - MultiplicativeExpression >> >> or did you just rewrite it incorrectly? >> > > I both rewritten it incorrectly and also there is a problem, if I > put MultiplicativeExpression first like on the standard and in the > mozilla one I get code like 5+4 interpreted as having one > multiplicative expression with 5 and somehow it doesn't consume all > the input and still thinks it is valid. > > The other problem was that I didn't know how to fix it, so I just > did what I could expecting that with some code people would help me > more (this one I got right :). The biggest problem with parsing is > that there is no one using it besides you... at least you help me a > lot so i'm not really complaning. > > About tests I'm having this problem, I want to create generator > tests like some that I have but I would like to do something like > having lots of lines of tests, maybe with comments and maybe > setting what would be the start simbol to try to match it. Also I > would like to be able compare the results doing tests like: > checking the number of a type of node on the tree (eg: 5+5 should > have 2 numeric literals). The perfect thing would be able to print > how the trace of the packratparser, seeing what rules it is > executing. But I really want to have a better testing suite, but > like just failing some test for a reason I do not know or passing a > test but doing things completely wrong is not sufficient for me > now... I will work to make this test tool. > > Now about semicolons, how should I deal with them? in the spec the > grammar doesn't deal with them and in the mozilla one I don't see > how they are doing it also. As we have set that as the parsing > module works today it is not possible to do automatic semicolon > insertion, can we do "forced semicolon presence" as seen on C and > Java? (some lightbulb just lightened up here, maybe I should look > for the grammar of any of those two languages) > > > >> Cheers, >> >> Carl Friedrich > > -- > Leonardo Santagada > santagada at gmail.com > > > -- Leonardo Santagada santagada at gmail.com From anto.cuni at gmail.com Sat Apr 28 14:02:36 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sat, 28 Apr 2007 14:02:36 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> Message-ID: <463337DC.2080503@gmail.com> Hi Niko, hi Paul! Here are some remarks/suggestions about your discussion. I'm also CC-ing pypy-dev, so we can also catch someone else's suggestions, if any :-). > First, there are those things are absolutely necessary to do to get PyPy > to translate: > > 1. weak refs > 2. I don't know what else goes here --- try and see I guess Apart from weak refs, I think the other big feature missing is support for external functions (mostly I/O functions). Unfortunately this part is a bit of a mess: since in the old days pypy's only targets were low-level platforms the I/O model of RPython is modeled after the unix file descriptors. This means that at the moment high level backends need to emulate the fd interface and forward the real work to the native I/O functions: to see how gencli does, look at the ll_os class in translator/cli/src/ll_os.cs. I guess this would be the easiest way also for genjvm, and probably for now we should pick this solution. I know, this it's both ugly and hackish; the good news is that I and Carl did some work towards a better solution: at the moment all the I/O inside the standard interpreter is done using the (interp-level) rlib/streamio.py library, which in turns uses the low-level file descriptor interface; the long-term solution would be to provide an alternative implementation of streamio.py that uses .NET/Java streams and let the backends using that instead of the current one. Finally, let me add another task that I think it's very important: 3. make most of existing tests passing with genjvm Look at cli/test directory: you will find a lot of files that import tests from somewhere else (mostly from rpython/test). genjvm do the same, but a lot of tests are still missing; once you get all those tests passing you can be quite confident (but still not sure, of course) that pypy would translate. Some of the tests in cli/tests are cli-specific or very old and not conforming to the new test framework; here is a list of the tests you should port these tests to jvm: - test_range.py - test_constant.py - test_tuple.py - test_float.py - test_builtin.py - test_int.py - test_exception.py Moreover, there is test_runtest, which test a bit of the gencli test framework itself; if the genjvm test framework is similar to the gencli's one, it could make sense to port this also this one. Niko, what do you think? > New features: > > 1. allow Java code to be invoked (need to look at the CLI work here) > 2. JSR 223 I have to admit that I don't know very much about JSR-223: Paul, could you send me the specification, please? I don't feel like to register to sun's website :-). If I understand correctly the level of integration provided by JSR-223 is simpler/less powerful than the one offered by Jython, right? If this is the case, it could make sense to implement JSR-223 first (also because it's part of Paul's SoC proposal), then try to provide the same level of integration as Jython (or IronPython for the .NET side); for the latter task, I think that some code could be shared between gencli and genjvm. > Internal Changes That "Would Be Nice": > > 1. Estimate the maximum size of the stack, rather than giving some > arbitrary "high" number as we do know. this would also be useful for gencli :-) > 2. Deal with JVM size limits for very large methods and classes. (never > hit those yet but probably will) probably this falls into the "try to translate pypy-jvm and see" task :-). > 3. Find a better way to handle exceptions. One suggestion was not to > map rpython's "Object" to the class Object but rather to an interface. Indeed. Also useful for gencli. > 4. Use protected access modifiers rather than public. Pedronis seems to > think this would be important. I don't see how it could be so important at this point. Am I missing something? > 5. Speed and performance optimizations, similar to those being done for C# let me add another point: 6. once we get pypy-jvm, run Python's regression tests and try to pass as much of them as possible. I personally think that point 6 is more important than point 5 at the beginning. > Of these ideas, I am not sure which are best for a "first project". I > would be happy to handle weakrefs, since it seems like they may be quite > easy for me but time consuming for you, but if you wanted to look into > them, that's fine with me too. Adding support for weakrefs to gencli was a very easy task: just wrap/unwrap the objects into the System.WeakReference when requested. > Antonio, you said you tried to translate PyPy as a whole with the JVM > backend? It seems that is the best way to find the list of things that > need doing, but I haven't done it myself yet. Yes, I tried: after a bit of hacking on the interpreter code I could get rid of weakrefs, but the translations failed on a KeyError inside jvm/database.py, then I gave up. To sum up, I think that priority wise the most important thing to do is to port the remaining tests to genjvm and make them passing: weakrefs and external functions are subtasks of this; then try-fail-fix-try until we get a running pypy-jvm. ciao Anto From fijal at genesilico.pl Sat Apr 28 14:35:07 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 28 Apr 2007 14:35:07 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <463337DC.2080503@gmail.com> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> Message-ID: <46333F7B.4020104@genesilico.pl> Also a note from my side: Would be cool to have config option to turn off weakrefs (and some functionallity provided by them, like subclasses). I trimmed (same as Antonio) weakref from the interpreter and shouldn't be hard and maybe will allow to translate whole pypy on js backend (well, not really sure, but let's see) Cheers, fijal From florian.schulze at gmx.net Sat Apr 28 14:47:19 2007 From: florian.schulze at gmx.net (Florian Schulze) Date: Sat, 28 Apr 2007 14:47:19 +0200 Subject: [pypy-dev] Fwd: New Javascript parser in the works References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> Message-ID: On Sat, 28 Apr 2007 02:50:38 +0200, Leonardo Santagada wrote: >> Now about semicolons, how should I deal with them? in the spec the >> grammar doesn't deal with them and in the mozilla one I don't see >> how they are doing it also. As we have set that as the parsing >> module works today it is not possible to do automatic semicolon >> insertion, can we do "forced semicolon presence" as seen on C and >> Java? (some lightbulb just lightened up here, maybe I should look >> for the grammar of any of those two languages) This is the biggest and hardest problem about js parsing. The spec does define how to handle it, I'm not sure now how the grammar reflects that though. Looking at C doesn't help, because there it always needs to be present and can't be replaced with newlines. The problem with a js parser is, that newlines aren't really whitespace, just like in python. But the rules are weird, because newlines are only sometimes relevant, not everytime. A js parser which doesn't handle this correctly is in my opinion just wrong. You couldn't parse any real world javascript with it. Regards, Florian Schulze From cfbolz at gmx.de Sat Apr 28 18:22:50 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 28 Apr 2007 18:22:50 +0200 Subject: [pypy-dev] Fwd: New Javascript parser in the works In-Reply-To: References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> Message-ID: <463374DA.6000007@gmx.de> Florian Schulze wrote: > On Sat, 28 Apr 2007 02:50:38 +0200, Leonardo Santagada > wrote: > > >>> Now about semicolons, how should I deal with them? in the spec the >>> grammar doesn't deal with them and in the mozilla one I don't see >>> how they are doing it also. As we have set that as the parsing >>> module works today it is not possible to do automatic semicolon >>> insertion, can we do "forced semicolon presence" as seen on C and >>> Java? (some lightbulb just lightened up here, maybe I should look >>> for the grammar of any of those two languages) > > This is the biggest and hardest problem about js parsing. The spec does > define how to handle it, I'm not sure now how the grammar reflects that > though. Looking at C doesn't help, because there it always needs to be > present and can't be replaced with newlines. The problem with a js parser > is, that newlines aren't really whitespace, just like in python. But the > rules are weird, because newlines are only sometimes relevant, not > everytime. A js parser which doesn't handle this correctly is in my > opinion just wrong. You couldn't parse any real world javascript with it. The part of the spec that describes the automatic semicolon insertion is completely silly, in my opinion. It goes something like this: When, as the program is parsed from left to right, a token (called the offending token) is encountered that is not allowed by any production of the grammar, then a semicolon is automatically inserted before the offending token if one or more of the following conditions is true: ... This is completely crazy, because it effectively forces you to write a parser using a left-to-right parsing technique and also a parser that works by doing exactly one token lookahead. The second part is what makes packrat parsing fail, since it uses arbitrary many tokens lookahead, so you cannot really determine what an "offending token" is since you cannot distinguish it from normal backtracking. I don't see how you can fix that, really. Now you have basically two choices: you can change force all semicolons to be inserted, which makes most code out there not parse. The other one is more brainstormy-like, it does not work as I describe but maybe someone has an idea to get it to work: you could change the grammar to be very lenient with semicolons (at least for a packrat parser this might be easy) meaning that it will programs as valid that existing Javascript engines will reject. Something like this: a = b c = d would be valid. This opens its own set of problems such as: a = b ++ c Which would most likely be parsed to be equivalent to: a = b++; c; Whereas with the spec it is: a = b; ++c; No clue how to fix that, yet. Cheers, Carl Friedrich From cfbolz at gmx.de Sat Apr 28 18:41:58 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 28 Apr 2007 18:41:58 +0200 Subject: [pypy-dev] Fwd: New Javascript parser in the works In-Reply-To: References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> Message-ID: <46337956.6020409@gmx.de> Florian Schulze wrote: > On Sat, 28 Apr 2007 02:50:38 +0200, Leonardo Santagada > wrote: > > >>> Now about semicolons, how should I deal with them? in the spec the >>> grammar doesn't deal with them and in the mozilla one I don't see >>> how they are doing it also. As we have set that as the parsing >>> module works today it is not possible to do automatic semicolon >>> insertion, can we do "forced semicolon presence" as seen on C and >>> Java? (some lightbulb just lightened up here, maybe I should look >>> for the grammar of any of those two languages) > > This is the biggest and hardest problem about js parsing. The spec does > define how to handle it, I'm not sure now how the grammar reflects that > though. Looking at C doesn't help, because there it always needs to be > present and can't be replaced with newlines. The problem with a js parser > is, that newlines aren't really whitespace, just like in python. But the > rules are weird, because newlines are only sometimes relevant, not > everytime. A js parser which doesn't handle this correctly is in my > opinion just wrong. You couldn't parse any real world javascript with it. The part of the spec that describes the automatic semicolon insertion is completely silly, in my opinion. It goes like this: When, as the program is parsed from left to right, a token (called the offending token) is encountered that is not allowed by any production of the grammar, then a semicolon is automatically inserted before the offending token if one or more of the following conditions is true: ... This is completely crazy, because it effectively forces you to write a parser using a left-to-right parsing technique with exactly one token lookahead, by hand (because a lot of parsing frameworks don't allow you to do the customization that is necessary). The lookahead part is what makes packrat parsing fail, since it uses arbitrary many tokens lookahead, so you cannot really determine what an "offending token" is since you cannot distinguish it from normal backtracking. I don't see how you can fix that, really. Now you have basically two choices: you can force the user to insert all semicolons, which makes most code out there not parse. The other one is more brainstormy-like, it does not work as I describe but maybe someone has an idea to get it to work: you could change the grammar to be very lenient with semicolons (at least for a packrat parser this might be easy) meaning that it will programs as valid that existing Javascript engines will reject. Something like this: a = b c = d would be valid. This opens its own set of problems such as: a = b ++ c Which would most likely be parsed to be equivalent to: a = b++; c; Whereas with the spec it is: a = b; ++c; No clue how to fix that, yet. Maybe you could do something similar to what Python does, inserting newline tokens into the token stream and removing those between matched pairs of parenthesis. Cheers, Carl Friedrich From lac at openend.se Sat Apr 28 18:59:53 2007 From: lac at openend.se (Laura Creighton) Date: Sat, 28 Apr 2007 18:59:53 +0200 Subject: [pypy-dev] Fwd: New Javascript parser in the works In-Reply-To: Message from Carl Friedrich Bolz of "Sat, 28 Apr 2007 18:22:50 +0200." <463374DA.6000007@gmx.de> References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> <463374DA.6000007@gmx.de> Message-ID: <200704281659.l3SGxrCt008735@theraft.openend.se> In a message of Sat, 28 Apr 2007 18:22:50 +0200, Carl Friedrich Bolz writes: >would be valid. This opens its own set of problems such as: > > a = b > ++ c > >Which would most likely be parsed to be equivalent to: > > a = b++; > c; > >Whereas with the spec it is: > > a = b; > ++c; > >No clue how to fix that, yet. > >Cheers, > >Carl Friedrich In the Canadian military for an analagous set of problems we did the following: First of all you do the laxness move -- i.e. you get rid of the semicolon restictions. Then you get a big a sample space of the problem you have as you are willing to deal with. In some way determine that it is 'typical' -- which is a research topic in itself, but if you are not in the field, then the rule is 'make a good guess, refine it later when you are wrong'. Then determine where it is better to stick in the semi based on how the language is actually used. So, should the spec differ from what you do, that is actually of no problem until you find that people really write code like the above and want the spec, not the way you parsed it. You would be surprised at how many of those problems go away because no human ever writes things like that. Should your problem space be dominated by machine-generated code, though, your personal ideas of 'how often will somebody write this' goes out the window. Laura, who thinks that the W3C validation project may be interested in what you are doing and have already got some problematic use cases to worry about. From santagada at gmail.com Sat Apr 28 16:08:46 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Sat, 28 Apr 2007 11:08:46 -0300 Subject: [pypy-dev] Fwd: New Javascript parser in the works In-Reply-To: References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> Message-ID: Em 28/04/2007, ?s 09:47, Florian Schulze escreveu: > > This is the biggest and hardest problem about js parsing. The spec > does > define how to handle it, I'm not sure now how the grammar reflects > that > though. Looking at C doesn't help, because there it always needs to be > present and can't be replaced with newlines. The problem with a js > parser > is, that newlines aren't really whitespace, just like in python. > But the > rules are weird, because newlines are only sometimes relevant, not > everytime. A js parser which doesn't handle this correctly is in my > opinion just wrong. You couldn't parse any real world javascript > with it internet exlporer doens't do ASI (automatic semicolon insertion for now own I will call it ASI) properly so for you account it is wrong... If I remember currently all modern javascript frameworks use ES (explicit semicolon) because of diferences in real world parsers. What we can do is provide a tool to put the ; on your legacy code, but that is something that people should be doing anyway so it is not a biggie for me. So I am going with ES for now... when cfbolz and I get some time together to work on making it work then we can try to be exactly to the spec. And the formal grammar doesn't reflect this at all, besides being wrong (there is an errata in the mozilla site that fixes it a bit) and have some parts missing, it completely ignores ASI and the present it as textual information (a very very informal specification). I'm going to focus on making a real test suite for my js parser... if it doesn't drive me crazy I will be back here. > . > > Regards, > Florian Schulze -- Leonardo Santagada santagada at gmail.com From cfbolz at gmx.de Sat Apr 28 19:25:43 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 28 Apr 2007 19:25:43 +0200 Subject: [pypy-dev] Fwd: New Javascript parser in the works In-Reply-To: <200704281659.l3SGxrCt008735@theraft.openend.se> References: <00AC2FE8-875D-419E-A36C-758B38895C28@gmail.com> <463374DA.6000007@gmx.de> <200704281659.l3SGxrCt008735@theraft.openend.se> Message-ID: <46338397.6080402@gmx.de> Laura Creighton wrote: > In a message of Sat, 28 Apr 2007 18:22:50 +0200, Carl Friedrich Bolz writes: >> would be valid. This opens its own set of problems such as: >> >> a = b >> ++ c >> >> Which would most likely be parsed to be equivalent to: >> >> a = b++; >> c; >> >> Whereas with the spec it is: >> >> a = b; >> ++c; >> >> No clue how to fix that, yet. >> >> Cheers, >> >> Carl Friedrich > > > In the Canadian military for an analagous set of problems we did the > following: > > First of all you do the laxness move -- i.e. you get rid of the > semicolon restictions. Then you get a big a sample space of the > problem you have as you are willing to deal with. In some way > determine that it is 'typical' -- which is a research topic in itself, > but if you are not in the field, then the rule is 'make a good guess, > refine it later when you are wrong'. Then determine where it is > better to stick in the semi based on how the language is actually > used. So, should the spec differ from what you do, that is actually > of no problem until you find that people really write code like the > above and want the spec, not the way you parsed it. > Very good points, all. Of course the real plan is to find a behavior that works in most cases and is easy to parse. Afterwards we make our interpreter very successful and people will be forced to comply with the behavior because they want to be compatible, thereby training the JavaScript programmers to write better code :-). > You would be surprised at how many of those problems go away because > no human ever writes things like that. Should your problem space > be dominated by machine-generated code, though, your personal > ideas of 'how often will somebody write this' goes out the window. I think that computer generated code is often more explicit since it has advantages and does not cost too much. Also, fixing a code-generator is easier than fixing handwritten code. Cheers, Carl Friedrich From cfbolz at gmx.de Sat Apr 28 19:56:11 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 28 Apr 2007 19:56:11 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <463337DC.2080503@gmail.com> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> Message-ID: <46338ABB.30808@gmx.de> Antonio Cuni wrote: > Hi Niko, hi Paul! > Here are some remarks/suggestions about your discussion. > I'm also CC-ing pypy-dev, so we can also catch someone else's > suggestions, if any :-). I have one :-). >> First, there are those things are absolutely necessary to do to get PyPy >> to translate: >> >> 1. weak refs >> 2. I don't know what else goes here --- try and see I guess > > Apart from weak refs, I think the other big feature missing is support > for external functions (mostly I/O functions). > Unfortunately this part is a bit of a mess: since in the old days pypy's > only targets were low-level platforms the I/O model of RPython is > modeled after the unix file descriptors. This means that at the moment > high level backends need to emulate the fd interface and forward the > real work to the native I/O functions: to see how gencli does, look at > the ll_os class in translator/cli/src/ll_os.cs. I guess this would be > the easiest way also for genjvm, and probably for now we should pick > this solution. > > I know, this it's both ugly and hackish; the good news is that I and > Carl did some work towards a better solution: at the moment all the I/O > inside the standard interpreter is done using the (interp-level) > rlib/streamio.py library, which in turns uses the low-level file > descriptor interface; the long-term solution would be to provide an > alternative implementation of streamio.py that uses .NET/Java streams > and let the backends using that instead of the current one. I think it is the time now to do away with the file descriptor simulation, it was useful at one point but is very silly now. Instead, a subclass of pypy.rlib.streamio.Stream should be created that only delegates to the Java/.NET Stream classes, probably making use of the facilities for buffering that the platforms offer. I think it is perfectly reasonable to not have os.open and friends on pypy.net as long as file works. If another placeof pypy still uses os.open I am strongly for fixing that. Cheers, Carl Friedrich From arigo at tunes.org Sat Apr 28 20:56:33 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 28 Apr 2007 20:56:33 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <46338ABB.30808@gmx.de> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> Message-ID: <20070428185633.GA3734@code0.codespeak.net> Hi Carl, On Sat, Apr 28, 2007 at 07:56:11PM +0200, Carl Friedrich Bolz wrote: > If another placeof pypy still uses os.open I am strongly > for fixing that. There is the app-level os.open(), which of course uses the interp-level os.open(). It means that if a backend only supports streamio, we can't easily provide the low-level os functions at app-level. This is probably fine, though: I think that Jython doesn't have them at all, for example. (Also, I guess that someone in a reverse-hacking mood could rewrite the interp-level code implementing os.open() to use streamio if necessary...) A bientot, Armin. From cfbolz at gmx.de Sat Apr 28 21:02:55 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sat, 28 Apr 2007 21:02:55 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <20070428185633.GA3734@code0.codespeak.net> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <20070428185633.GA3734@code0.codespeak.net> Message-ID: <46339A5F.4060408@gmx.de> Hi Armin! Armin Rigo wrote: > On Sat, Apr 28, 2007 at 07:56:11PM +0200, Carl Friedrich Bolz wrote: >> If another placeof pypy still uses os.open I am strongly >> for fixing that. > > There is the app-level os.open(), which of course uses the interp-level > os.open(). It means that if a backend only supports streamio, we can't > easily provide the low-level os functions at app-level. This is > probably fine, though: I think that Jython doesn't have them at all, for > example. (Also, I guess that someone in a reverse-hacking mood could > rewrite the interp-level code implementing os.open() to use streamio if > necessary...) That's more or less exactly what I meant by the following: "I think it is perfectly reasonable to not have os.open and friends on pypy.net as long as file works. If another placeof pypy still uses os.open I am strongly for fixing that." But it was probably confusing, because it mixed the applevel os.open (first occurrence) with the interplevel os.oppen (second occurrence). Thanks for clarifying. Cheers, Carl Friedrich From fijal at genesilico.pl Sat Apr 28 21:14:57 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 28 Apr 2007 21:14:57 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <20070428185633.GA3734@code0.codespeak.net> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <20070428185633.GA3734@code0.codespeak.net> Message-ID: <46339D31.4060902@genesilico.pl> Armin Rigo wrote: > Hi Carl, > > On Sat, Apr 28, 2007 at 07:56:11PM +0200, Carl Friedrich Bolz wrote: > >> If another placeof pypy still uses os.open I am strongly >> for fixing that. >> > > There is the app-level os.open(), which of course uses the interp-level > os.open(). It means that if a backend only supports streamio, we can't > easily provide the low-level os functions at app-level. This is > probably fine, though: I think that Jython doesn't have them at all, for > example. (Also, I guess that someone in a reverse-hacking mood could > rewrite the interp-level code implementing os.open() to use streamio if > necessary... That includes os.dup and few others (I can imagine implementing os.dup using streamio, but that would be insane reverse-hacking mood). Probably one good step would be to make our tools (mostly py.test) work without applevel os.dup and friends (it uses it in few places, also for capturing, but that's quite shallow and capturing can be even tuned with options). From anto.cuni at gmail.com Sun Apr 29 10:58:18 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sun, 29 Apr 2007 10:58:18 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <46338ABB.30808@gmx.de> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> Message-ID: <46345E2A.2080103@gmail.com> Carl Friedrich Bolz wrote: > I think it is the time now to do away with the file descriptor > simulation, it was useful at one point but is very silly now. Instead, a > subclass of pypy.rlib.streamio.Stream should be created that only > delegates to the Java/.NET Stream classes, probably making use of the > facilities for buffering that the platforms offer. I think it is > perfectly reasonable to not have os.open and friends on pypy.net as long > as file works. If another placeof pypy still uses os.open I am strongly > for fixing that. I agree, and this is why I mentioned the problem :-). I think there are two ways to make it working: 1) write a dummy CliFile (or JvmFile) subclass of stream, then special-case that class in the backend to map directly to System.Io.FileStream (or the Java equivalent) 2) make CliFile or JvmFile real classes, using the interpret-level bindings to forward the methods; then, we should modify open_file_as_stream and construct_stream_tower to instantiate these classes instead of the standard ones. In both cases I also think it's not trivial to get all the combination of mode/options working, because .NET uses a slightly different set of options than posix to determine how to open a file (I don't know about Java). I think that solution (2) is easier to implement and more readable, but so far it's possible only for gencli because genjvm doens't provide interp-level bindings to java libraries. By contrast Solution (1) is not trivial to implement if the interfaces of our Stream class and the Java's one are very different. Maybe a better solution would be to map the dummy streamio.JvmFile to a class written in Java doing the necessary conversions and forwarding to the native stream class. About the app-level os.* functions; I also think that for now we could simply omit them, but in the long term we should write an alternative implementation based on streamio (IronPython does it in a similar way). ciao Anto From anto.cuni at gmail.com Sun Apr 29 11:00:05 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Sun, 29 Apr 2007 11:00:05 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <46339D31.4060902@genesilico.pl> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <20070428185633.GA3734@code0.codespeak.net> <46339D31.4060902@genesilico.pl> Message-ID: <46345E95.7090004@gmail.com> Maciek Fijalkowski wrote: > Probably one good step would be to make our tools (mostly py.test) work > without applevel os.dup and friends (it uses it in few places, also for > capturing, but that's quite shallow and capturing can be even tuned with > options). +1 (and maybe add a new --noposix option that turns off all those features when running on a platform != posix) From cfbolz at gmx.de Sun Apr 29 12:34:15 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Sun, 29 Apr 2007 12:34:15 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <46345E2A.2080103@gmail.com> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <46345E2A.2080103@gmail.com> Message-ID: <463474A7.7050607@gmx.de> Hi Anto! Antonio Cuni wrote: > I agree, and this is why I mentioned the problem :-). > > I think there are two ways to make it working: > > 1) write a dummy CliFile (or JvmFile) subclass of stream, then > special-case that class in the backend to map directly to > System.Io.FileStream (or the Java equivalent) > > 2) make CliFile or JvmFile real classes, using the interpret-level > bindings to forward the methods; This is indeed the solution that I had in mind. Of course I forgot that there are no interpreter-level bindings for Java yet, how hard would it be to get them? We will need them anyway later. And writing them might still be less work than writing the file descriptor implementation. > then, we should modify open_file_as_stream and construct_stream_tower to > instantiate these classes instead of the standard ones. The result will be very cool because you can then use rlib.streamio and get good cross-platform behavior. > In both cases I also think it's not trivial to get all the combination > of mode/options working, because .NET uses a slightly different set of > options than posix to determine how to open a file (I don't know about > Java). I don't think it is so bad. If you don't want to map the Python semantics to the .NET ones, you can use the .NET stream only as the lowestone in the stream tower and leave buffering and line endings to the existing code. Or you can find an intermediate solution. > About the app-level os.* functions; I also think that for now we could > simply omit them, but in the long term we should write an alternative > implementation based on streamio (IronPython does it in a similar way). Yes, but this can probably even be done on applevel. Cheers, Carl Friedrich From fijal at genesilico.pl Sun Apr 29 12:02:38 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 29 Apr 2007 12:02:38 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <463474A7.7050607@gmx.de> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <46345E2A.2080103@gmail.com> <463474A7.7050607@gmx.de> Message-ID: <46346D3E.8010601@genesilico.pl> Carl Friedrich Bolz wrote: > Hi Anto! > > Antonio Cuni wrote: > > I agree, and this is why I mentioned the problem :-). > > > > I think there are two ways to make it working: > > > > 1) write a dummy CliFile (or JvmFile) subclass of stream, then > > special-case that class in the backend to map directly to > > System.Io.FileStream (or the Java equivalent) > > > > 2) make CliFile or JvmFile real classes, using the interpret-level > > bindings to forward the methods; > > This is indeed the solution that I had in mind. Of course I forgot that > there are no interpreter-level bindings for Java yet, how hard would it > be to get them? We will need them anyway later. And writing them might > still be less work than writing the file descriptor implementation. > Part of it is a SoC anyway, so I don't think we care in what order SoC is done. (and personally I think it makes sense to provide Java bindings first and than to care about the translation). From anto.cuni at gmail.com Mon Apr 30 10:34:37 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 30 Apr 2007 10:34:37 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <463474A7.7050607@gmx.de> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <46345E2A.2080103@gmail.com> <463474A7.7050607@gmx.de> Message-ID: <4635AA1D.2080704@gmail.com> Carl Friedrich Bolz wrote: > > 2) make CliFile or JvmFile real classes, using the interpret-level > > bindings to forward the methods; > > This is indeed the solution that I had in mind. Of course I forgot that > there are no interpreter-level bindings for Java yet, how hard would it > be to get them? We will need them anyway later. And writing them might > still be less work than writing the file descriptor implementation. The implementation of file descriptor by itself is quite easy, but I agree that in this case the correct solution is to develop java bindings for RPython, which will probably also be useful for next tasks such as JSR-223. Also, java bindings could share (as usual :-)) most of the rtyper code used by the .NET bindings. > > then, we should modify open_file_as_stream and construct_stream_tower to > > instantiate these classes instead of the standard ones. > > The result will be very cool because you can then use rlib.streamio and > get good cross-platform behavior. this would be a big bonus. Going further, how hard would it be to rtype the builtin "file" to use streamio? > I don't think it is so bad. If you don't want to map the Python > semantics to the .NET ones, you can use the .NET stream only as the > lowestone in the stream tower and leave buffering and line endings to > the existing code. Or you can find an intermediate solution. Yes, it makes sense. .NET streams are always buffered when reading, I don't know if/how another layer of buffering could affect performances. Moreover, I think there is no easy way to open a stream in "text" mode: on Windows if you open a text file with the low level stream interface you get "\r\n" at the end of lines, so it might be necessary to insert another layer that does the conversion (the same as I'm doing now in the C# implementation -- see the CRLFTextFile class in cli/srs/ll_os.cs). > > About the app-level os.* functions; I also think that for now we could > > simply omit them, but in the long term we should write an alternative > > implementation based on streamio (IronPython does it in a similar way). > > Yes, but this can probably even be done on applevel. sure, and it would be probably reusable both for cli and jvm! ciao Anto From anto.cuni at gmail.com Mon Apr 30 10:42:43 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 30 Apr 2007 10:42:43 +0200 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <46346D3E.8010601@genesilico.pl> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <46345E2A.2080103@gmail.com> <463474A7.7050607@gmx.de> <46346D3E.8010601@genesilico.pl> Message-ID: <4635AC03.9050107@gmail.com> Maciek Fijalkowski wrote: > Part of it is a SoC anyway, so I don't think we care in what order SoC > is done. (and personally I think it makes sense to provide Java bindings > first and than to care about the translation). Well, strictly speaking java bindings for rpython are not part of Paul's SoC proposal but indeed they are probably the most effective way to implement other features that he promised. Btw, I'm not sure it's a good idea to develop them as the first task in PyPy, because it's not straightforward if you have no experience with the rtyper. My suggestion is to start by porting tests to genjvm and fixing the discovered bugs, because it should be a more newcomer-friendly task; then java-bindings for rpython and I/O layer; finally finally translation of pypy-jvm. Paul, what do you think of this plan? ciao Anto From paul.degrandis at gmail.com Mon Apr 30 13:41:43 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Mon, 30 Apr 2007 07:41:43 -0400 Subject: [pypy-dev] PyPy JVM Backend In-Reply-To: <4635AC03.9050107@gmail.com> References: <0BB159E0-B748-40ED-8D08-B838FB7C4ECA@alum.mit.edu> <463337DC.2080503@gmail.com> <46338ABB.30808@gmx.de> <46345E2A.2080103@gmail.com> <463474A7.7050607@gmx.de> <46346D3E.8010601@genesilico.pl> <4635AC03.9050107@gmail.com> Message-ID: <9c0bb8a00704300441k151f9fe0t8806b8f152200e11@mail.gmail.com> Anto, I think this is a good strategy for me. It'll allow me to put something material to the information I gained from your thesis and gain knowledge that will help me in the long run with PyPy. The tasks for me will be: 1.) Port tests, fix uncovered bugs 2.) Java bindings 3.) I/O layer as discussed on this list 4.) pypy-jvm translation 5.) JSR 223 (The Scripting API) For more information on JSR 223, here's a little tutorial/article: http://www.onjava.com/pub/a/onjava/2006/04/26/mustang-meets-rhino-java-se-6-scripting.html And here are the languages with bindings for it now: https://scripting.dev.java.net/ Regards, Paul On 4/30/07, Antonio Cuni wrote: > > Maciek Fijalkowski wrote: > > > Part of it is a SoC anyway, so I don't think we care in what order SoC > > is done. (and personally I think it makes sense to provide Java bindings > > first and than to care about the translation). > > Well, strictly speaking java bindings for rpython are not part of Paul's > SoC proposal but indeed they are probably the most effective way to > implement other features that he promised. > > Btw, I'm not sure it's a good idea to develop them as the first task in > PyPy, because it's not straightforward if you have no experience with > the rtyper. > > My suggestion is to start by porting tests to genjvm and fixing the > discovered bugs, because it should be a more newcomer-friendly task; > then java-bindings for rpython and I/O layer; finally finally > translation of pypy-jvm. > > Paul, what do you think of this plan? > > ciao Anto > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20070430/edb5fcfc/attachment.htm From paul.degrandis at gmail.com Mon Apr 30 15:45:54 2007 From: paul.degrandis at gmail.com (Paul deGrandis) Date: Mon, 30 Apr 2007 09:45:54 -0400 Subject: [pypy-dev] What about a Slate front-end? Message-ID: <9c0bb8a00704300645p463b04f6vec7a3ca226a3c4a4@mail.gmail.com> Hi all, Not too long ago there was a discussion about possible front-ends. I know someone here is part of the tunes project... so how about a slate front end? There has been recent talk on the slate mailing list about the deficiencies with the current VM and some weird setup and handling of the stack... perhaps pypy is a better setup for the language. Just a thought. Regards, Paul -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/pypy-dev/attachments/20070430/837db82d/attachment-0001.htm From arigo at tunes.org Wed May 2 20:42:49 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 2 May 2007 20:42:49 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: References: <20070414090553.GA20210@code0.codespeak.net> Message-ID: <20070502184249.GA14450@code0.codespeak.net> Hi Terry, hi all, On Sat, Apr 14, 2007 at 07:21:56PM -0400, Terry Reedy wrote: > | > fwiw, I was considering to submit my Prolog paper (or a variation of > it) > | > there - but we can do some common planning, not sure whether we should > | > submit two PyPy papers. > | > | I think that it's a good place to submit a "core" PyPy paper, and I also > | think that your Prolog paper is independent enough to not count as > | another PyPy paper. In fact I'd imagine that if we were "academic > | enough", at this point, several of us should try to publish several > | papers independently on several aspects of PyPy... > > For instance, if there is anything innovative in the JIT work, then that > might make a nice paper in itself. Now that the last-minute report writing is done - at least the technical reports - we should think about Dyla again. Remember that it is not a conference but a workshop, which as far as I understand it means that everybody is a speaker. We need to send in "position papers" - "we really think that this is the way this should be done". The obvious topics for us are "nobody should write complicated {virtual machines, interpreters, JIT compilers} by hand". Thoughts, topics? We have plenty of topics in and around PyPy, but let's keep in mind that each paper needs a "position paper" angle. A bientot, Armin. From arigo at tunes.org Wed May 2 20:51:05 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 2 May 2007 20:51:05 +0200 Subject: [pypy-dev] What about a Slate front-end? In-Reply-To: <9c0bb8a00704300645p463b04f6vec7a3ca226a3c4a4@mail.gmail.com> References: <9c0bb8a00704300645p463b04f6vec7a3ca226a3c4a4@mail.gmail.com> Message-ID: <20070502185105.GB14450@code0.codespeak.net> Hi Paul, On Mon, Apr 30, 2007 at 09:45:54AM -0400, Paul deGrandis wrote: > Not too long ago there was a discussion about possible front-ends. > I know someone here is part of the tunes project... so how about a slate > front end? Slate is a strange language, but I guess its basics are not hard to write in RPython, if someone is interested. A Smalltalk front-end might be more useful to start with, though. I am the one here linked with the Tunes project - the e-mail address is some hint - but I have to say that I don't find Slate particularly exciting, beyond the fact that it starts from an interesting idea for multimethods. The very short file http://codespeak.net/svn/pypy/dist/pypy/tool/pairtype.py was inspired by this. A bientot, Armin. From lac at openend.se Sun May 6 09:16:41 2007 From: lac at openend.se (Laura Creighton) Date: Sun, 6 May 2007 09:16:41 +0200 Subject: [pypy-dev] Did anything happen with the mini coding Sprint in Poland idea? Message-ID: <200705060716.l467GfEk004421@theraft.openend.se> Dethe Elza is planning his European vacation, and he is flying into Amsterdam, August 16th and then going to Prague. The bulk of the vacation will be in Sofia, with some side trips, and then back to Amsterdam for a return trip Sept 11. So he misses EuroPython. He'd like to meet pypyers face to faqce and hack if possible. Do we have anything planned yet? Laura From stephan.diehl at gmx.net Mon May 7 09:00:56 2007 From: stephan.diehl at gmx.net (Stephan Diehl) Date: Mon, 07 May 2007 09:00:56 +0200 Subject: [pypy-dev] Did anything happen with the mini coding Sprint in Poland idea? In-Reply-To: <200705060716.l467GfEk004421@theraft.openend.se> References: <200705060716.l467GfEk004421@theraft.openend.se> Message-ID: <463ECEA8.5050805@gmx.net> What about PyCon UK on 8/9.9. (http://www.pyconuk.org/)? Just a thought Stephan Laura Creighton wrote: > Dethe Elza is planning his European vacation, and he is flying into > Amsterdam, August 16th and then going to Prague. The bulk of the vacation > will be in Sofia, with some side trips, and then back to Amsterdam for > a return trip Sept 11. So he misses EuroPython. He'd like to meet > pypyers face to faqce and hack if possible. Do we have anything > planned yet? > > Laura > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From holger at merlinux.de Tue May 8 08:51:38 2007 From: holger at merlinux.de (holger krekel) Date: Tue, 8 May 2007 08:51:38 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: <20070502184249.GA14450@code0.codespeak.net> References: <20070414090553.GA20210@code0.codespeak.net> <20070502184249.GA14450@code0.codespeak.net> Message-ID: <20070508065138.GJ25045@solar.trillke> Hi Armin, hi all, On Wed, May 02, 2007 at 20:42 +0200, Armin Rigo wrote: > Hi Terry, hi all, > > On Sat, Apr 14, 2007 at 07:21:56PM -0400, Terry Reedy wrote: > > | > fwiw, I was considering to submit my Prolog paper (or a variation of > > it) > > | > there - but we can do some common planning, not sure whether we should > > | > submit two PyPy papers. > > | > > | I think that it's a good place to submit a "core" PyPy paper, and I also > > | think that your Prolog paper is independent enough to not count as > > | another PyPy paper. In fact I'd imagine that if we were "academic > > | enough", at this point, several of us should try to publish several > > | papers independently on several aspects of PyPy... > > > > For instance, if there is anything innovative in the JIT work, then that > > might make a nice paper in itself. > > Now that the last-minute report writing is done - at least the technical > reports - we should think about Dyla again. Remember that it is not a > conference but a workshop, which as far as I understand it means that > everybody is a speaker. We need to send in "position papers" - "we > really think that this is the way this should be done". The obvious > topics for us are "nobody should write complicated {virtual machines, > interpreters, JIT compilers} by hand". > > Thoughts, topics? We have plenty of topics in and around PyPy, but > let's keep in mind that each paper needs a "position paper" angle. I hope we can re-use parts of our technical EU reports (most of which haven't been published yet at a conference). Currently I could imagine two position papers: nobody should write complicated VMs/JIT-compilers by hand interpreters should offer {security,persistence,distribution} features directly (instead of adding Application APIs) Until Friday, i am still quite busy with lots of EU finalisation issues - not sure how much i can do till then but on saturday i could be there to work/help on the paper submissions (and it's the submission deadline IIUC, sigh). best, holger From arigo at tunes.org Tue May 8 13:35:11 2007 From: arigo at tunes.org (Armin Rigo) Date: Tue, 8 May 2007 13:35:11 +0200 Subject: [pypy-dev] Forward: Dyla 07 In-Reply-To: <20070508065138.GJ25045@solar.trillke> References: <20070414090553.GA20210@code0.codespeak.net> <20070502184249.GA14450@code0.codespeak.net> <20070508065138.GJ25045@solar.trillke> Message-ID: <20070508113511.GA20580@code0.codespeak.net> Hi, On Tue, May 08, 2007 at 08:51:38AM +0200, holger krekel wrote: > Currently I could imagine two position papers: > > nobody should write complicated VMs/JIT-compilers by hand > > interpreters should offer {security,persistence,distribution} > features directly (instead of adding Application APIs) That would make sense, but of course the limiting factor is going to be who wants to write them. I have started drafting the first one, but for now I just hope I can get it reasonably done by the deadline. I don't see me helping on a second paper as well. BTW, googling "how to write a position paper" gives useful hints about the expected structure and content. A bientot, Armin. From jgustak at gmail.com Wed May 9 23:05:17 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Wed, 9 May 2007 23:05:17 +0200 Subject: [pypy-dev] scheme parser. Message-ID: I have been experimenting with rlib.parsing.ebnfparse today. One thing which is not described, if you want to use ebnfparse you have to: > py.test.config.parse([]) Can anyone explain what happens there? Is there any other "magical" initialization stuff which i have to know before going on? ebnfparse seems to work just fine, so I will stick with it, unless there is any contraindication. Now I am considering if the parser should generate some bytecode? Maybe pickling (not rpythonic) AST tree will be just fine (like in js interpreter)? Sincerely, Jakub Gustak From santagada at gmail.com Thu May 10 00:24:19 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Wed, 9 May 2007 19:24:19 -0300 Subject: [pypy-dev] scheme parser. In-Reply-To: References: Message-ID: <4AB71EA1-0AF0-4437-BFC9-674122A8D32F@gmail.com> Em 09/05/2007, ?s 18:05, Jakub Gustak escreveu: > I have been experimenting with rlib.parsing.ebnfparse today. One thing > which is not described, if you want to use ebnfparse you have to: >> py.test.config.parse([]) > I don't do that, so thats probably not necessary. Wy not put your code on lang/scheme so we can all see it? > Can anyone explain what happens there? Is there any other "magical" > initialization stuff which i have to know before going on? > > ebnfparse seems to work just fine, so I will stick with it, unless > there is any contraindication. parsing is a great language parser generator, and probably has everything a scheme interpreter will need. > Now I am considering if the parser should generate some bytecode? > Maybe pickling (not rpythonic) AST tree will be just fine (like in js > interpreter)? I am pickling the tree only if running on top of CPython, because RPython don't support that. And even then, it is really just a hack because parsing took a lot of time running on top of cpython. Iam going to give you the same advice I got from armin when I started... begin with AST and then when you have your parser ready we can see if using bytecode really get you faster. -- Leonardo Santagada santagada at gmail.com From arigo at tunes.org Thu May 10 14:59:15 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 10 May 2007 14:59:15 +0200 Subject: [pypy-dev] scheme parser. In-Reply-To: <4AB71EA1-0AF0-4437-BFC9-674122A8D32F@gmail.com> References: <4AB71EA1-0AF0-4437-BFC9-674122A8D32F@gmail.com> Message-ID: <20070510125915.GA20089@code0.codespeak.net> Hi, On Wed, May 09, 2007 at 07:24:19PM -0300, Leonardo Santagada wrote: > > Now I am considering if the parser should generate some bytecode? > > Maybe pickling (not rpythonic) AST tree will be just fine (like in js > > interpreter)? > > I am pickling the tree only if running on top of CPython, because > RPython don't support that. And even then, it is really just a hack > because parsing took a lot of time running on top of cpython. Iam > going to give you the same advice I got from armin when I started... > begin with AST and then when you have your parser ready we can see if > using bytecode really get you faster. Yes, that's even more true for Scheme, where the AST is very simple. For now you should just interpret it, and we can see later if speed-ups can be achieved with bytecode. Also, if you are directly using rlib.parsing, then you don't need any pickling: parse the source into an AST, and pass the AST objects directly to the interpreter. (We can think later if it makes sense to add an equivalent of the .pyc files, but that's really questions for the very long term only.) A bientot, Armin. From cfbolz at gmx.de Thu May 10 16:20:54 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 10 May 2007 16:20:54 +0200 Subject: [pypy-dev] scheme parser. In-Reply-To: <20070510125915.GA20089@code0.codespeak.net> References: <4AB71EA1-0AF0-4437-BFC9-674122A8D32F@gmail.com> <20070510125915.GA20089@code0.codespeak.net> Message-ID: <46432A46.1060501@gmx.de> Armin Rigo wrote: > Yes, that's even more true for Scheme, where the AST is very simple. > For now you should just interpret it, and we can see later if speed-ups > can be achieved with bytecode. Also, if you are directly using > rlib.parsing, then you don't need any pickling: parse the source into an > AST, and pass the AST objects directly to the interpreter. (We can > think later if it makes sense to add an equivalent of the .pyc files, > but that's really questions for the very long term only.) I kind of hope that the parser is fast enough to not have to cache the trees for "reasonably" sized programs. I haven't measured them, but the lexing and parsing algorithms have nice bounds. In addition, it's probably simple to have a generic pickle format for parse trees that could be re-used across all users of rlib/parsing. But for Scheme even that is probably too much work, since the data structures should be very easy to serialize. Cheers, Carl Friedrich From len-l at telus.net Fri May 11 00:07:42 2007 From: len-l at telus.net (Lenard Lindstrom) Date: Thu, 10 May 2007 15:07:42 -0700 Subject: [pypy-dev] scheme parser. In-Reply-To: <20070510125915.GA20089@code0.codespeak.net> References: <4AB71EA1-0AF0-4437-BFC9-674122A8D32F@gmail.com> <20070510125915.GA20089@code0.codespeak.net> Message-ID: <464397AE.1050902@telus.net> Armin Rigo wrote: > Hi, > > On Wed, May 09, 2007 at 07:24:19PM -0300, Leonardo Santagada wrote: > >>> Now I am considering if the parser should generate some bytecode? >>> Maybe pickling (not rpythonic) AST tree will be just fine (like in js >>> interpreter)? >>> >> I am pickling the tree only if running on top of CPython, because >> RPython don't support that. And even then, it is really just a hack >> because parsing took a lot of time running on top of cpython. Iam >> going to give you the same advice I got from armin when I started... >> begin with AST and then when you have your parser ready we can see if >> using bytecode really get you faster. >> > > Yes, that's even more true for Scheme, where the AST is very simple. > For now you should just interpret it, and we can see later if speed-ups > can be achieved with bytecode. Also, if you are directly using > rlib.parsing, then you don't need any pickling: parse the source into an > AST, and pass the AST objects directly to the interpreter. (We can > think later if it makes sense to add an equivalent of the .pyc files, > but that's really questions for the very long term only.) > > The lisp interpreter loop body is very simple: (print (eval (read))) First implement read and print. Implement eval when input can be reliably converted to s-expressions. There is no need for a distinct parser or special AST structures if that is the plan. That would come into play in writing a compiler. Lenard Lindstrom From holger at merlinux.de Fri May 11 00:52:38 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 11 May 2007 00:52:38 +0200 Subject: [pypy-dev] quick review of final activity report? Message-ID: <20070510225238.GG25045@solar.trillke> Hi all! here is our preliminary "final activity report" draft which summarizes the whole PyPy EU project period on several levels. (Heh, the EU part is not finished yet - we are heading towards the final review on 31st May). http://codespeak.net/~hpk/pypy-final-activity-may-10-2007.pdf If you ad-hoc review and feedback on the report during the next 12-14 hours (until friday 11th 9am UTC+2) i could still try to incorporate changes before it is declared finished and published ... particularly readability and sense-making are of interest :) feedback is welcome also later, of course. best & thanks, holger -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From arigo at tunes.org Sun May 13 15:12:30 2007 From: arigo at tunes.org (Armin Rigo) Date: Sun, 13 May 2007 15:12:30 +0200 Subject: [pypy-dev] Dyla 07 Message-ID: <20070513131230.GA6079@code0.codespeak.net> Hi all, I didn't do any work on any Dyla'07 paper so I was about to post some apology and thanks to the people that proposed to help me even though they were more busy than me with EU reports. Recently I have not been up to working on anything at all (which is not an excuse but just a fact I guess). However, the submission deadline was extended to May 27th. I still don't know if I'm going to find enough motivation, but that gives us a second chance... A bientot, Armin From alexandre.fayolle at free.fr Mon May 14 14:26:45 2007 From: alexandre.fayolle at free.fr (Alexandre Fayolle) Date: Mon, 14 May 2007 14:26:45 +0200 Subject: [pypy-dev] building pypy on a powerpc running linux Message-ID: <1179145605.464855851cd5e@imp.free.fr> Hi, I know pypy builds on macos X, but has anyone tried (and succeeded) to translate on a powerpc running Linux? The Debian autobuilders have failed to compile it, and I wonder if this is an issue in my package or just that this platform is not supported. The build logs are available at http://buildd.debian.org/build.php?pkg=pypy Thanks in advance -- Alexandre Fayolle From fijal at genesilico.pl Mon May 14 17:04:27 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 14 May 2007 17:04:27 +0200 Subject: [pypy-dev] building pypy on a powerpc running linux In-Reply-To: <1179145605.464855851cd5e@imp.free.fr> References: <1179145605.464855851cd5e@imp.free.fr> Message-ID: <46487A7B.3050608@genesilico.pl> Alexandre Fayolle wrote: > Hi, > > I know pypy builds on macos X, but has anyone tried (and succeeded) to translate > on a powerpc running Linux? The Debian autobuilders have failed to compile it, > and I wonder if this is an issue in my package or just that this platform is not > supported. > > The build logs are available at http://buildd.debian.org/build.php?pkg=pypy > > Thanks in advance > > -- > Alexandre Fayolle > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > > :ms > > The nightly benchmarks are running on power pc processor. http://tuatara.cs.uni-duesseldorf.de/benchmark.html From fijal at genesilico.pl Mon May 14 17:06:05 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 14 May 2007 17:06:05 +0200 Subject: [pypy-dev] building pypy on a powerpc running linux In-Reply-To: <1179145605.464855851cd5e@imp.free.fr> References: <1179145605.464855851cd5e@imp.free.fr> Message-ID: <46487ADD.8050805@genesilico.pl> Alexandre Fayolle wrote: > Hi, > > I know pypy builds on macos X, but has anyone tried (and succeeded) to translate > on a powerpc running Linux? The Debian autobuilders have failed to compile it, > and I wonder if this is an issue in my package or just that this platform is not > supported. > > The build logs are available at http://buildd.debian.org/build.php?pkg=pypy > > Thanks in advance > > For me it seems like build started swapping (not enough memory or so?) and was killed due to inactivity. From holger at merlinux.de Mon May 14 20:59:27 2007 From: holger at merlinux.de (holger krekel) Date: Mon, 14 May 2007 20:59:27 +0200 Subject: [pypy-dev] [ann] pypy EU project finishing up Message-ID: <20070514185927.GW25045@solar.trillke> Hi all, puh! ... The 28 months of the PyPy[*] EU project period have been an intense experience on many levels. We wrote a summary of results, development methods and experiences here: http://codespeak.net/pypy/extradoc/eu-report/PYPY-EU-Final-Activity-Report.pdf and are now heading for the final review meeting in Bruxelles (31st May). Most involved people look forward to some resting and letting the dust settle before tackling next steps and sprints. best, holger [*] The PyPy project aims to generate flexible and fast Python Implementations for various target environments and provides largely re-usable capabilities for implementing other dynamic languages. More info: http://codespeak.net/pypy -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From bea at changemaker.nu Tue May 15 06:04:30 2007 From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=) Date: Tue, 15 May 2007 06:04:30 +0200 Subject: [pypy-dev] PyPy Europython talks? Message-ID: <4649314E.2020303@changemaker.nu> Hi there Back from Japan and thinking a bit about Europython 2007. Submission deadline is closing in and I wonder if we are to do any PyPy talks at EP 2007? I would be prepared to do one PyPy talk on the methodology - such as an experience summary on how we worked with sprints etc.... What do we want to do on the technical side? Submission deadline 18th of May. Sorry to ask about this when I know many are exhausted and focused on finalizing the EU-review - but I felt the need to ask.... Cheers Bea From faassen at startifact.com Tue May 15 13:00:05 2007 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 15 May 2007 13:00:05 +0200 Subject: [pypy-dev] PyPy Europython talks? In-Reply-To: <4649314E.2020303@changemaker.nu> References: <4649314E.2020303@changemaker.nu> Message-ID: Beatrice D?ring wrote: > Hi there > > Back from Japan and thinking a bit about Europython 2007. > > Submission deadline is closing in and I wonder if we are to do any PyPy > talks at EP 2007? I would be prepared to do one PyPy talk on the methodology > - such as an experience summary on how we worked with sprints etc.... > > What do we want to do on the technical side? > > Submission deadline 18th of May. > > Sorry to ask about this when I know many are exhausted and focused > on finalizing the EU-review - but I felt the need to ask.... I would like to see PyPy talks at EuroPython! It's a tradition! :) Regards, Martijn From fijal at genesilico.pl Tue May 15 13:51:52 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Tue, 15 May 2007 13:51:52 +0200 Subject: [pypy-dev] PyPy Europython talks? In-Reply-To: <4649314E.2020303@changemaker.nu> References: <4649314E.2020303@changemaker.nu> Message-ID: <46499ED8.7060008@genesilico.pl> Beatrice D?ring wrote: > Hi there > > Back from Japan and thinking a bit about Europython 2007. > > Submission deadline is closing in and I wonder if we are to do any PyPy > talks at EP 2007? I would be prepared to do one PyPy talk on the methodology > - such as an experience summary on how we worked with sprints etc.... > > What do we want to do on the technical side? > > Submission deadline 18th of May. > > Sorry to ask about this when I know many are exhausted and focused > on finalizing the EU-review - but I felt the need to ask.... > > Cheers > > Bea Hi Bea! There were some discussions on IRC regarding technical talks on the Europython. The rough outline what was proposed was to give 3 technical talks. 1. "usual pypy talk" 2. RPython talk (anto was talking about it) 3. unique features talk (regarding interpreter). (me & hpk are planning to do this). This is just my POV, but there are some ideas to give those talks :) From bea at changemaker.nu Tue May 15 19:27:23 2007 From: bea at changemaker.nu (=?UTF-8?B?QmVhdHJpY2UgRMO8cmluZw==?=) Date: Tue, 15 May 2007 19:27:23 +0200 Subject: [pypy-dev] PyPy Europython talks? In-Reply-To: <46499ED8.7060008@genesilico.pl> References: <4649314E.2020303@changemaker.nu> <46499ED8.7060008@genesilico.pl> Message-ID: <4649ED7B.4040109@changemaker.nu> Hi there Maciek Fijalkowski skrev: > Beatrice D?ring wrote: >> Hi there >> >> Back from Japan and thinking a bit about Europython 2007. >> >> Submission deadline is closing in and I wonder if we are to do any PyPy >> talks at EP 2007? I would be prepared to do one PyPy talk on the >> methodology >> - such as an experience summary on how we worked with sprints etc.... >> >> What do we want to do on the technical side? >> >> Submission deadline 18th of May. >> >> Sorry to ask about this when I know many are exhausted and focused >> on finalizing the EU-review - but I felt the need to ask.... >> >> Cheers >> >> Bea > Hi Bea! > > There were some discussions on IRC regarding technical talks on the > Europython. The rough outline what was proposed was to give 3 > technical talks. > > 1. "usual pypy talk" > 2. RPython talk (anto was talking about it) > 3. unique features talk (regarding interpreter). (me & hpk are > planning to do this). > > This is just my POV, but there are some ideas to give those talks :) > > Thanks for the answer Maciek - it sounds like we are upholding the PyPy tradition Martijn holds so dear ;-) Cheers Bea From anto.cuni at gmail.com Thu May 17 20:34:33 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 17 May 2007 20:34:33 +0200 Subject: [pypy-dev] DLS paper on RPython In-Reply-To: <419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu> References: <4643232C.9020007@gmail.com> <08FB5296-C3B7-48C3-B6A1-1750D085ACDB@alum.mit.edu> <46436E08.2040801@gmail.com> <464462EA.9070301@gmail.com> <180B9E23-9667-4278-A811-DBB1E77AB2F9@alum.mit.edu> <46481697.4040101@disi.unige.it> <7ADA7D08-1E1A-4874-BC92-3E69C83985BB@alum.mit.edu> <46499E9B.6020404@disi.unige.it> <96E254B7-C6B0-4F89-887F-48D5632CA499@alum.mit.edu> <4649CFD3.9000204@disi.unige.it> <4649D0A6.8000806@disi.unige.it> <464C15AF.6090205@disi.unige.it> <464C19A6.2020809@gmail.com> <464C3232.5010405@disi.unige.it> <6C5016FD-7582-4361-BA2B-071AEC729187@alum.mit.edu> <464C62D2.7040801@gmail.com> <464C7437.7090105@gmail.com> <419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu> Message-ID: <464CA039.8080706@gmail.com> [I respond only to you because I don't think Massimo and Davide are interested in such details; I'm also cc-ing pypy-dev] Niko Matsakis wrote: > Ok. I'll look at that some. I may have to rewrite my section on > exceptions to describe the type-wrapping solution; I wrote it to > describe the newer system I started to checked in today, but then I > found that RPython allows any object, not just Exceptions, to be thrown, > meaning that this newer system won't work as well as I thought (and so > is currently disabled). Probably the simplest way to solve the problem is not to allow arbitrary objects to be thrown in RPython; I quick grep inside objspace/std and interpret didn't show any case in which this feature is used, and in any case it should be very easy to patch. Else, you could special-case the raise/except when using subclasses of Exception, and don't do the wrapping in those cases; of course you would have to be careful when catching all the exceptions, something like this: try: foo() except: bar() needs to be translated to: try { foo() } catch(Exception e) { // this is __builtin__.Exception bar() } catch(ExceptionWrapper e) { bar() } or something similar. I think we should really try to avoid ExceptionWrapper beacuse it's both ugly and probably veeery slow. ciao Anto From niko at alum.mit.edu Thu May 17 22:07:12 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Thu, 17 May 2007 22:07:12 +0200 Subject: [pypy-dev] Throwing arbitrary objects as exceptions (was Re: DLS paper on RPython) In-Reply-To: <464CA039.8080706@gmail.com> References: <4643232C.9020007@gmail.com> <08FB5296-C3B7-48C3-B6A1-1750D085ACDB@alum.mit.edu> <46436E08.2040801@gmail.com> <464462EA.9070301@gmail.com> <180B9E23-9667-4278-A811-DBB1E77AB2F9@alum.mit.edu> <46481697.4040101@disi.unige.it> <7ADA7D08-1E1A-4874-BC92-3E69C83985BB@alum.mit.edu> <46499E9B.6020404@disi.unige.it> <96E254B7-C6B0-4F89-887F-48D5632CA499@alum.mit.edu> <4649CFD3.9000204@disi.unige.it> <4649D0A6.8000806@disi.unige.it> <464C15AF.6090205@disi.unige.it> <464C19A6.2020809@gmail.com> <464C3232.5010405@disi.unige.it> <6C5016FD-7582-4361-BA2B-071AEC729187@alum.mit.edu> <464C62D2.7040801@gmail.com> <464C7437.7090105@gmail.com> <419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu> <464CA039.8080706@gmail.com> Message-ID: I guess the question is whether people consider the ability to throw arbitrary objects (rather than only those whose type is a subtype of Exception) a feature or a bug. I'm on the fence myself: it makes translating to the jvm and clr easier if we prohibit arbitrary objects, and it seems like a marginal use-case, but on the other hand, well, Python can do it, so why not? If anyone wanted to give me a few pointers as to what has to be changed to force exceptions to be subtypes of Exception, I'd be excited to try hacking it, just so I can learn more about that part of PyPy. On the other hand, I think that a hybrid wrapping strategy could actually work reasonably well and avoid wrapping in the 99% of cases where Exception sub-types are thrown and caught. Therefore, I'd say the real question is how people on the list think that RPython "ought" to behave. Niko From pedronis at openendsystems.com Fri May 18 10:29:02 2007 From: pedronis at openendsystems.com (Samuele Pedroni) Date: Fri, 18 May 2007 10:29:02 +0200 Subject: [pypy-dev] Throwing arbitrary objects as exceptions (was Re: DLS paper on RPython) In-Reply-To: References: <4643232C.9020007@gmail.com> <08FB5296-C3B7-48C3-B6A1-1750D085ACDB@alum.mit.edu> <46436E08.2040801@gmail.com> <464462EA.9070301@gmail.com> <180B9E23-9667-4278-A811-DBB1E77AB2F9@alum.mit.edu> <46481697.4040101@disi.unige.it> <7ADA7D08-1E1A-4874-BC92-3E69C83985BB@alum.mit.edu> <46499E9B.6020404@disi.unige.it> <96E254B7-C6B0-4F89-887F-48D5632CA499@alum.mit.edu> <4649CFD3.9000204@disi.unige.it> <4649D0A6.8000806@disi.unige.it> <464C15AF.6090205@disi.unige.it> <464C19A6.2020809@gmail.com> <464C3232.5010405@disi.unige.it> <6C5016FD-7582-4361-BA2B-071AEC729187@alum.mit.edu> <464C62D2.7040801@gmail.com> <464C7437.7090105@gmail.com> <419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu> <464CA039.8080706@gmail.com> Message-ID: <464D63CE.3060300@openendsystems.com> Niko Matsakis wrote: > I guess the question is whether people consider the ability to throw > arbitrary objects (rather than only those whose type is a subtype of > Exception) a feature or a bug. > > I'm on the fence myself: it makes translating to the jvm and clr > easier if we prohibit arbitrary objects, and it seems like a marginal > use-case, but on the other hand, well, Python can do it, so why not? > the idea is that only subclasses of Exception/BaseException can be thrown in RPython, we have no code to enforce that right now OTOH if you find places in our RPython code that don't do that those are bugs to fix. > If anyone wanted to give me a few pointers as to what has to be > changed to force exceptions to be subtypes of Exception, I'd be > excited to try hacking it, just so I can learn more about that part > of PyPy. > > On the other hand, I think that a hybrid wrapping strategy could > actually work reasonably well and avoid wrapping in the 99% of cases > where Exception sub-types are thrown and caught. > > Therefore, I'd say the real question is how people on the list think > that RPython "ought" to behave. > > > Niko > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From niko at alum.mit.edu Fri May 18 11:14:22 2007 From: niko at alum.mit.edu (Niko Matsakis) Date: Fri, 18 May 2007 11:14:22 +0200 Subject: [pypy-dev] Throwing arbitrary objects as exceptions (was Re: DLS paper on RPython) In-Reply-To: <464D63CE.3060300@openendsystems.com> References: <4643232C.9020007@gmail.com> <08FB5296-C3B7-48C3-B6A1-1750D085ACDB@alum.mit.edu> <46436E08.2040801@gmail.com> <464462EA.9070301@gmail.com> <180B9E23-9667-4278-A811-DBB1E77AB2F9@alum.mit.edu> <46481697.4040101@disi.unige.it> <7ADA7D08-1E1A-4874-BC92-3E69C83985BB@alum.mit.edu> <46499E9B.6020404@disi.unige.it> <96E254B7-C6B0-4F89-887F-48D5632CA499@alum.mit.edu> <4649CFD3.9000204@disi.unige.it> <4649D0A6.8000806@disi.unige.it> <464C15AF.6090205@disi.unige.it> <464C19A6.2020809@gmail.com> <464C3232.5010405@disi.unige.it> <6C5016FD-7582-4361-BA2B-071AEC729187@alum.mit.edu> <464C62D2.7040801@gmail.com> <464C7437.7090105@gmail.com> <419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu> <464CA039.8080706@gmail.com> <464D63CE.3060300@openendsystems.com> Message-ID: <4BA670AE-8EC2-4E47-8756-9BE46E0DE2A2@alum.mit.edu> >> > the idea is that only subclasses of Exception/BaseException can be > thrown in RPython, we have no code > to enforce that right now OTOH if you find places in our RPython > code that don't do that those > are bugs to fix. Okay, I guess the right thing to do then is to insert a CHECKCAST when the static type of a thrown exception is not a sub-type of Exception. "Correct" RPython should work, and the rest will throw a ClassCastException. If the annotator/rtyper were later changed to provide a static type better than Object, then the casts won't be generated. That's what I've done for now. It fixes the broken test, at least. :) Niko From tjreedy at udel.edu Sat May 19 05:51:13 2007 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 18 May 2007 23:51:13 -0400 Subject: [pypy-dev] Throwing arbitrary objects as exceptions (was Re: DLSpaper on RPython) References: <4643232C.9020007@gmail.com><08FB5296-C3B7-48C3-B6A1-1750D085ACDB@alum.mit.edu><46436E08.2040801@gmail.com><464462EA.9070301@gmail.com><180B9E23-9667-4278-A811-DBB1E77AB2F9@alum.mit.edu><46481697.4040101@disi.unige.it><7ADA7D08-1E1A-4874-BC92-3E69C83985BB@alum.mit.edu><46499E9B.6020404@disi.unige.it><96E254B7-C6B0-4F89-887F-48D5632CA499@alum.mit.edu><4649CFD3.9000204@disi.unige.it> <4649D0A6.8000806@disi.unige.it><464C15AF.6090205@disi.unige.it> <464C19A6.2020809@gmail.com><464C3232.5010405@disi.unige.it><6C5016FD-7582-4361-BA2B-071AEC729187@alum.mit.edu><464C62D2.7040801@gmail.com><464C7437.7090105@gmail.com><419A3BF2-694D-4A50-A340-EABBD563D37E@alum.mit.edu><464CA039.8080706@gmail.com> Message-ID: "Niko Matsakis" wrote in message news:FF531020-FE60-47B2-A30F-CB13D6E696F0 at alum.mit.edu... |I guess the question is whether people consider the ability to throw | arbitrary objects (rather than only those whose type is a subtype of | Exception) a feature or a bug. | | I'm on the fence myself: it makes translating to the jvm and clr | easier if we prohibit arbitrary objects, and it seems like a marginal | use-case, but on the other hand, well, Python can do it, so why not? 'Python can do it'??? In 2.4, only throw (old-style) classes or instances thereof or (deprecated) strings. In 2.5, I believe base exception was made new style. In 3.0, I expect subtype thereof will be enforced. tjr From mauriceling at gmail.com Mon May 21 09:14:49 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Mon, 21 May 2007 17:14:49 +1000 Subject: [pypy-dev] Implementing R (GNU S) in PyPy Message-ID: <465146E9.7000806@acm.org> Hi all, I'm attempting to implement a R (Splus language) interpreter in PyPy so that R statistical modules can be used in Python, as an alternative to R runtime + RPy platform. Currently, I have R language in EBNF, how do I go from here? Thanks in advance, maurice From lac at openend.se Mon May 21 10:39:16 2007 From: lac at openend.se (Laura Creighton) Date: Mon, 21 May 2007 10:39:16 +0200 Subject: [pypy-dev] Implementing R (GNU S) in PyPy In-Reply-To: Message from Maurice Ling of "Mon, 21 May 2007 17:14:49 +1000." <465146E9.7000806@acm.org> References: <465146E9.7000806@acm.org> Message-ID: <200705210839.l4L8dGFT028175@theraft.openend.se> In a message of Mon, 21 May 2007 17:14:49 +1000, Maurice Ling writes: >Hi all, > >I'm attempting to implement a R (Splus language) interpreter in PyPy so >that R statistical modules can be used in Python, as an alternative to R >runtime + RPy platform. Currently, I have R language in EBNF, how do I >go from here? > >Thanks in advance, >maurice >_______________________________________________ >pypy-dev at codespeak.net >http://codespeak.net/mailman/listinfo/pypy-dev Talk to people on #pypy on irc.freenode.net and I am busy with other things this day. Laura From mauriceling at gmail.com Mon May 21 12:37:47 2007 From: mauriceling at gmail.com (Maurice Ling) Date: Mon, 21 May 2007 20:37:47 +1000 Subject: [pypy-dev] Implementing R (GNU S) in PyPy In-Reply-To: <200705210839.l4L8dGFT028175@theraft.openend.se> References: <465146E9.7000806@acm.org> <200705210839.l4L8dGFT028175@theraft.openend.se> Message-ID: <4651767B.3090102@acm.org> Laura Creighton wrote: >In a message of Mon, 21 May 2007 17:14:49 +1000, Maurice Ling writes: > > >>Hi all, >> >>I'm attempting to implement a R (Splus language) interpreter in PyPy so >>that R statistical modules can be used in Python, as an alternative to R >>runtime + RPy platform. Currently, I have R language in EBNF, how do I >>go from here? >> >>Thanks in advance, >>maurice >>_______________________________________________ >>pypy-dev at codespeak.net >>http://codespeak.net/mailman/listinfo/pypy-dev >> >> > >Talk to people on #pypy on irc.freenode.net > >and I am busy with other things this day. > >Laura > > > I do wish that I can find someone to talk on #pypy. I'm in Australia and apparently is awake when everyone is off to bed. maurice From tismer at stackless.com Tue May 22 15:36:53 2007 From: tismer at stackless.com (Christian Tismer) Date: Tue, 22 May 2007 15:36:53 +0200 Subject: [pypy-dev] Licensing: [gnu.org #335118] References: Message-ID: <7E3F9EEC-5685-4C03-A472-5E847B020737@stackless.com> Hi colleagues, finally I got this answer from gnu.org which I didn't expect. I'm not sure that I understand what he's saying, I thought it is a problem. What should I reply? cheers - chris Begin forwarded message: > From: "Yoni Rabkin via RT" > Date: 22. Mai 2007 10:55:00 MESZ > To: tismer at stackless.com > Subject: [gnu.org #335118] > Reply-To: licensing at fsf.org > > > Hello, > > Please accept our apologies for the delay in getting back to you. We > rely on volunteer effort and often have difficulties keeping up. > >> My question is if there is any chance to distribute binaries which >> use >> readline, without having to change PyPy's license? > > Here is the situation as I see it: The parts of PyPy have 4 different > licenses: The Pyrex license, the Python license, the Expat license and > the Unicode license. The parts which interact with GNU Readline > seem to > be under GPL compatible licenses. So there should not be any problem > with linking to the GNU Readline library and distributing under the > GNU > GPL. > > Is that a problem? > > -- > I am not a lawyer, the above is not legal advice > > Regards, Yoni Rabkin > > > > From tismer at stackless.com Wed May 23 02:47:49 2007 From: tismer at stackless.com (Christian Tismer) Date: Wed, 23 May 2007 02:47:49 +0200 Subject: [pypy-dev] [gnu.org #335118] In-Reply-To: References: <874pm5fdeh.fsf@actcom.com> Message-ID: On 22.05.2007, at 10:55, Yoni Rabkin via RT wrote: > > Hello, > > Please accept our apologies for the delay in getting back to you. We > rely on volunteer effort and often have difficulties keeping up. > Thank you, keep up the good work. >> My question is if there is any chance to distribute binaries which >> use >> readline, without having to change PyPy's license? > > Here is the situation as I see it: The parts of PyPy have 4 different > licenses: The Pyrex license, the Python license, the Expat license and > the Unicode license. The parts which interact with GNU Readline > seem to > be under GPL compatible licenses. So there should not be any problem > with linking to the GNU Readline library and distributing under the > GNU > GPL. > > Is that a problem? Seems to be ok with us. Thank you very much! cheers - chris From jgustak at gmail.com Thu May 24 01:29:30 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Thu, 24 May 2007 01:29:30 +0200 Subject: [pypy-dev] Mini sprint in Poland Message-ID: I would like to organize mini sprint (2-3 days) in Wroclaw. It would be hold on Wroclaw University of Technology. There are possible two alternative dates: 2-3th June 16(15)-17th June It depends how fast we can organize ourselves. Is there anybody, who would like to participate? Cheers, Jakub Gutsak p.s. I have a feeling, that I spend too much time doing that kind of stuff instead of coding my project. From holger at merlinux.de Mon May 28 12:05:24 2007 From: holger at merlinux.de (holger krekel) Date: Mon, 28 May 2007 12:05:24 +0200 Subject: [pypy-dev] CFP 5th June for CC Camp [camp-orga@events.ccc.de: FINAL Call for Papers: Chaos Communication Camp 2007] Message-ID: <20070528100524.GB25045@solar.trillke> Hello all, below is an invitation and CFP for the Chaos Computer Camp in August 2007 around Berlin. (extended) deadline for proposals is 5th June. Maybe I submit a py.execnet/higher level networking talk ("building a flexible and fast platform for computer viruses" or so) and co-submit a "PyPy" one - question being: do some of us maybe want to meet up there and sprint a bit? Seems one is giving talks in airport hangars, btw, ... best, holger ----- Forwarded message from camp-orga at events.ccc.de ----- Date: Wed, 23 May 2007 22:38:53 +0200 From: camp-orga at events.ccc.de To: hpk at trillke.net Subject: FINAL Call for Papers: Chaos Communication Camp 2007 X-Spambayes-Classification: ham; 0.00 [snip personalized prae-ambel] FINAL Call for Papers: Chaos Communication Camp 2007, Berlin Chaos Communication Camp 2007 "In Fairy Dust We Trust!" August, 8th to 12th, 2007 Airport Museum Finowfurt (Finow Airport) near Berlin, Germany http://events.ccc.de/camp/2007/ Final Call for Paper Deadline: June 5th 2007, 23:59 CET =Overview= The Chaos Communication Camp is an international, five-day open-air event for hackers, builders, and makers organized by the Chaos Computer Club (CCC). The camp provides a relaxed atmosphere for free exchange of technical, social and political ideas. Discuss, sunbathe, and enjoy camping with some of the most interesting people you might ever meet. And all that with internet, power, an abundance of weird self-made gadgets, and people willing to explain them to you, right next to your tent. The Camp area is themed as a starship launch site and split into various thematic villages organized by participating groups from all over the world. Have a look at the incredible new location, right next to an old Russian military airfield (we'll launch drones and UFOs - no kidding!). All this together with vintage MiG fighter planes the Soviet airforce left there when they left East Germany and helicopters, oh my! Two huge main hangars (room for 400 people each) will feature conference tracks with lectures and presentations, while workshops will take place in a central workshop area and in the various villages. To get a first glimpse of what to expect have a look at our self-organizing participants at the Camp Wiki: http://events.ccc.de/camp/2007/Villages. Furthermore there will be a hangar for not-so-lonely hacking nights and days together with hundreds of others, and a hangar featuring art and beauty created with or without a computer. Of course there will also be a lounge with nice electronic music, German beer, great food, and a choice of caffeinated hacker drinks. The conference languages are English and German. Don't worry about language issues though - the Camp is a truly international event! Everyone will be happy to communicate with you in the languages of the Internet - broken English and 1337speak. =Topics= In general, lectures and workshops dealing with technology, ethics, science, security, art, philosophy, politics, culture and cooking are welcome. The main theme of this year's camp is the world we want to live in tomorrow. Any cool tech, research, ideas, and presentations interesting to our audience, are interesting to us. And even beyond the presentations, the camp lives from participation, so bring those gadgets, Go boards, webapps, p2p-simulators, knitted hardware protectors, drones, and lockpicking sets, and expect people to come to you wanting to learn more. =Travel= For this year's Camp there's a special travel opportunity. The Hacker Foundation (http://hackerfoundation.org) has put together a package of nothing less than legendary proportions. For US$1337 (roundtrip from the US and Canada) or ???1337 (roundtrip from Europe) you get admission to both the Camp and the Defcon hacker conference in Las Vegas just a few days before the Camp, a flight from Defcon to Frankfurt (Germany) and from there directly to Finow Airfield on a specially chartered "Hackers on a Plane" flight. This offer includes optional accomodation at the Camp. See for yourself: http://www.hackersonaplane.info/ For general travel Information see http://events.ccc.de/camp/2007/Getting_there =Lecture Requirements= Lectures are expected to be highly relevant in practice or better be darn funny. Sales droids and PR-people have been known to disappear without traces on past events. Interactive workshops are welcome. Hands-on anything are even more welcome. Final presentations for talks should be up to 60 minutes, for workshops up to 60 or 120 minutes long. Additionally, a question-and-answer period will be provided. Follow-up discussions and hands-on workshops are strongly encouraged, there will be space for such activities available outside the main lecture shelters (if you don't prefer a nice sit-in on the grass in the sun). Audio and video recordings of the lectures will be published online in various formats. All material will be available under a Creative Commons licence allowing free non-commercial redistribution of the material as long as the original credit to authors and publishers is retained. =Submissions= All proposals MUST be submitted online using our lecture submission system at https://pentabarf.cccv.de/submission/Camp+2007 . Please follow the instructions given there. You can provide papers and slides for the digital conference pack upon submission. Please make sure your submission contains all information we need to review your talk and send us everything in one go. If you have any questions regarding your submission, feel free to contact us at camp-content at cccv.de but do NOT submit your lecture via e-mail. Accepted speakers are asked to hand in slides used in their talks. Please use a well-known format for your slides. =Dates and deadlines= We've already finished our first CFP. We offer a scheduled second deadline till June, 5th. Submissions received before June 5th, 23:59 CET, will be allocated to remaining slots. Notification of acceptance will be sent by June, 27th, or earlier. Early submissions will be treated with higher priority. ----- End forwarded message ----- -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From fijal at genesilico.pl Tue May 29 18:44:19 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Tue, 29 May 2007 18:44:19 +0200 Subject: [pypy-dev] Unusual problem. Message-ID: <465C5863.50902@genesilico.pl> I want to wrap termios lib, also for RPython. everything works quite smooth except error reporting. termios.tcgetattr (and others as well) use termios.error for error reporting, which is exception which cannot be used in a translated version. Hence my ll_termios_tcgetattr raises OSError instead (it's not mentioned anywhere in documentation of termios library, nor present in any CPython tests, but well...) So than arises a problem - ll version rises OSError, while default one raises termios.error, which makes everything harder to test (ie the same RPython program behaves differently before and after translation). What is casual solution here? Also any magic wrapping doesn't work here, since termios is a builtin module. Cheers, fijal From arigo at tunes.org Wed May 30 21:46:36 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 30 May 2007 21:46:36 +0200 Subject: [pypy-dev] Unusual problem. In-Reply-To: <465C5863.50902@genesilico.pl> References: <465C5863.50902@genesilico.pl> Message-ID: <20070530194636.GA21548@code0.codespeak.net> Hi Maciek, On Tue, May 29, 2007 at 06:44:19PM +0200, Maciek Fijalkowski wrote: > termios.tcgetattr (and others as well) use termios.error for error > reporting, which is exception which cannot be used in a translated > version. Why not? Armin From fijal at genesilico.pl Fri Jun 1 08:45:54 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 01 Jun 2007 08:45:54 +0200 Subject: [pypy-dev] Using pypy for real world applications part pi/2 Message-ID: <465FC0A2.5070100@genesilico.pl> I've found an unusual problem regarding rctypes - in implementation of fcntl.ioctl, ioctl can return a string containing 0 bytes, because the length is somehow defined. Our rctypes (or implementation itself, I failed to find by now) assume that string is null terminated. The unusual thing about this is that on top of ctypes it works perfectly well. Any ideas how to fix it? I think I'll go to bed now and wait for someone else to take a look... Cheers, fijal From holger at merlinux.de Fri Jun 1 18:34:27 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 1 Jun 2007 18:34:27 +0200 Subject: [pypy-dev] PyPy fully passed its final EU review! Message-ID: <20070601163426.GI25045@solar.trillke> Hi all, we yesterday successfully passed the EU review of our 28 month PyPy EU project period! We gave almost 6 hours worth of talks and got a lot of good questions and remarks ... all of our results met appreciation! I am relieved - and my thanks to everybody involved and to everyone who helped! cheers & best, holger P.S.: As i've heard the thought from some people that the EU might "prolong funding" if PyPy is successful: this is not how it works. We had a research contract and we fullfilled that contract - if we were to have a follow up project, we'd have to submit a new research proposal and go through another application and contractual process. P.P.S.: the EU project is still not 100% final - money and budget finalisation / audits remain ... -- merlinux GmbH Steinbergstr. 42 31139 Hildesheim http://merlinux.de tel +49 5121 20800 75 (fax 77) From micahel at gmail.com Fri Jun 1 19:01:27 2007 From: micahel at gmail.com (Michael Hudson) Date: Fri, 1 Jun 2007 18:01:27 +0100 Subject: [pypy-dev] PyPy fully passed its final EU review! In-Reply-To: <20070601163426.GI25045@solar.trillke> References: <20070601163426.GI25045@solar.trillke> Message-ID: On 01/06/07, holger krekel wrote: > Hi all, > > we yesterday successfully passed the EU review of > our 28 month PyPy EU project period! Hooray! Cheers, mwh From anto.cuni at gmail.com Fri Jun 1 19:48:26 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 01 Jun 2007 19:48:26 +0200 Subject: [pypy-dev] PyPy fully passed its final EU review! In-Reply-To: <20070601163426.GI25045@solar.trillke> References: <20070601163426.GI25045@solar.trillke> Message-ID: <46605BEA.5050307@gmail.com> holger krekel wrote: > Hi all, > > we yesterday successfully passed the EU review of > our 28 month PyPy EU project period! cool :-). Congratulations to all the people involved in the review! From fijal at genesilico.pl Fri Jun 1 23:35:47 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 01 Jun 2007 23:35:47 +0200 Subject: [pypy-dev] pypy-c and pyrepl Message-ID: <46609133.8080402@genesilico.pl> Is running right now :) but on branch. From bea at changemaker.nu Fri Jun 1 23:52:35 2007 From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=) Date: Fri, 01 Jun 2007 23:52:35 +0200 Subject: [pypy-dev] PyPy fully passed its final EU review! In-Reply-To: <46605BEA.5050307@gmail.com> References: <20070601163426.GI25045@solar.trillke> <46605BEA.5050307@gmail.com> Message-ID: <46609523.40300@changemaker.nu> Hi there Antonio Cuni skrev: > holger krekel wrote: > >> Hi all, >> >> we yesterday successfully passed the EU review of >> our 28 month PyPy EU project period! >> > > cool :-). > > Congratulations to all the people involved in the review! > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > Our Project Officer at the Commission summarized their approval with stating that we had achieved "excellent technical results"! So that means a huge thanks and a big congratulation coming right back at you Anto - and all others contributing during this time! Cheers Bea From fijal at genesilico.pl Sat Jun 2 16:47:38 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 02 Jun 2007 16:47:38 +0200 Subject: [pypy-dev] Using pypy for real world applications part pi/2 In-Reply-To: <465FC0A2.5070100@genesilico.pl> References: <465FC0A2.5070100@genesilico.pl> Message-ID: <4661830A.4010009@genesilico.pl> Maciek Fijalkowski wrote: > I've found an unusual problem regarding rctypes - in implementation of > fcntl.ioctl, ioctl > can return a string containing 0 bytes, because the length is somehow > defined. Our rctypes (or implementation itself, I failed to find by now) > assume that string is null terminated. The unusual thing about this is > that on top of ctypes it works perfectly well. > > Any ideas how to fix it? > > I think I'll go to bed now and wait for someone else to take a look... > > Cheers, > fijal Actually, this was a minor bug in fcntl module, fixed now. Cheers, fijal From fijal at genesilico.pl Sat Jun 2 16:50:44 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 02 Jun 2007 16:50:44 +0200 Subject: [pypy-dev] Unusual problem. In-Reply-To: <20070530194636.GA21548@code0.codespeak.net> References: <465C5863.50902@genesilico.pl> <20070530194636.GA21548@code0.codespeak.net> Message-ID: <466183C4.5040809@genesilico.pl> Armin Rigo wrote: > Hi Maciek, > > On Tue, May 29, 2007 at 06:44:19PM +0200, Maciek Fijalkowski wrote: > >> termios.tcgetattr (and others as well) use termios.error for error >> reporting, which is exception which cannot be used in a translated >> version. >> > > Why not? > > > Armin > Because than, annotator complains about except termios.error, e: e.args[1] I fixed the bug with doing strange hacks in the module-initialization code, I've subclassed the exception and by temporarily re-raising it in module startup, I force e.args to be more regular, but this is a bit ugly trick (there is module/termios/__init__.py function startup for this purpose). Breaks for example compilemodule and it's ugly. Cheers, fijal From arigo at tunes.org Sat Jun 2 16:55:29 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 2 Jun 2007 16:55:29 +0200 Subject: [pypy-dev] Mini sprint in Poland In-Reply-To: References: Message-ID: <20070602145529.GA2634@code0.codespeak.net> Hi Jakub, On Thu, May 24, 2007 at 01:29:30AM +0200, Jakub Gustak wrote: > I would like to organize mini sprint (2-3 days) in Wroclaw. > > It would be hold on Wroclaw University of Technology. > There are possible two alternative dates: > 2-3th June > 16(15)-17th June > It depends how fast we can organize ourselves. Sorry for not coming back to you earlier on this. I was (probably just like many others) quite too much involved in preparing the EU review for this... I will also need to rest now, so I'm not up to travelling to Wroclaw at the moment. It seems that you didn't get much feedback about the sprint... I can confirm that we plan a PyPy-related sprint at EuroPython in about 1 month now - after EuroPython, so 12-14 July. See http://wiki.python.org/moin/EuroPython2007Sprints. A bientot, Armin. From arigo at tunes.org Sat Jun 2 16:57:05 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 2 Jun 2007 16:57:05 +0200 Subject: [pypy-dev] Unusual problem. In-Reply-To: <466183C4.5040809@genesilico.pl> References: <465C5863.50902@genesilico.pl> <20070530194636.GA21548@code0.codespeak.net> <466183C4.5040809@genesilico.pl> Message-ID: <20070602145705.GB2634@code0.codespeak.net> Hi Maciek, On Sat, Jun 02, 2007 at 04:50:44PM +0200, Maciek Fijalkowski wrote: > Because than, annotator complains about except termios.error, e: e.args[1] Ah, because you need to carry an error code. That's the same as with OSError, so you should fix it in the same (hackish) way - grep for OSError in pypy/annotation/ :-) A bientot, Armin. From santagada at gmail.com Sun Jun 3 17:34:44 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Sun, 3 Jun 2007 12:34:44 -0300 Subject: [pypy-dev] Problems with parsing Message-ID: I'm having 2 problems with parsing js files, so i'm sending this email in a call for help: 1) My string regex is not working the one i'm using now: stringliteral : | ; SINGLESTRING : "'([^']|\\')*'" ; DOUBLESTRING : "\"([^\"]|\\\")*\"" ; don't work with this valid js string: "\'\"\\" 2) The parsing module is reaching recursion limit on /pypy/lang/js/ test/ecma/FunctionObjects/15.3.1.1-3.js if you uncomment the code. I get the idea of where the problem lies, we have there a function with 2k params, but still, it works on spidermonkey and I think I should support this. you can test this by going to the dir and type "py.test --ecna 15.3.1.1-3.js" to see what happens. thanks in advance, -- Leonardo Santagada "If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family anatidae on our hands." - Douglas Adams From fijal at genesilico.pl Wed Jun 6 18:58:12 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Wed, 06 Jun 2007 18:58:12 +0200 Subject: [pypy-dev] PyPy sync next week? Message-ID: <4666E7A4.9030407@genesilico.pl> I know you're quite tired right now and all, but I think it would be good to make some kind of pypy-sync next week or even the week after (we need to decide on sprint topics anyway). Particularly I would like to know people's opinion on pypy's scope. What we would like to support, what we would like to drop support for, what are the short-term project directions etc. What do you think? Cheers, fijal From delza at livingcode.org Wed Jun 6 19:55:34 2007 From: delza at livingcode.org (Dethe Elza) Date: Wed, 6 Jun 2007 10:55:34 -0700 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <4666E7A4.9030407@genesilico.pl> References: <4666E7A4.9030407@genesilico.pl> Message-ID: On 6-Jun-07, at 9:58 AM, Maciek Fijalkowski wrote: > I know you're quite tired right now and all, but I think it would be > good to make some kind of pypy-sync next week or even the week > after (we > need to decide on sprint topics anyway). I'm quite new to this list. Could you explain what a sync is? Would it be face-to-face, via this list, or IRC? > Particularly I would like to know people's opinion on pypy's scope. > What > we would like to support, what we would like to drop support for, what > are the short-term project directions etc. I'm quite interested in the scope question. I would love to see Python adopt Erlang-like concurrency primitives, but that doesn't appear to be a direction Guido and the C-Python team are willing to take, making PyPy my best hope for this. Thanks! --Dethe Simple things should be declarative. Complex things should be procedural. --Adam Bosworth From cfbolz at gmx.de Wed Jun 6 23:03:37 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Wed, 06 Jun 2007 23:03:37 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: References: <4666E7A4.9030407@genesilico.pl> Message-ID: <46672129.8000603@gmx.de> Dethe Elza wrote: > On 6-Jun-07, at 9:58 AM, Maciek Fijalkowski wrote: >> I know you're quite tired right now and all, but I think it would be >> good to make some kind of pypy-sync next week or even the week >> after (we >> need to decide on sprint topics anyway). > > I'm quite new to this list. Could you explain what a sync is? Would > it be face-to-face, via this list, or IRC? A sync-meeting is a 30 minute IRC meeting where active PyPy developers meet to discuss current issues. There is a fixed part, where everybody posts three prepared lines (LAST, NEXT, BLOCKERS) about what he worked in the last days, what he plans to work on and what is blocking his work. Afterwards there are topics that vary between the meetings. Feel free to attend! >> Particularly I would like to know people's opinion on pypy's scope. This sounds more like a mailing list discussion to me, since it can quickly get out of hand. I would propose to have a smaller discussion at a sync-meeting about the plans of the individuals in the next months, to get a bit of an overview. Afterwards we can start some common planning of what good next steps would be (which would include a rough sprint planning). How does that sound? >> What >> we would like to support, what we would like to drop support for, what >> are the short-term project directions etc. > > I'm quite interested in the scope question. I would love to see > Python adopt Erlang-like concurrency primitives, but that doesn't > appear to be a direction Guido and the C-Python team are willing to > take, making PyPy my best hope for this. If we discuss scope we are probably discussing how to reduce our scope, since we are doing a lot already. So if you want any new features, you would have to implement them yourself. I don't know Erlang, but it's possible that you can just implement this stuff on top of PyPy's Python interpreter with the help of stackless features rather easily. Cheers, Carl Friedrich From bea at changemaker.nu Thu Jun 7 15:20:06 2007 From: bea at changemaker.nu (=?UTF-8?B?QmVhdHJpY2UgRMO8cmluZw==?=) Date: Thu, 07 Jun 2007 15:20:06 +0200 Subject: [pypy-dev] [Fwd: Call for International Journal Papers,] Message-ID: <46680606.6010609@changemaker.nu> Hi there Anyone (still) interested in writing papers ;-)? Cheers Bea -------------- next part -------------- An embedded message was scrubbed... From: Subject: Call for International Journal Papers, Date: Thu, 7 Jun 2007 12:27:56 +0900 Size: 12825 Url: http://codespeak.net/pipermail/pypy-dev/attachments/20070607/64cda13f/attachment.eml From santagada at gmail.com Thu Jun 7 20:34:36 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 7 Jun 2007 15:34:36 -0300 Subject: [pypy-dev] play1.pypy.org not working Message-ID: I tried it today and found a 502 proxy error. -- Leonardo Santagada "If it looks like a duck, and quacks like a duck, we have at least to consider the possibility that we have a small aquatic bird of the family anatidae on our hands." - Douglas Adams From fijal at genesilico.pl Sun Jun 10 20:44:50 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sun, 10 Jun 2007 20:44:50 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <4666E7A4.9030407@genesilico.pl> References: <4666E7A4.9030407@genesilico.pl> Message-ID: <466C46A2.9040108@genesilico.pl> Maciek Fijalkowski wrote: > I know you're quite tired right now and all, but I think it would be > good to make some kind of pypy-sync next week or even the week after (we > need to decide on sprint topics anyway). > > Particularly I would like to know people's opinion on pypy's scope. What > we would like to support, what we would like to drop support for, what > are the short-term project directions etc. > > What do you think? > > Cheers, > fijal > Ok, so what about usual Thursday 17:00? From anto.cuni at gmail.com Mon Jun 11 10:18:58 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Mon, 11 Jun 2007 10:18:58 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <466C46A2.9040108@genesilico.pl> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> Message-ID: <466D0572.8050205@gmail.com> Maciek Fijalkowski wrote: > Ok, so what about usual Thursday 17:00? Fine for me. ciao Anto From lac at openend.se Mon Jun 11 10:26:12 2007 From: lac at openend.se (Laura Creighton) Date: Mon, 11 Jun 2007 10:26:12 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: Message from Antonio Cuni of "Mon, 11 Jun 2007 10:18:58 +0200." <466D0572.8050205@gmail.com> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> <466D0572.8050205@gmail.com> Message-ID: <200706110826.l5B8QCu6000900@theraft.openend.se> In a message of Mon, 11 Jun 2007 10:18:58 +0200, Antonio Cuni writes: >Maciek Fijalkowski wrote: > >> Ok, so what about usual Thursday 17:00? > >Fine for me. > >ciao Anto me too. Laura From arigo at tunes.org Mon Jun 11 12:47:26 2007 From: arigo at tunes.org (Armin Rigo) Date: Mon, 11 Jun 2007 12:47:26 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <466C46A2.9040108@genesilico.pl> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> Message-ID: <20070611104726.GA28732@code0.codespeak.net> Hi Maciek, On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote: > Ok, so what about usual Thursday 17:00? Ok. Armin From elmo13 at jippii.fi Wed Jun 13 14:55:54 2007 From: elmo13 at jippii.fi (=?UTF-8?B?RWxtbyBNw6RudHluZW4=?=) Date: Wed, 13 Jun 2007 15:55:54 +0300 Subject: [pypy-dev] About adding a blist implementation... Message-ID: <466FE95A.6020805@jippii.fi> Fijal mentioned blist on irc and I got interested and read about it. This really seems the kind of thing I might even accomplish in a reasonable time, so if no one hasn't started on it, I'd like to try. I'm somewhat familiar with the multilist interface (seems very clear), so the challenge for me would be the [efficient] algorithmic and rpythonic coding style, the algorithm itself, and teaching myself all of that. It is very much possible that I will again abandon this as I have done with some other "projects", but I'd like to finally do something with/for pypy. If I get something usedul done, I'll report back and we'll think about adding it to the official repository... Elmo From arigo at tunes.org Thu Jun 14 18:45:54 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 14 Jun 2007 18:45:54 +0200 Subject: [pypy-dev] Work plan for PyPy Message-ID: <20070614164554.GA654@code0.codespeak.net> Hi all, Here is a summary of what we discussed today on #pypy-sync. We tried to list the tasks that now lie ahead of us. There are a number of relatively independent tasks, but there is also a knot of dependent tasks that we'll have to untie bit by bit. RCtypes and the extension compiler are somewhat of a blocker for other tasks. A controversial solution is to drop them. This means: * To support calling external libraries, we would instead shift the focus on supporting the full ctypes at app-level. This means that we could support out of the box the same ctypes-based pure Python modules as CPython. There are issues here (e.g. is the regular ctypes really suitable to wrap large libraries?) but they are not worse than with rctypes. * Of course, we keep the general support for compiling stand-alone RPython programs, even if rctypes dies. * In RPython, fijal worked on a more basic replacement for rctypes, called rtti (almost finished already). It's a PyPy-specific interface to call external functions in RPython. Calls written in this interface can be compiled to good C code, but testing is a bit less convenient than with rctypes because rtti calls cannot be directly executed on top of CPython. * The CPy Object Space is going away initially too because it is based on rctypes. It should not be too hard to port it to rtti in order to continue supporting compiling extension modules for CPython. We need to think a bit more here - how can we compile ext modules for Jython and IronPython? - but it's likely not a high priority at the moment. Any comments or objections? Are there people that already make heavy use of rctypes, or the extension compiler? The graph of dependent tasks ("A --- B" means that B depends on A): merge rtti branch ---- app-level ctypes ---- kill rctypes and extcompiler \ / \ \-- port our modules --- kill keepalives \ \ -- pyrepl -- moving GCs / _______________________________/ / / malloc cleanup ----- gc transform polishing / \ fix weakrefs -----/ \ \ -- c/llvm backend sharing Additionally, some of the independent tasks mentioned: * move some app-level parts of the Python interpreter to interp-level * complete the support for Python 2.5 * switch to LLVM 2.0 * port the stackless transform to ootypesystem * JIT work A bientot, Armin. From fijal at genesilico.pl Thu Jun 14 19:28:30 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 14 Jun 2007 19:28:30 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070614164554.GA654@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> Message-ID: <46717ABE.8010708@genesilico.pl> Armin Rigo wrote: > * In RPython, fijal worked on a more basic replacement for rctypes, > called rtti (almost finished already). It's a PyPy-specific interface > to call external functions in RPython. Calls written in this > interface can be compiled to good C code, but testing is a bit less > convenient than with rctypes because rtti calls cannot be directly > executed on top of CPython. > > This is actually rffi, not rtti (like Rpython Foreign Function Interface). There are probably few other, unmentioned issues: * What we do with ootypesystem backends and external functions? Right now this is implemented by backends which tends to be a bit ugly implementation. My idea would be to have backend-sensitive implementations which access backend-specific RPython functions for accessing underlaying platform classes/functions/whatever. * Platform specifics - how much do we care about portability of a generated C code? I've got several approaches in mind: - Have special kind of if which renders to #ifdef in the C code (hard) - There is a caching mechanism for basic types (size_t, int, etc.) which renders sizes of types per-platform. This works for multiple platforms, but renders generated C code non-portable. unsure if this is a problem. Cheers, fijal From anto.cuni at gmail.com Fri Jun 15 01:01:17 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 15 Jun 2007 01:01:17 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070614164554.GA654@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> Message-ID: <4671C8BD.4050707@gmail.com> Armin Rigo wrote: > Hi all, Hi, sorry, I completely forgot the meeting :-(. I was even at home and online at that time, but I missed it. At least I've read the logs :-). Here are some notes from my side. My personal short-term plan is to work a bit to make gencli more or less usable as a general .NET compiler, in particular for generating DLLs to be imported by others .NET programs. I personally would like not to work on rpython too much, but I have to convince my professor first :-). After having convinced him, I would like to work on the JIT, in particular to port it to ootype (and then add gencli support). > * The CPy Object Space is going away initially too because it is based > on rctypes. It should not be too hard to port it to rtti in order to > continue supporting compiling extension modules for CPython. We need > to think a bit more here - how can we compile ext modules for Jython > and IronPython? - but it's likely not a high priority at the moment. Armin, I read that you asked how hard would be to generate extension modules for Jython; the same question apply to IronPython. I think that the easiest way in the short term is to generate plain Java/.NET modules that can be imported by any Java/C# program, and rely on Jython/IronPython's mechanism to import it (thus, not relying on the CPy objectspace at all). I've already started in this direction, regarding gencli: the name of the front-end is SilveRPython, though I think I will rename it to avoid confusion with microsoft silverlight. Currently it mostly works, but I have to convince the rtyper not to mangle the name of the classes and the methods, because they need to be publicly available from the outside. > Additionally, some of the independent tasks mentioned: > > * port the stackless transform to ootypesystem I also would have spotted this point if I were at the meeting :-). I think it could be a nice topic for sprint, couldn't it? Finally, a note which is not related with the work plan but to a question that appeared in the logs; someone asked whether we want to "sell" RPython as a stand-alone product. Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, RPython was perceived as the only (or the most relevant) usable product of the pypy project. I tried hardly to explain that rpython is basically only an accident and not very usable, but people are still impressed by the 300000% speedup. It would be interesting to know if this perception is world-wide or only limited to the attendants of my talk (maybe because I didn't stress enough the pros of the other pypy goals or the cons of rpython). ciao Anto From anto.cuni at gmail.com Fri Jun 15 01:06:24 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Fri, 15 Jun 2007 01:06:24 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <46717ABE.8010708@genesilico.pl> References: <20070614164554.GA654@code0.codespeak.net> <46717ABE.8010708@genesilico.pl> Message-ID: <4671C9F0.5030002@gmail.com> Maciek Fijalkowski wrote: > * What we do with ootypesystem backends and external functions? Right > now this is implemented by backends which tends to be a bit ugly > implementation. My idea would be to have backend-sensitive > implementations which access backend-specific RPython functions for > accessing underlaying platform classes/functions/whatever. I agree. Supporting external functions directly in the backend is easier, but now that gencli can call .NET code we don't need those functions to be external anymore, we can just provide a real implementation for them. The problem is that at the moment genjvm can't call java functions, so this solution would not work for it. ciao Anto From janzert at janzert.com Fri Jun 15 02:28:02 2007 From: janzert at janzert.com (Janzert) Date: Thu, 14 Jun 2007 20:28:02 -0400 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <4671C8BD.4050707@gmail.com> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> Message-ID: Antonio Cuni wrote: [snip] > I tried hardly to explain that rpython is basically > only an accident and not very usable, but people are still impressed by > the 300000% speedup. > It would be interesting to know if this perception is world-wide or only > limited to the attendants of my talk (maybe because I didn't stress > enough the pros of the other pypy goals or the cons of rpython). > > ciao Anto I certainly don't know if it's universal. I do know that the extcompiler with a time sensitive module I ported to rpython is the only real practical use for pypy that I have had. I don't have the interest or time to offer to maintain the extcompiler (only used it for a personal hobby project which is on the back burner for the moment anyway) and after getting longs to work in the cpy object space I do appreciate the pain it may be to maintain. Janzert P.S. There is a certain irony that ShedSkin is growing a rudimentary extension compiler at the moment. :} From lac at openend.se Fri Jun 15 07:53:02 2007 From: lac at openend.se (Laura Creighton) Date: Fri, 15 Jun 2007 07:53:02 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: Message from Antonio Cuni of "Fri, 15 Jun 2007 01:01:17 +0200." <4671C8BD.4050707@gmail.com> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> Message-ID: <200706150553.l5F5r26d031897@theraft.openend.se> In a message of Fri, 15 Jun 2007 01:01:17 +0200, Antonio Cuni writes: > >Finally, a note which is not related with the work plan but to a >question that appeared in the logs; someone asked whether we want to >"sell" RPython as a stand-alone product. >Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, >RPython was perceived as the only (or the most relevant) usable product >of the pypy project. I tried hardly to explain that rpython is basically >only an accident and not very usable, but people are still impressed by >the 300000% speedup. >It would be interesting to know if this perception is world-wide or only >limited to the attendants of my talk (maybe because I didn't stress >enough the pros of the other pypy goals or the cons of rpython). > >ciao Anto I don't think it is you. I think that most people are not very sophisticated in their appreciation for computer langauges. Culturally deprived, poor things. :-) Maybe we need to run some 'pypy appreciation courses'. :-) I suspect the designers of the Ferrari have the same problem. They probably want to discuss engine-design and Italian styling with their would-be customers. Laura From holger at merlinux.de Fri Jun 15 08:30:17 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 15 Jun 2007 08:30:17 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <4671C8BD.4050707@gmail.com> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> Message-ID: <20070615063017.GK3311@solar.trillke> Hi Anto, all, On Fri, Jun 15, 2007 at 01:01 +0200, Antonio Cuni wrote: > Finally, a note which is not related with the work plan but to a > question that appeared in the logs; someone asked whether we want to > "sell" RPython as a stand-alone product. > Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, > RPython was perceived as the only (or the most relevant) usable product > of the pypy project. I tried hardly to explain that rpython is basically > only an accident and not very usable, but people are still impressed by > the 300000% speedup. > It would be interesting to know if this perception is world-wide or only > limited to the attendants of my talk (maybe because I didn't stress > enough the pros of the other pypy goals or the cons of rpython). FYI: Last week i gave a talk at the german Zope annual meeting (extradoc/talk/dzug2007/dzug2007.txt). To briefly summarize: I talked about the .NET backend, python interpreter prototypes/speed/status and the general usability of PyPy and its JIT for implementing other languages. I also noted that the extension module area is currently in flux. Questions at the talk then were mostly about the Python Interpreter, e.g. how to use persistence/Transparent proxy features e.g. with ZODB. Jim Fulton also noted that zope only needs few extension modules and often has pure python versions of its C-coded modules. There wasn't a strong focus on RPython or the ext-compiler but there was e.g. interest (prior to the talk) to port the Zope TAL interpreter to RPython in order to use it with CPython. Here is a thought: if our Python Interpreter becomes more usable and can be used as a replacement for CPython in more cases, people *can* write RPython modules for speed reasons if they want to. Practically, i guess, it'd then be important to untangle the compilation of such written-for-speed extension modules from the translation of the Python Interpreter itself. (this all apart from the question if a better JIT could obsolete the aim for writing RPython modules). best, holger From fijal at genesilico.pl Fri Jun 15 11:11:27 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 15 Jun 2007 11:11:27 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070615063017.GK3311@solar.trillke> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> Message-ID: <467257BF.5060709@genesilico.pl> I would be very, very careful about what people talk about. Even if they talk about RPython and speed, they really don't know what they're talking about. There are some reasons why they use python and not ocaml or haskell for their purposes. One of the reasons is that it's easy to write well-working python program. With rpython it's not that easy, and we don't explain it in talks, how hard it is. Even giving enough effort to the translation toolchain error reporting, type inference problems are always hard to read and debug. That's why I think RPython will never become a language of choice of most of the people (despite few guys who know what they're doing and why). Also I (personally) think that good enough JIT would be a way better place to start writing speedy programs. You write it jit-friendly, you get speedup. From my POV maintaining all of parts required to use RPython as a general purpose language is a bit overkill, and people are really interested in parts which they yet don't know they're interested in. Cheers, fijal From holger at merlinux.de Fri Jun 15 11:19:26 2007 From: holger at merlinux.de (holger krekel) Date: Fri, 15 Jun 2007 11:19:26 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <467257BF.5060709@genesilico.pl> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> Message-ID: <20070615091926.GO3311@solar.trillke> On Fri, Jun 15, 2007 at 11:11 +0200, Maciek Fijalkowski wrote: > I would be very, very careful about what people talk about. quick question: your mail appears in my Mutt as a reply to my previous mail ("Hi Anto, all") but your mail is not meant as relating to it specifically, right? holger From fijal at genesilico.pl Fri Jun 15 11:24:24 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 15 Jun 2007 11:24:24 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070615091926.GO3311@solar.trillke> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> <20070615091926.GO3311@solar.trillke> Message-ID: <46725AC8.9010407@genesilico.pl> holger krekel wrote: > On Fri, Jun 15, 2007 at 11:11 +0200, Maciek Fijalkowski wrote: > >> I would be very, very careful about what people talk about. >> > > quick question: your mail appears in my Mutt as a reply to my > previous mail ("Hi Anto, all") but your mail is not meant > as relating to it specifically, right? > > holger > > :. > > Yes. Just being too lazy to answer to a proper mail. From arigo at tunes.org Fri Jun 15 11:55:56 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 15 Jun 2007 11:55:56 +0200 Subject: [pypy-dev] Pygame viewer Message-ID: <20070615095556.GA16579@code0.codespeak.net> Hi all, I've refactored a bit the Pygame viewer. Can people with OS/X or Windows give it a try? It is currently in the branch pypy/branch/graphserver-dist/. I'll merge it when I'm sure there are no platform issues. It is supposed to support the following features: * all pygame usage is done in a subprocess, which cannot take away the whole parent process if it segfaults. * regular usage should continue to work as before, e.g. t.view() in translatorshell.py, as well as the translate.py extended pdb+ commands like "classhier" to show the class hierarchy. * closing the Pygame window now really closes it and shuts down the subprocess, instead of the previous hack that left the title bar of the window around just because it's not always possible to re-open Pygame in the same process. * the tool should be really stand-alone now: if you check out only http://codespeak.net/svn/pypy/branch/graphserver-dist/dotviewer then the dotviewer.py included there should have no pypy or py lib dependency. It should also contact the CGI script on codespeak automatically, to compile graphs if 'dot' is not installed locally. It won't install Pygame for you nor brew your coffee, though. * I've kept the best for the end: *all* regular usage (t.view() and anything display graphs from translate.py) works remotely too. Here's an example setup (pick your own port number instead of 8515): - on your local machine, run: pypy/bin/dotviewer.py --server 127.0.0.1:8515 - connect to a remote machine, allowing the remote side to contact your local dotviewer: ssh -R8515:localhost:8515 machinename - set the $GRAPHSERVER env var on the remote machine to point to the redirected port: export GRAPHSERVER=:8515 - then any graph-viewing command you execute while $GRAPHSERVER is set will reach your local dotviewer and basically appear to work completely transparently. - to automate steps 2 and 3, I've put the following entry in my local .ssh/config: Host wyvern HostName wyvern.cs.uni-duesseldorf.de Port 922 RemoteForward 8515 127.0.0.1:8515 (hence, please pick another port number, because if two people log in trying to redirect the port there is a conflict :-) and I've added the "export GRAPHSERVER=:8515" line to my .bashrc on wyvern. A bientot, Armin. From faassen at startifact.com Fri Jun 15 12:41:06 2007 From: faassen at startifact.com (Martijn Faassen) Date: Fri, 15 Jun 2007 12:41:06 +0200 Subject: [pypy-dev] PyPy and production use In-Reply-To: <4671C8BD.4050707@gmail.com> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> Message-ID: Hey, I'm changing the title of this thread as this is another topic. It's one I care about as an observer of PyPy. Antonio Cuni wrote: [snip] > Finally, a note which is not related with the work plan but to a > question that appeared in the logs; someone asked whether we want to > "sell" RPython as a stand-alone product. I think you should. I'll explain why below. > Last sunday I gave a PyPy talk at Pycon italy and, surprisingly enough, > RPython was perceived as the only (or the most relevant) usable product > of the pypy project. I tried hardly to explain that rpython is basically > only an accident and not very usable, but people are still impressed by > the 300000% speedup. > It would be interesting to know if this perception is world-wide or only > limited to the attendants of my talk (maybe because I didn't stress > enough the pros of the other pypy goals or the cons of rpython). I got this impression back at the presentation last year at EuroPython. I therefore think this impression is more widespread than just your audience in Italy. This is not because you've communicated the state or future potential of PyPy wrongly, but because the audience likely has a different perspective than you do. I realize that PyPy is much more than RPython and that RPython is somewhat incidental to the project. For people working on production projects right now, these other features are less relevant at this point in time, however. Let's take PyPy's generated interpreter. If I'm currently using the CPython platform (as most Python programmers are), choosing the PyPy-C interpreter would mean that I have far less extension module support, and a slightly slower interpreter. I'd need some pretty unusual use cases currently for the benefits of PyPy-C to outweigh its drawbacks. The PyPy developers themselves of course *do* have unusual use cases. They want to experiment with interpreter generation, different language features, different platforms, and so on. I expect this to change over time. Once the PyPy (generated) JIT matures, PyPy will become a lot more attractive. If you could then somehow figure out a way to use CPython modules with PyPy's interpreter you'd get even more people. Right now, unless I really really need some unusual language feature in my production project, I don't have a motivation to use PyPy's interpreter. Maturing the PyPy interpreter will take quite a lot of time. You've come a long way and it's quite impressive, but at the same time I don't expect to be using a PyPy-generated interpreter to develop web applications any time soon. Now RPython. RPython generated modules are usable from CPython. This means I can write relatively plain Python code, generate a module, and start using this module in my web application today. The benefit is vastly increased speed for my module. Using RPython with CPython is attractive for the same reasons Pyrex is attractive, with the benefits that it's more well-engineered than Pyrex. Pyrex is currently more mature to use. Using RPython in this way is a bit of a struggle still in comparison. This is however tool support that I feel can improve relatively quickly. An open source projects needs mindshare. It needs users. PyPy is an open source project, and I'd recommend getting any users you can get. Granted, you'll get opportunists like me who are only there for production benefits. These people are less interested in language experimentation and new features - they're there for the speed primarily: the main thing that PyPy, through RPython, can offer to a Python programmer today. But if your aim is to provide a platform that is suitable for production use, you *need* opportunists like that. They can help mature the platform. Open source projects thrive by attracting developers that are in it for their own (enlightened) self-interest. Some of these opportunists might actually help out beyond just RPython. They might in fact be interested in such language experimentation. It might however not be a priority for them. For instance, I'm interested, but I need an excuse to spend more time on PyPy. Production benefits are a great excuse. Using RPython extensions with CPython is the shortest route to production benefits. Besides gaining new users, supporting RPython as a product will help PyPy in other ways as well. It will get you more module support. New users will come in and help you write more extension modules in RPython. They'll be doing this initially so they can use them in CPython, but it's a nice side effect. Currently I'm working with Godefroid Chapelle (well, he's been doing the most work so far) to try to make an RPython version of a web templating engine (Zope's page template engine). We hope we will be able to use this technology to speed up the generation of web templates. We'll be using this within a CPython server. We see the potential of PyPy. Godefroid made the observation that a web templating engine implements an interpreter. PyPy has technology to generate a JIT for arbitrary interpreters in RPython. This means that in the future we might be able to have a JIT-ed templating language, hopefully boosting performance even more. (besides the huge coolness factor) Just the potential for future JITting isn't motivating enough to experiment with PyPy right now though. The ability to use RPython with CPython is the critical bit. This is what gives us the hope we'll get a payoff in our production work. The JIT is a nice bonus that might become important later. So if people conclude that RPython is currently the most relevant product of the PyPy project for them, it doesn't mean that they haven't understood the potential of PyPy in the future. It's because they're thinking about how PyPy is relevant to their own work today. PyPy has always been a unique project: it's a research project and it's an open source project aiming for real-world use. It takes ideas from ivory towers and applies them to a pragmatic, real-world language like Python. I think this balance between pragmatic and research goals is one of this project's greatest strengths. I encourage PyPy developers to continue to embrace both goals. Regards, Martijn From faassen at startifact.com Fri Jun 15 12:45:19 2007 From: faassen at startifact.com (Martijn Faassen) Date: Fri, 15 Jun 2007 12:45:19 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <200706150553.l5F5r26d031897@theraft.openend.se> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <200706150553.l5F5r26d031897@theraft.openend.se> Message-ID: Laura Creighton wrote: [snip] > I don't think it is you. I think that most people are not very > sophisticated in their appreciation for computer langauges. > Culturally deprived, poor things. :-) Maybe we need to run some 'pypy > appreciation courses'. :-) I suspect the designers of the Ferrari have > the same problem. They probably want to discuss engine-design and > Italian styling with their would-be customers. I know you're joking here. Still, I'll reiterate the point I tried to make in my longer mail just now: people might in fact be quite cultured in their appreciation for computer language technology and still approach you about RPython. They just have different goals. Don't underestimate your audience. People might appreciate Ferraris but still not want to drive one to work every day. Regards, Martijn From faassen at startifact.com Fri Jun 15 13:04:32 2007 From: faassen at startifact.com (Martijn Faassen) Date: Fri, 15 Jun 2007 13:04:32 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <467257BF.5060709@genesilico.pl> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> Message-ID: Maciek Fijalkowski wrote: > I would be very, very careful about what people talk about. > > Even if they talk about RPython and speed, they really don't know what > they're talking about. I understand that you need to be careful about not overselling RPython. I would also suggest you be careful about underestimating your audience. You just implied I don't know what I'm talking about because I'm interested in using RPython for speed. While my perspective on PyPy is definitely quite different than yours, I do think I have a reasonable picture of what it is about. [snip] > Also I (personally) think that good enough JIT would be a way better > place to start writing speedy programs. You write it jit-friendly, you > get speedup. People will then rightfully ask you the question when you think they can benefit from a JITed interpreter in their Zope/Django/Pylons projects. RPython, immature as the toolchain might be, is at least somewhat usable today. You could tell me that I'd be more productive if I contributed to the JIT generator, but then I'd go away again and you'd lose a potential contributor. If I can speed up my templating language using RPython I might stick around. I realize that I personally am of small potential value to the project, but who knows who else you might draw in this way? > From my POV maintaining all of parts required to use RPython as a > general purpose language is a bit overkill, and people are really > interested in parts which they yet don't know they're interested in. Two points: * in an open source project, others might be helping you maintain this toolchain, so the cost might be relatively little to you. * you will likely still have some maintenance cost. This could be an investment: maintaining something not part of your core goal may draw in sufficient new contributors to actually benefit the core goals as well. I realize I'm speaking from a quite different cultural perspective than many PyPy developers. I also realize I'm arguing from a self-interested perspective. I also genuinely believe that taking these other perspectives into account may help your project. So please cut me some slack here. :) I don't think you have to be worried on the short term about being flooded with clueless people taking up your time. Even if that happened, I'd consider it a luxury problem, as some percentage of new people would not be clueless and valuable to your project. Regards, Martijn From lypanov at mac.com Fri Jun 15 13:16:06 2007 From: lypanov at mac.com (Alexander Kellett) Date: Fri, 15 Jun 2007 13:16:06 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> Message-ID: <1BB835BB-691C-480A-A941-86454586AB4A@mac.com> On 15 Jun 2007, at 13:04, Martijn Faassen wrote: > You could tell me that I'd be more productive if I contributed to the > JIT generator, but then I'd go away again and you'd lose a potential > contributor. If I can speed up my templating language using RPython I > might stick around. I realize that I personally am of small potential > value to the project, but who knows who else you might draw in this > way? It seems like many are in this boat. Are there really such difficult stumbling blocks involved in making a (r)python lint and improving the current error messages while at the same time letting users of the system build a FAQ on the more common issues based in their experience? If not, why not branch off now? Alex --- Alexander Kellett PGP - 0x6BFA8EF3, FPR: DA65 D6DE 56A9 D715 EFB6 A948 B2EF 6622 6BFA 8EF3 From fijal at genesilico.pl Fri Jun 15 14:07:33 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Fri, 15 Jun 2007 14:07:33 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> Message-ID: <46728105.1060607@genesilico.pl> Martijn Faassen wrote: > Maciek Fijalkowski wrote: > >> I would be very, very careful about what people talk about. >> >> Even if they talk about RPython and speed, they really don't know what >> they're talking about. >> > > I understand that you need to be careful about not overselling RPython. > I would also suggest you be careful about underestimating your audience. > You just implied I don't know what I'm talking about because I'm > interested in using RPython for speed. While my perspective on PyPy is > definitely quite different than yours, I do think I have a reasonable > picture of what it is about. > Sorry for underestimating you :) If I didn't add a disclaimer, I should at least - I meant *most* of the people are not that interested in speed, I kind of believe that you are the one who might be. > [snip] > >> Also I (personally) think that good enough JIT would be a way better >> place to start writing speedy programs. You write it jit-friendly, you >> get speedup. >> > > People will then rightfully ask you the question when you think they can > benefit from a JITed interpreter in their Zope/Django/Pylons projects. > RPython, immature as the toolchain might be, is at least somewhat usable > today. > the real problem is that RPython is quite mature for building stand-alone applications, while it's usefulness as a CPython extension generator is somehow debatable. There are few steps needed for that, like improving our reference counting GC implementation, moving from rctypes to new interface and so on and so on. This would greatly improve performance and usability. These are not small and uninvolved tasks - it's very much in the core. I think we would welcome someone who will do that, but seems unlikely, as first someone from the core team would need to spend a lot of effort on that. It's not question about whether it's worth that - I think it is. The open question is who is willing to spend time on this - if there is someone, good, I'll not try to stop him. > You could tell me that I'd be more productive if I contributed to the > JIT generator, but then I'd go away again and you'd lose a potential > contributor. If I can speed up my templating language using RPython I > might stick around. I realize that I personally am of small potential > value to the project, but who knows who else you might draw in this way? > I'm not saying this. I wouldn't be productive contributing to JIT. > > Two points: > > * in an open source project, others might be helping you maintain this > toolchain, so the cost might be relatively little to you. > > * you will likely still have some maintenance cost. This could be an > investment: maintaining something not part of your core goal may draw in > sufficient new contributors to actually benefit the core goals as well. > > I realize I'm speaking from a quite different cultural perspective than > many PyPy developers. I also realize I'm arguing from a self-interested > perspective. I also genuinely believe that taking these other > perspectives into account may help your project. So please cut me some > slack here. :) > It's not about cultural differencies I think. It's more about my own time and how I would like to use it. I would rather spend my own time on making extension modules or making CPython's extension modules work on top of PyPy than on extcompiler, which needs some more work. From simon at arrowtheory.com Sat Jun 16 02:44:54 2007 From: simon at arrowtheory.com (Simon Burton) Date: Fri, 15 Jun 2007 17:44:54 -0700 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070614164554.GA654@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> Message-ID: <20070615174454.6e87fb1a.simon@arrowtheory.com> On Thu, 14 Jun 2007 18:45:54 +0200 Armin Rigo wrote: > > Any comments or objections? Are there people that already make heavy > use of rctypes, or the extension compiler? Indeed. Mainly rctypes. As well as an ever expanding use of libc, we recently have interfaced to libSDL and cairo for some very funky rpython graphics apps. And we use libpython so that we can extract stuff from python modules. Also, we do byte shuffling with rctypes: reading binary data to/from buffers, etc. I hope we can do all this with rffi, but there are two really cool things about rctypes: 1) it runs on cpython (our main app takes 30minutes+ to compile) 2) it has an automatic code generator (ctypes can autogenerate python wrappers from C header files) Simon. From fijal at genesilico.pl Sat Jun 16 13:03:16 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 16 Jun 2007 13:03:16 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070615174454.6e87fb1a.simon@arrowtheory.com> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> Message-ID: <4673C374.2060806@genesilico.pl> Simon Burton wrote: > On Thu, 14 Jun 2007 18:45:54 +0200 > Armin Rigo wrote: > > >> Any comments or objections? Are there people that already make heavy >> use of rctypes, or the extension compiler? >> > > Indeed. Mainly rctypes. > > As well as an ever expanding use of libc, > we recently have interfaced to libSDL and cairo for some > very funky rpython graphics apps. And we use libpython > so that we can extract stuff from python modules. > Also, we do byte shuffling with rctypes: reading binary data > to/from buffers, etc. > > I hope we can do all this with rffi, but there are two really > cool things about rctypes: > 1) it runs on cpython (our main app takes 30minutes+ to compile) > 2) it has an automatic code generator (ctypes can autogenerate > python wrappers from C header files) > Very valid points indeed. The 1st point sounds unlikely, also because you might use C macros with rffi, which somehow invalidates possible uses on top of ctypes. What I do is I test (by compiling) several parts (some small code snippets involving ext functions). That make debugging a bit harder (reading C code), but pretty straightforward - for example memory management is done by hand in this part, which makes you responsible of tracking C structures and how long they need to live. The second point is much more likely to happen and to be better than ctypes code generation. (Because rffi is way more Cish than ctypes are). Also it makes stuff a bit harder when you don't submit your bindings back. If you plan to come to the post-EP pypy sprint, we can sit a bit on the automatic code generator for rffi. Cheers, fijal From tismer at stackless.com Sat Jun 16 15:26:39 2007 From: tismer at stackless.com (Christian Tismer) Date: Sat, 16 Jun 2007 15:26:39 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <20070611104726.GA28732@code0.codespeak.net> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> <20070611104726.GA28732@code0.codespeak.net> Message-ID: <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com> On 11.06.2007, at 12:47, Armin Rigo wrote: > Hi Maciek, > > On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote: >> Ok, so what about usual Thursday 17:00? Ok here, too. From arigo at tunes.org Sat Jun 16 18:46:48 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 16 Jun 2007 18:46:48 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> <20070611104726.GA28732@code0.codespeak.net> <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com> Message-ID: <20070616164648.GA31262@code0.codespeak.net> Hi Christian, On Sat, Jun 16, 2007 at 03:26:39PM +0200, Christian Tismer wrote: > > On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote: > >> Ok, so what about usual Thursday 17:00? > > Ok here, too. Just to make sure: that was about Thursday the 14th. Answering "Ok" on Saturday the 16th seems strange :-) A bientot, Armin. From arigo at tunes.org Sat Jun 16 19:10:54 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 16 Jun 2007 19:10:54 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070615174454.6e87fb1a.simon@arrowtheory.com> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> Message-ID: <20070616171054.GA695@code0.codespeak.net> Hi Simon, (This is also a follow-up on concerns raised by Christian on #pypy today) On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote: > I hope we can do all this with rffi, but there are two really > cool things about rctypes: > 1) it runs on cpython (our main app takes 30minutes+ to compile) > 2) it has an automatic code generator (ctypes can autogenerate > python wrappers from C header files) For 1), this would actually be possible with rffi too. The trick would be to have two implementations of the lltypesystem objects (array, structures, etc.). The existing implementation would only be used when ctypes is not available; if ctypes can be imported, then the other implementation could systematically be used. With a bit of effort it could have a fully compatible interface, but be internally implemented as ctypes objects (e.g. lltype.malloc(S) would build a ctypes structure type corresponding to S, and internally instantiate that). Then we could really call rffi functions, by turning it internally into a ctypes function and calling it with the ctypes objects underlying the passed-in arguments. This might even not be completely insane - in my opinion, less than implementing and maintaining rctypes itself. It's also something that can be done by someone not deeply into the code base, as long as he knows the lltype interface and how to use ctypes. A bientot, Armin. From tismer at stackless.com Sat Jun 16 22:25:38 2007 From: tismer at stackless.com (Christian Tismer) Date: Sat, 16 Jun 2007 22:25:38 +0200 Subject: [pypy-dev] PyPy sync next week? In-Reply-To: <20070616164648.GA31262@code0.codespeak.net> References: <4666E7A4.9030407@genesilico.pl> <466C46A2.9040108@genesilico.pl> <20070611104726.GA28732@code0.codespeak.net> <6686E8E7-CDCB-470C-8365-C17B3A33CB48@stackless.com> <20070616164648.GA31262@code0.codespeak.net> Message-ID: <4F3FD51F-F246-49E7-8B98-34ACE9FE2FBC@stackless.com> > Hi Armin, > On Sat, Jun 16, 2007 at 03:26:39PM +0200, Christian Tismer wrote: >>> On Sun, Jun 10, 2007 at 08:44:50PM +0200, Maciek Fijalkowski wrote: >>>> Ok, so what about usual Thursday 17:00? >> >> Ok here, too. > > Just to make sure: that was about Thursday the 14th. Answering > "Ok" on > Saturday the 16th seems strange :-) Oupps, yes. I saw this today, and thought it was about next Thursday as well. Sorry about the confusion. ciao - chris From fijal at genesilico.pl Sat Jun 16 22:37:16 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 16 Jun 2007 22:37:16 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070616171054.GA695@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> Message-ID: <467449FC.2040909@genesilico.pl> Armin Rigo wrote: > Hi Simon, > > (This is also a follow-up on concerns raised by Christian on #pypy > today) > > On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote: > >> I hope we can do all this with rffi, but there are two really >> cool things about rctypes: >> 1) it runs on cpython (our main app takes 30minutes+ to compile) >> 2) it has an automatic code generator (ctypes can autogenerate >> python wrappers from C header files) >> > > For 1), this would actually be possible with rffi too. The trick would > be to have two implementations of the lltypesystem objects (array, > structures, etc.). The existing implementation would only be used when > ctypes is not available; if ctypes can be imported, then the other > implementation could systematically be used. With a bit of effort it > could have a fully compatible interface, but be internally implemented > as ctypes objects (e.g. lltype.malloc(S) would build a ctypes structure > type corresponding to S, and internally instantiate that). Then we > could really call rffi functions, by turning it internally into a ctypes > function and calling it with the ctypes objects underlying the passed-in > arguments. > > This might even not be completely insane - in my opinion, less than > implementing and maintaining rctypes itself. It's also something that > can be done by someone not deeply into the code base, as long as he > knows the lltype interface and how to use ctypes. > > > A bientot, > > Armin. > > Just a sidenote - rffi supports (and uses) macros. Not sure how this will look like in a ctypes-based solution. Cheers, fijal From tismer at stackless.com Sat Jun 16 23:04:14 2007 From: tismer at stackless.com (Christian Tismer) Date: Sat, 16 Jun 2007 23:04:14 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070616171054.GA695@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> Message-ID: Hi Armin, On 16.06.2007, at 19:10, Armin Rigo wrote: > Hi Simon, > > (This is also a follow-up on concerns raised by Christian on #pypy > today) Well, reacting a bit to it, here... The reason why this stuff kept me busy thinking since the meeting was the fact that we are about to trash exactly the little stuff people are actually using. We do have very little users, and most of them are involved right here. So if we are going to trasg stuff, then we should go and offer help to those few users, to not getting distracted from PyPy. So my thoughts were around these ideas: - Try to extend CTypes to support primitive-enough concepts that do translate to RPython or - offer support to our users, to make transition to rffi smooth and easy. I think both ways are possible, I also guess the latter woulod be supported even if we asked for help or some funding for this necessary work. Whatever way we choose, I think it would be best for PyPy if we try to be as user-friendly as possible, saying like """we believe we must change this and that immediately, but we will not leave you in the rain, but actively help porting your stuff.""" This also relates a bit to my former thoughts about things which we have donated to the open source. We can, physically, just trash anything, of course. But from a point of view of users and our growing community, this is a matter of belief and trust that we should be careful about. I'm not saying that we don't care, these are just my thoughts and feelings about what I want to be not questionable. > On Fri, Jun 15, 2007 at 05:44:54PM -0700, Simon Burton wrote: >> I hope we can do all this with rffi, but there are two really >> cool things about rctypes: >> 1) it runs on cpython (our main app takes 30minutes+ to compile) >> 2) it has an automatic code generator (ctypes can autogenerate >> python wrappers from C header files) > > For 1), this would actually be possible with rffi too. The trick > would > be to have two implementations of the lltypesystem objects (array, > structures, etc.). I think the idea is not bad, but unless I'm missing something, don't we exactly loose the testing feature by this, by having to have two different implementations, the benefit of simple CPython testing for lltypesystem would be gone? > The existing implementation would only be used when > ctypes is not available; if ctypes can be imported, then the other > implementation could systematically be used. With a bit of effort it > could have a fully compatible interface, but be internally implemented > as ctypes objects (e.g. lltype.malloc(S) would build a ctypes > structure > type corresponding to S, and internally instantiate that). Then we > could really call rffi functions, by turning it internally into a > ctypes > function and calling it with the ctypes objects underlying the > passed-in > arguments. Not sure that I immediately get it. You would use CTypes to emulate rffi functions if we are in Python, and use them directly when you are in RPython? How would the memory ownership problem be handled? Ah, the problem is circumvented by doing all this stuff on app-level, dropping high efficiency constraints? No I am confused. > This might even not be completely insane - in my opinion, less than > implementing and maintaining rctypes itself. It's also something that > can be done by someone not deeply into the code base, as long as he > knows the lltype interface and how to use ctypes. By 'it', you mean the CTypes implementation, at app-level, based upon rffi? I hope I'm not completely lost, here :-) ciao - chris From faassen at startifact.com Mon Jun 18 16:36:10 2007 From: faassen at startifact.com (Martijn Faassen) Date: Mon, 18 Jun 2007 16:36:10 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> Message-ID: Hey, Christian Tismer wrote: > Whatever way we choose, I think it would be best for PyPy > if we try to be as user-friendly as possible, saying like > """we believe we must change this and that immediately, but > we will not leave you in the rain, but actively help porting > your stuff.""" > > This also relates a bit to my former thoughts about things > which we have donated to the open source. We can, physically, > just trash anything, of course. But from a point of view > of users and our growing community, this is a matter of belief > and trust that we should be careful about. > > I'm not saying that we don't care, these are just my thoughts > and feelings about what I want to be not questionable. Thanks Christian for making these points and having these concerns. I think that the project is wise to consider these issues once every while. I only understand a little of the technical discussion. I understand that RCTypes has problems, and that people anticipate having to break it and reimplement it in a different way in order to solve these problems. I understand that one feature that RCTypes offers, namely the ability to test your RPython code on CPython, might also go away. There are currently, as far as I can see it, two reasons to use RPython for CPython extensions: a) benefit from performance gains by rewriting your CPython code into RPython. This is what we want to do with the page template interpreter. In case of language implementations, this might lead to the ability to use other PyPy technology later (such as JIT generation and so on). Some others might purely be interested in performance though and aren't implementing a language. b) better ways to write extensions that use C libraries in CPython. The typical Pyrex use case, though it can do the former as well. This would use RCTypes. Use case a) probably has less interested people than use case b). Eventually JIT technologies will help here, which will make use case a) less important. I happen to be interested in use case a) though. It is quite exciting to me that we could have a faster template language interpreter implementation that we can use in CPython today, with the potential benefits of PyPy for interpreter translation in the future as well. We are basically language implementors, one of your core target audiences. We just feel more comfortable doing work if we know we have payoffs in the near future, i.e. use in CPython. If we were implementing this template language in isolation and had no hope we could deploy it in production any time soon, we would be less interested. I am actually quite interested in use case b) as well. While developing lxml, I've done quite a bit of work with Pyrex and I can see how RPython and RCTypes might potentially help projects like that. I've been taking a wait and see attitude here though, as the technology needed more maturing. As I understand it, the proposed changes wouldn't affect use case a). Or is the ability to generate CPython extensions also going to be broken due to this work? Please let us know, as if CPython support is going to be broken, we would likely stop our efforts for the time being. Use case b) would certainly be broken due to this work. What's more, I understand that testing your extension code in CPython first wouldn't work anymore (at least if it uses CTypes). This is a very nice feature of RPython that Pyrex doesn't have, by the way. You mentioned that there are potential ways to reimplement this feature again, but I'm not sure whether you're committing to such implementation. If I were writing C extension modules with PyPy I'd like to know things like: * what are the reasons this change is going to be made? * would this benefit just core PyPy or would it benefit extension module writers too? * when do you expect the new way to be finalized? * will you make a real commitment to make the "Test in CPython first" feature work again? Or is did you just sketched out possible ways to implement this in the new system and you hope some contributor is going to do the work? * will you make a commitment to make this new way useful and supported for extension module writers, or should they invest their time in some other technology? I.e. what are the changes a future shift will make me change things again? * How well documented will this be for CPython extension developers? Will you market this feature to developers? Things like this indicate your commitment in another way. :) Please note that I'm not actually *asking* for anything but information. I'm not telling the PyPy developers they should make commitments and do various kinds of work. That's up to you. I'm just trying to guess what other developers might want to know. Of course, the quality of your communication and the answers you actually give will make people decide whether to continue to use PyPy or move on. This will affect the way your open source community will grow and develop. I would urge the PyPy developers to consider investing in the open source community by supporting features like this and promoting them. This would mean serving admittedly self-interested developers with short term goals. The promise of immediate payoff is a great way to attract people to an open source project. If there is the hope of continuing payoff (and PyPy has tremendous potential here), some of those new people can be expected to become valuable contributors to the project as a whole. And yes, I'm self-interested in saying all of this - that's my point. Regards, Martijn From alexandre.fayolle at logilab.fr Tue Jun 19 09:54:52 2007 From: alexandre.fayolle at logilab.fr (Alexandre Fayolle) Date: Tue, 19 Jun 2007 09:54:52 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <1BB835BB-691C-480A-A941-86454586AB4A@mac.com> References: <20070614164554.GA654@code0.codespeak.net> <4671C8BD.4050707@gmail.com> <20070615063017.GK3311@solar.trillke> <467257BF.5060709@genesilico.pl> <1BB835BB-691C-480A-A941-86454586AB4A@mac.com> Message-ID: <20070619075452.GA8790@crater.logilab.fr> On Fri, Jun 15, 2007 at 01:16:06PM +0200, Alexander Kellett wrote: > On 15 Jun 2007, at 13:04, Martijn Faassen wrote: > > You could tell me that I'd be more productive if I contributed to the > > JIT generator, but then I'd go away again and you'd lose a potential > > contributor. If I can speed up my templating language using RPython I > > might stick around. I realize that I personally am of small potential > > value to the project, but who knows who else you might draw in this > > way? > > It seems like many are in this boat. > > Are there really such difficult stumbling blocks involved in making a > (r)python lint Some work has been done on this, using pylint which has an rpython checker. Contributed additional checks are welcome. http://www.logilab.org/projects/pylint -- Alexandre Fayolle LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Informatique scientifique: http://www.logilab.fr/science Reprise et maintenance de sites CPS: http://www.migration-cms.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 481 bytes Desc: Digital signature Url : http://codespeak.net/pipermail/pypy-dev/attachments/20070619/616c8fe7/attachment.pgp From elmo13 at jippii.fi Tue Jun 19 10:18:59 2007 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Tue, 19 Jun 2007 11:18:59 +0300 Subject: [pypy-dev] About adding a blist implementation... In-Reply-To: <466FE95A.6020805@jippii.fi> References: <466FE95A.6020805@jippii.fi> Message-ID: <46779173.3050902@jippii.fi> Since I'm adding a list implementation, I'd like to know what benchmarks have you used to test the other ones. Also, if you know apps with heavy usage of lists and some that use very long lists, I'd be interested very much. Elmo From faassen at startifact.com Wed Jun 20 12:23:06 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 20 Jun 2007 12:23:06 +0200 Subject: [pypy-dev] about breaking RPython, RCTypes Message-ID: Hi there, I posted this to a longer thread the other day but have received no answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I spent a bit of time writing this post, and I ask some questions in it that I think should be answered. Perhaps by telling me I don't understand what I'm talking about or that my analysis is entirely incorrect. Perhaps it was simply too long? Whatever it is, I do think the PyPy community needs to be aware of how technical decisions have an impact on attracting or repelling potential contributors. "answering by silence", which happened so far, is one way to make potential contributors worried. :) I only understand a little of the technical discussion. I understand that RCTypes has problems, and that people anticipate having to break it and reimplement it in a different way in order to solve these problems. I understand that one feature that RCTypes offers, namely the ability to test your RPython code on CPython, might also go away. There are currently, as far as I can see it, two reasons to use RPython for CPython extensions: a) benefit from performance gains by rewriting your CPython code into RPython. This is what we want to do with the page template interpreter. In case of language implementations, this might lead to the ability to use other PyPy technology later (such as JIT generation and so on). Some others might purely be interested in performance though and aren't implementing a language. b) better ways to write extensions that use C libraries in CPython. The typical Pyrex use case, though it can do the former as well. This would use RCTypes. Use case a) probably has less interested people than use case b). Eventually JIT technologies will help here, which will make use case a) less important. I happen to be interested in use case a) though. It is quite exciting to me that we could have a faster template language interpreter implementation that we can use in CPython today, with the potential benefits of PyPy for interpreter translation in the future as well. We are basically language implementors, one of your core target audiences. We just feel more comfortable doing work if we know we have payoffs in the near future, i.e. use in CPython. If we were implementing this template language in isolation and had no hope we could deploy it in production any time soon, we would be less interested. I am actually quite interested in use case b) as well. While developing lxml, I've done quite a bit of work with Pyrex and I can see how RPython and RCTypes might potentially help projects like that. I've been taking a wait and see attitude here though, as the technology needed more maturing. As I understand it, the proposed changes wouldn't affect use case a). Or is the ability to generate CPython extensions also going to be broken due to this work? Please let us know, as if CPython support is going to be broken, we would likely stop our efforts for the time being. Use case b) would certainly be broken due to this work. What's more, I understand that testing your extension code in CPython first wouldn't work anymore (at least if it uses CTypes). This is a very nice feature of RPython that Pyrex doesn't have. You mentioned that there are potential ways to reimplement this feature again, but I'm not sure whether you're committing to such implementation work. If I were writing C extension modules with PyPy I'd like to know things like: * what are the reasons this change is going to be made? * would this benefit just core PyPy or would it benefit extension module writers too? * when do you expect the new way to be finalized? * will you make a real commitment to make the "Test in CPython first" feature work again? Or is did you just sketched out possible ways to implement this in the new system and you hope some contributor is going to do the work? * will you make a commitment to make this new way useful and supported for extension module writers, or should they invest their time in some other technology? I.e. what are the changes a future shift will make me change things again? * How well documented will this be for CPython extension developers? Will you market this feature to developers? Things like this indicate your commitment in another way. :) Please note that I'm not actually *asking* for anything but information. I'm not telling the PyPy developers they should make commitments and do various kinds of work. That's up to you. I'm just trying to guess what other developers might want to know. Of course, the quality of your communication and the answers you actually give will make people decide whether to continue to use PyPy or move on. This will affect the way your open source community will grow and develop. I would urge the PyPy developers to consider investing in the open source community by supporting features like this and promoting them. This would mean serving admittedly self-interested developers with short term goals. The promise of immediate payoff is a great way to attract people to an open source project. If there is the hope of continuing payoff (and PyPy has tremendous potential here), some of those new people can be expected to become valuable contributors to the project as a whole. And yes, I'm self-interested in saying all of this - that's my point. Regards, Martijn From bea at changemaker.nu Wed Jun 20 12:57:06 2007 From: bea at changemaker.nu (=?ISO-8859-1?Q?Beatrice_D=FCring?=) Date: Wed, 20 Jun 2007 12:57:06 +0200 Subject: [pypy-dev] about breaking RPython, RCTypes In-Reply-To: References: Message-ID: <46790801.80407@changemaker.nu> Hi there Martijn Faassen skrev: > Hi there, > > > I posted this to a longer thread the other day but have received no > answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I > spent a bit of time writing this post, and I ask some questions in it > that I think should be answered. Perhaps by telling me I don't > understand what I'm talking about or that my analysis is entirely > incorrect. Perhaps it was simply too long? > > Whatever it is, I do think the PyPy community needs to be aware of how > technical decisions have an impact on attracting or repelling potential > contributors. "answering by silence", which happened so far, is one way > to make potential contributors worried. :) > > Although I can?t answer your questions I do appreciate the time and effort you and others have spent on this issue during the last couple of days, and I am sure the PyPy core gang appreciates your input. The current plan is to meet at EuroPython in Vilnius to have strategic technical discussions regarding direction and among them decisions regarding the issues you raise in this email. I interpret the silence as people feeling that they can?t answer this until such a discussion (???) but that does not excuse the total silence - hence this email (that does not give you the answers either). As you might have noticed - this topic is also on the agenda as a sprint topic for the PyPy Vilnius sprint at which time we hope to have made some decisions. Since you are coming to EuroPython I look forward to meet you again and discuss this and make sure that a PyPy strategic session takes place which gives you answers - answers you might like - or not like ;-) I hope this response does not frustrate you too much.... Cheers Bea > I only understand a little of the technical discussion. I understand > that RCTypes has problems, and that people anticipate having to break it > and reimplement it in a different way in order to solve these problems. > I understand that one feature that RCTypes offers, namely the ability to > test your RPython code on CPython, might also go away. > > There are currently, as far as I can see it, two reasons to use RPython > for CPython extensions: > > a) benefit from performance gains by rewriting your CPython code into > RPython. This is what we want to do with the page template interpreter. > In case of language implementations, this might lead to the ability to > use other PyPy technology later (such as JIT generation and so on). Some > others might purely be interested in performance though and aren't > implementing a language. > > b) better ways to write extensions that use C libraries in CPython. The > typical Pyrex use case, though it can do the former as well. This would > use RCTypes. > > Use case a) probably has less interested people than use case b). > Eventually JIT technologies will help here, which will make use case a) > less important. > > I happen to be interested in use case a) though. It is quite exciting to > me that we could have a faster template language interpreter > implementation that we can use in CPython today, with the potential > benefits of PyPy for interpreter translation in the future as well. We > are basically language implementors, one of your core target audiences. > We just feel more comfortable doing work if we know we have payoffs in > the near future, i.e. use in CPython. If we were implementing this > template language in isolation and had no hope we could deploy it in > production any time soon, we would be less interested. > > I am actually quite interested in use case b) as well. While developing > lxml, I've done quite a bit of work with Pyrex and I can see how RPython > and RCTypes might potentially help projects like that. I've been taking > a wait and see attitude here though, as the technology needed more maturing. > > As I understand it, the proposed changes wouldn't affect use case a). Or > is the ability to generate CPython extensions also going to be broken > due to this work? Please let us know, as if CPython support is going to > be broken, we would likely stop our efforts for the time being. > > Use case b) would certainly be broken due to this work. What's more, I > understand that testing your extension code in CPython first wouldn't > work anymore (at least if it uses CTypes). This is a very nice feature > of RPython that Pyrex doesn't have. You mentioned that there > are potential ways to reimplement this feature again, but I'm not sure > whether you're committing to such implementation work. > > If I were writing C extension modules with PyPy I'd like to know things > like: > > * what are the reasons this change is going to be made? > > * would this benefit just core PyPy or would it benefit extension module > writers too? > > * when do you expect the new way to be finalized? > > * will you make a real commitment to make the "Test in CPython first" > feature work again? Or is did you just sketched out possible ways to > implement this in the new system and you hope some contributor is going > to do the work? > > * will you make a commitment to make this new way useful and supported > for extension module writers, or should they invest their time in some > other technology? I.e. what are the changes a future shift will make me > change things again? > > * How well documented will this be for CPython extension developers? > Will you market this feature to developers? Things like this indicate > your commitment in another way. :) > > Please note that I'm not actually *asking* for anything but information. > I'm not telling the PyPy developers they should make commitments and do > various kinds of work. That's up to you. I'm just trying to guess what > other developers might want to know. > > Of course, the quality of your communication and the answers you > actually give will make people decide whether to continue to use PyPy or > move on. This will affect the way your open source community will grow > and develop. > > I would urge the PyPy developers to consider investing in the open > source community by supporting features like this and promoting them. > This would mean serving admittedly self-interested developers with short > term goals. The promise of immediate payoff is a great way to attract > people to an open source project. If there is the hope of continuing > payoff (and PyPy has tremendous potential here), some of those new > people can be expected to become valuable contributors to the project as > a whole. And yes, I'm self-interested in saying all of this - that's my > point. > > Regards, > > Martijn > > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev > From faassen at startifact.com Wed Jun 20 13:14:30 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 20 Jun 2007 13:14:30 +0200 Subject: [pypy-dev] about breaking RPython, RCTypes In-Reply-To: <46790801.80407@changemaker.nu> References: <46790801.80407@changemaker.nu> Message-ID: Hey Beatrice, An answer of "we're thinking about it, we can't answer it yet" is totally acceptable. Thanks! I realize fully that the answers you come up with may be answers I won't like. That's fine (well, it's not, but it's not my project so I can't complain :) I am just trying to help in articulating hopefully relevant questions. Regards, Martijn From radix at twistedmatrix.com Wed Jun 20 15:48:59 2007 From: radix at twistedmatrix.com (Christopher Armstrong) Date: Wed, 20 Jun 2007 09:48:59 -0400 Subject: [pypy-dev] about breaking RPython, RCTypes In-Reply-To: References: Message-ID: <60ed19d40706200648l56d57a79o16add35338d85036@mail.gmail.com> On 6/20/07, Martijn Faassen wrote: > There are currently, as far as I can see it, two reasons to use RPython > for CPython extensions: > > a) benefit from performance gains by rewriting your CPython code into > RPython. This is what we want to do with the page template interpreter. > In case of language implementations, this might lead to the ability to > use other PyPy technology later (such as JIT generation and so on). Some > others might purely be interested in performance though and aren't > implementing a language. > > b) better ways to write extensions that use C libraries in CPython. The > typical Pyrex use case, though it can do the former as well. This would > use RCTypes. While this is perhaps only an instance of your first use case (although not really performance-oriented), using RPython code as a CPython extension module is how I was planning on Safelisp being used (a small safe language written in RPython, meant for embedding). I can imagine a similar situation for PyPy's Python interpreter itself, so one could "sneak" PyPy-using Python code into their production system by importing the PyPy Python interpreter as a CPython module. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/ From arigo at tunes.org Wed Jun 20 18:16:51 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 20 Jun 2007 18:16:51 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> Message-ID: <20070620161651.GA23345@code0.codespeak.net> Hi Martijn, On Mon, Jun 18, 2007 at 04:36:10PM +0200, Martijn Faassen wrote: > If I were writing C extension modules with PyPy I'd like to know things > like: > > * what are the reasons this change is going to be made? > > * would this benefit just core PyPy or would it benefit extension module > writers too? > > (...) That's a very good list of questions. Indeed, we don't have clear answers for them yet, but just vague thoughts. I think it's fair to say that we'll discuss this at EuroPython and make some commitments then; for now we're only tossing ideas around. A bientot, Armin. From faassen at startifact.com Wed Jun 20 20:32:26 2007 From: faassen at startifact.com (Martijn Faassen) Date: Wed, 20 Jun 2007 20:32:26 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070620161651.GA23345@code0.codespeak.net> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> <20070620161651.GA23345@code0.codespeak.net> Message-ID: <8928d4e90706201132h46c83df0n5fc1f885d1fb1d91@mail.gmail.com> Hey, On 6/20/07, Armin Rigo wrote: [my questions] > That's a very good list of questions. Indeed, we don't have clear > answers for them yet, but just vague thoughts. I think it's fair to say > that we'll discuss this at EuroPython and make some commitments then; > for now we're only tossing ideas around. Thanks, that's totally understandable. :) Regards, Martijn From scott+pypy-dev at scottdial.com Thu Jun 21 09:11:15 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Thu, 21 Jun 2007 03:11:15 -0400 Subject: [pypy-dev] lltypesystem typecache Message-ID: <467A2493.8060603@scottdial.com> Greetings, The typecache in subversion repository is missing the 32bit windows platform that I am running the windows tests against, and on occasion (like today) I noticed it caused a update conflict due to Armin's changes in r44398. If someone could add this one, thanks. -Scott Index: pypy/rpython/lltypesystem/typecache.py =================================================================== --- pypy/rpython/lltypesystem/typecache.py (revision 44403) +++ pypy/rpython/lltypesystem/typecache.py (working copy) @@ -1,6 +1,7 @@ # this is automatically generated cache files for c types platforms = { ('', ('32bit', 'ELF'), 'Linux'):{'short': 16, 'int': 32, 'unsigned char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 32, 'long long': 64, 'mode_t': 32, 'unsigned long long': 64, 'size_t': 32, 'unsigned int': 32}, +('', ('32bit', 'WindowsPE'), 'Windows'):{'short': 16, 'int': 32, 'unsigned char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 32, 'long long': 64, 'unsigned long long': 64, 'size_t': 32, 'unsigned int': 32}, ('i386', ('32bit', ''), 'Darwin'):{'short': 16, 'int': 32, 'unsigned char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 32, 'long long': 64, 'mode_t': 16, 'unsigned long long': 64, 'size_t': 32, 'unsigned int': 32}, ('Intel(R) Pentium(R) M processor 1.73GHz', ('32bit', ''), 'Linux'):{'short': 16, 'int': 32, 'unsigned char': 8, 'long': 32, 'char': 8, 'unsigned short': 16, 'unsigned long': 32, 'long long': 64, 'mode_t': 32, 'unsigned long long': 64, 'size_t': 32, 'unsigned int': 32} } -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From amauryfa at gmail.com Thu Jun 21 10:05:31 2007 From: amauryfa at gmail.com (Amaury Forgeot d'Arc) Date: Thu, 21 Jun 2007 10:05:31 +0200 Subject: [pypy-dev] lltypesystem typecache In-Reply-To: References: <467A2493.8060603@scottdial.com> Message-ID: Scott Dial wrote: > Greetings, > > The typecache in subversion repository is missing the 32bit windows > platform that I am running the windows tests against, and on occasion > (like today) I noticed it caused a update conflict due to Armin's > changes in r44398. If someone could add this one, thanks. > > -Scott I have seen the same behaviour, and my diffs are the same. I just commited them. -- Amaury Forgeot d'Arc From jgustak at gmail.com Thu Jun 21 21:53:42 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Thu, 21 Jun 2007 21:53:42 +0200 Subject: [pypy-dev] Scheme interpreter - evaluation context Message-ID: I was looking into some code, JS interpreter to be exact. I wonder what are the pros and cons of implementing evaluation context as stack/list (as in JS interpreter) in opposite to implementing it as dictionary. If there is one name space for all objects in Scheme (I mean, there is no distinction between variable or function when naming them), dictionary would be the most straight forward. There is no need for a special way to resolve naming conflicts and so on. Or am I missing something? Cheers, Jakub Gustak From santagada at gmail.com Fri Jun 22 00:10:11 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 21 Jun 2007 19:10:11 -0300 Subject: [pypy-dev] Scheme interpreter - evaluation context In-Reply-To: References: Message-ID: <79CA6D75-19D8-4CA9-8EC5-6B5CB79D5C28@gmail.com> Em 21/06/2007, ?s 16:53, Jakub Gustak escreveu: > I was looking into some code, JS interpreter to be exact. > > I wonder what are the pros and cons of implementing evaluation context > as stack/list (as in JS interpreter) in opposite to implementing it as > dictionary. > > If there is one name space for all objects in Scheme (I mean, there is > no distinction between variable or function when naming them), > dictionary would be the most straight forward. There is no need for a > special way to resolve naming conflicts and so on. > > Or am I missing something? On the js interpreter I only used a stack of objects because the specs said to do so... I never felt that that was a good idea, so you are probably better following what you think is best/easier. Good luck, -- Leonardo Santagada Sent from my iPhone From simon at arrowtheory.com Fri Jun 22 03:10:01 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 21 Jun 2007 18:10:01 -0700 Subject: [pypy-dev] About adding a blist implementation... In-Reply-To: <46779173.3050902@jippii.fi> References: <466FE95A.6020805@jippii.fi> <46779173.3050902@jippii.fi> Message-ID: <20070621181001.d9322aef.simon@arrowtheory.com> On Tue, 19 Jun 2007 11:18:59 +0300 Elmo M?ntynen wrote: > > Since I'm adding a list implementation, I'd like to know what benchmarks > have you used to test the other ones. Also, if you know apps with heavy > usage of lists and some that use very long lists, I'd be interested very > much. Can you explain what this blist is ? Is it a btree ? I think I need an rpython btree. Simon. From santagada at gmail.com Fri Jun 22 04:13:59 2007 From: santagada at gmail.com (Leonardo Santagada) Date: Thu, 21 Jun 2007 23:13:59 -0300 Subject: [pypy-dev] About adding a blist implementation... In-Reply-To: <20070621181001.d9322aef.simon@arrowtheory.com> References: <466FE95A.6020805@jippii.fi> <46779173.3050902@jippii.fi> <20070621181001.d9322aef.simon@arrowtheory.com> Message-ID: Em 21/06/2007, ?s 22:10, Simon Burton escreveu: > On Tue, 19 Jun 2007 11:18:59 +0300 > Elmo M?ntynen wrote: > >> >> Since I'm adding a list implementation, I'd like to know what >> benchmarks >> have you used to test the other ones. Also, if you know apps with >> heavy >> usage of lists and some that use very long lists, I'd be >> interested very >> much. > > Can you explain what this blist is ? Is it a btree ? I think I need > an rpython btree. > > Simon. I dunno but this guy is probably talking about something like this http://mail.python.org/pipermail/python-3000/2007-May/thread.html#7127 If it is not related I would also like to understand the diferences. -- Leonardo Santagada Sent from my iPhone From arigo at tunes.org Fri Jun 22 10:51:31 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 22 Jun 2007 10:51:31 +0200 Subject: [pypy-dev] lltypesystem typecache In-Reply-To: <467A2493.8060603@scottdial.com> References: <467A2493.8060603@scottdial.com> Message-ID: <20070622085129.GA16685@code0.codespeak.net> Hi Scott, On Thu, Jun 21, 2007 at 03:11:15AM -0400, Scott Dial wrote: > (like today) I noticed it caused a update conflict due to Armin's > changes in r44398. My checkin was mostly accidental. I think that the way the typecache is both a regular svn-controlled file deep within the hierarchy, and modified automatically, will keep causing conflicts in the future. Also, the entry that was generated on my laptop and that I checked in while doing other stuff is: > ('Intel(R) Pentium(R) M processor 1.73GHz', ('32bit', ''), 'Linux'):... That looks like a far too precise key. I don't have a nice alternative, I'm just saying that the current way it works is a bit troublesome :-) A bientot, Armin. From scott+pypy-dev at scottdial.com Fri Jun 22 11:01:37 2007 From: scott+pypy-dev at scottdial.com (Scott Dial) Date: Fri, 22 Jun 2007 05:01:37 -0400 Subject: [pypy-dev] lltypesystem typecache In-Reply-To: <20070622085129.GA16685@code0.codespeak.net> References: <467A2493.8060603@scottdial.com> <20070622085129.GA16685@code0.codespeak.net> Message-ID: <467B8FF1.50301@scottdial.com> Armin Rigo wrote: > My checkin was mostly accidental. I think that the way the typecache is > both a regular svn-controlled file deep within the hierarchy, and > modified automatically, will keep causing conflicts in the future. > I thought this as well, but I am not familiar with its purpose. If it is merely a cache, then I don't understand why it is under SVN control at all. I suppose it allows developers without those other platforms available to know about types, but that seems like a marginal reason. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From arigo at tunes.org Fri Jun 22 11:05:36 2007 From: arigo at tunes.org (Armin Rigo) Date: Fri, 22 Jun 2007 11:05:36 +0200 Subject: [pypy-dev] Vilnius/Post EuroPython PyPy Sprint 12-14th of July Message-ID: <20070622090536.GB16685@code0.codespeak.net> ======================================================== Vilnius/Post EuroPython PyPy Sprint 12-14th of July ======================================================== The PyPy team is sprinting at EuroPython again and we invite you to participate in our 3 day long sprint at the conference hotel - Reval Hotel Lietuva. If you plan to attend the sprint we recommend you to listen to the PyPy technical talks (`EuroPython schedule`_) during the conference since it will give you a good overview of the status of development. On the morning of the first sprint day (12th) we will also have a tutorial session for those new to PyPy development. As 3 days is relatively short for a PyPy sprint we suggest to travel back home on the 15th if possible (but it is ok to attend less than 3 days too). ------------------------------ Goals and topics of the sprint ------------------------------ There are many possible and interesting sprint topics to work on - here we list some possible task areas: * completing the missing python 2.5 features and support * write or port more extension modules (e.g. zlib is missing) * identify slow areas of PyPy through benchmarking and work on improvements, possibly moving app-level parts of the Python interpreter to interp-level if useful. * there are some parts of PyPy in need of refactoring, we may spend some time on those, for example: - rctypes and the extension compiler need some rethinking - support for LLVM 2.0 for the llvm backend - ... * some JIT improvement work * port the stackless transform to ootypesystem * other interesting stuff that you would like to work on ...;-) ------------ Registration ------------ If you'd like to come, please subscribe to the `pypy-sprint mailing list`_ and drop a note about your interests and post any questions. More organisational information will be sent to that list. Please register by adding yourself on the following list (via svn): http://codespeak.net/svn/pypy/extradoc/sprintinfo/post-ep2007/people.txt or on the pypy-sprint mailing list if you do not yet have check-in rights: http://codespeak.net/mailman/listinfo/pypy-sprint --------------------------------------- Preparation (if you feel it is needed): --------------------------------------- * read the `getting-started`_ pages on http://codespeak.net/pypy * for inspiration, overview and technical status you are welcome to read `the technical reports available and other relevant documentation`_ * please direct any technical and/or development oriented questions to pypy-dev at codespeak.net and any sprint organizing/logistical questions to pypy-sprint at codespeak.net * if you need information about the conference, potential hotels, directions etc we recommend to look at http://www.europython.org. We are looking forward to meet you at the Vilnius Post EuroPython PyPy sprint! The PyPy team .. See also .. .. _getting-started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html .. _`pypy-sprint mailing list`: http://codespeak.net/mailman/listinfo/pypy-sprint .. _`the technical reports available and other relevant documentation`: http://codespeak.net/pypy/dist/pypy/doc/index.html .. _`EuroPython schedule`: http://indico.cern.ch/conferenceTimeTable.py?confId=13919&showDate=all&showSession=all&detailLevel=contribution&viewMode=room From micahel at gmail.com Fri Jun 22 11:11:46 2007 From: micahel at gmail.com (Michael Hudson) Date: Fri, 22 Jun 2007 10:11:46 +0100 Subject: [pypy-dev] lltypesystem typecache In-Reply-To: <20070622085129.GA16685@code0.codespeak.net> References: <467A2493.8060603@scottdial.com> <20070622085129.GA16685@code0.codespeak.net> Message-ID: On 22/06/07, Armin Rigo wrote: > Hi Scott, > > On Thu, Jun 21, 2007 at 03:11:15AM -0400, Scott Dial wrote: > > (like today) I noticed it caused a update conflict due to Armin's > > changes in r44398. > > My checkin was mostly accidental. I think that the way the typecache is > both a regular svn-controlled file deep within the hierarchy, and > modified automatically, will keep causing conflicts in the future. Indeed, since the rffi merge, we haven't had two builds in a row on tuatara :( Cheers, mwh From elmo13 at jippii.fi Fri Jun 22 16:10:44 2007 From: elmo13 at jippii.fi (=?ISO-8859-1?Q?Elmo_M=E4ntynen?=) Date: Fri, 22 Jun 2007 17:10:44 +0300 Subject: [pypy-dev] About adding a blist implementation... In-Reply-To: References: <466FE95A.6020805@jippii.fi> <46779173.3050902@jippii.fi> <20070621181001.d9322aef.simon@arrowtheory.com> Message-ID: <467BD864.8040505@jippii.fi> Leonardo Santagada wrote: > Em 21/06/2007, ?s 22:10, Simon Burton escreveu: > > >> On Tue, 19 Jun 2007 11:18:59 +0300 >> Elmo M?ntynen wrote: >> >> >>> Since I'm adding a list implementation, I'd like to know what >>> benchmarks >>> have you used to test the other ones. Also, if you know apps with >>> heavy >>> usage of lists and some that use very long lists, I'd be >>> interested very >>> much. >>> >> Can you explain what this blist is ? Is it a btree ? I think I need >> an rpython btree. >> >> Simon. >> > > I dunno but this guy is probably talking about something like this > http://mail.python.org/pipermail/python-3000/2007-May/thread.html#7127 > > If it is not related I would also like to understand the diferences. > > -- > Leonardo Santagada > > Sent from my iPhone > > Hi. It is an list implementation that has much better performance with very big lists, available as an extension to CPython, and soon included in pypy by me. This cheeseshop entry is a good place to start: http://www.python.org/pypi/blist/ The thread might be about integrading it into CPython or something, haven't read. It is based on btree, but the implementation I'm working on is based on the python prototype included in the tar distrib, which won't probably be useful if you specifically need a btree. In the future, maybe some refactoring would be useful, but for now I'm happy with the way I started. I'd still be interested in ways to test the different list implementations =) Elmo From arigo at tunes.org Sat Jun 23 10:25:38 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 23 Jun 2007 10:25:38 +0200 Subject: [pypy-dev] [LLVMdev] Vilnius/Post EuroPython PyPy Sprint 12-14th of July In-Reply-To: <1182535152.6173.25.camel@asl.dorms.spbu.ru> References: <20070622090903.GA19700.SS3552SS@code0.codespeak.net> <1182535152.6173.25.camel@asl.dorms.spbu.ru> Message-ID: <20070623082538.GB358@code0.codespeak.net> Hi Anton, (CC'ing to pypy-dev) On Fri, Jun 22, 2007 at 09:59:12PM +0400, Anton Korobeynikov wrote: > > - support for LLVM 2.0 for the llvm backend > Is there any detailed information, how the sprint will touch this topic? I don't have many details, but there are three subtasks regarding LLVM: on the one hand, we did a small addition to the translation toolchain that is not yet supported by our backend generating LLVM code, so the first subtask is this "maintenance" task, which should be not too much efforts if done while pairing with someone that knows already a bit about the backend. Then, there is the idea of changing the LLVM assembler produced - at the moment, it's for LLVM 1.9, but we want to move to 2.0. I can't tell how much effort this is, as I'm not enough into LLVM or the LLVM backend. The third, longer-term task that we may not start during the sprint would be to refactor and clean up the LLVM backend and the C backend, so that they can share code a bit more naturally (at the moment the LLVM backend fishes for internal routines of the C backend). A bientot, Armin. From rxe at ukshells.co.uk Sat Jun 23 10:35:34 2007 From: rxe at ukshells.co.uk (Richard Emslie) Date: Sat, 23 Jun 2007 01:35:34 -0700 Subject: [pypy-dev] [LLVMdev] Vilnius/Post EuroPython PyPy Sprint 12-14th of July In-Reply-To: <20070623082538.GB358@code0.codespeak.net> References: <20070622090903.GA19700.SS3552SS@code0.codespeak.net> <1182535152.6173.25.camel@asl.dorms.spbu.ru> <20070623082538.GB358@code0.codespeak.net> Message-ID: Hi, Just a word of warning - i am (was) very much planning to do some of these tasks pre-sprint. Not sure I can make the sprint anyhow. A 4th larger task also is to make gc and stackless work. A 5th task is to take pbcs work again! ;-) I'll be on irc over the weekend - right now i need some sleep. Cheers, Richard On Jun 23, 2007, at 1:25 AM, Armin Rigo wrote: > Hi Anton, > > (CC'ing to pypy-dev) > > On Fri, Jun 22, 2007 at 09:59:12PM +0400, Anton Korobeynikov wrote: >>> - support for LLVM 2.0 for the llvm backend >> Is there any detailed information, how the sprint will touch this >> topic? > > I don't have many details, but there are three subtasks regarding > LLVM: > on the one hand, we did a small addition to the translation toolchain > that is not yet supported by our backend generating LLVM code, so the > first subtask is this "maintenance" task, which should be not too much > efforts if done while pairing with someone that knows already a bit > about the backend. Then, there is the idea of changing the LLVM > assembler produced - at the moment, it's for LLVM 1.9, but we want to > move to 2.0. I can't tell how much effort this is, as I'm not enough > into LLVM or the LLVM backend. The third, longer-term task that we > may > not start during the sprint would be to refactor and clean up the LLVM > backend and the C backend, so that they can share code a bit more > naturally (at the moment the LLVM backend fishes for internal routines > of the C backend). > > > A bientot, > > Armin. > _______________________________________________ > pypy-dev at codespeak.net > http://codespeak.net/mailman/listinfo/pypy-dev From fijal at genesilico.pl Mon Jun 25 13:37:18 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Mon, 25 Jun 2007 13:37:18 +0200 Subject: [pypy-dev] lltypesystem typecache In-Reply-To: References: <467A2493.8060603@scottdial.com> <20070622085129.GA16685@code0.codespeak.net> Message-ID: <467FA8EE.4010206@genesilico.pl> Hum. The idea was to keep this stuff in svn in order to keep type information to-be-copied around. If this is troublesome, I propose to kill whole idea of having cache-per-platform, since we cannot specify good platform key and have it in svn:ignore (or move to _cache) and just keep it on a platform level. Than one would need to have gcc installed at least once to import rffi. I don't know if it's good or bad. From elmo13 at jippii.fi Mon Jun 25 15:45:53 2007 From: elmo13 at jippii.fi (=?UTF-8?B?RWxtbyBNw6RudHluZW4=?=) Date: Mon, 25 Jun 2007 16:45:53 +0300 Subject: [pypy-dev] Some bugfixes to the multilistimplementation Message-ID: <467FC711.100@jippii.fi> Hi. I've made the different [slow] list implementations even slower ! While integrating the blist implementation, I found out that the chosen implementation isn't even used when instantiating with "list()", and neither in some other usecases. By correcting that, a bug in the chunked implementation got uncovered, which I also fixed. I also added "make_implementation_with_one_item()" to have the instantiation of the implementations in one place so that it's easier to add new ones. If they seem good, could someone commit these changes (is that diff format good?). Would you trust me with commit rights, or should I still send patches? The test demonstrates the bugs, but I thougth of later adding a BaseAppTest_ListMultiObject class with magic to test if the implementations that the subclasses test are really used. Does that sound good? Index: objspace/std/listmultiobject.py =================================================================== --- objspace/std/listmultiobject.py (revision 44498) +++ objspace/std/listmultiobject.py (working copy) @@ -12,7 +12,7 @@ # An empty list always is an EmptyListImplementation. # -# RDictImplementation -- standard implementation +# RListImplementation -- standard implementation # StrListImplementation -- lists consisting only of strings # ChunkedListImplementation -- when having set the withchunklist option # SmartResizableListImplementation -- when having set the @@ -87,7 +87,7 @@ for i in range(slicelength): res_w[i] = self.getitem(start) start += step - return RListImplementation(self.space, res_w) + return make_implementation(self.space, res_w) def delitem_slice_step(self, start, stop, step, slicelength): n = self.length() @@ -368,8 +368,8 @@ length = self._length self._grow() - for j in range(length - 1, 0, -1): - self.setitem(j + 1, self.getitem(j)) + for j in range(length, i, -1): + self.setitem(j, self.getitem(j-1)) self.setitem(i, w_item) return self @@ -399,13 +399,7 @@ class EmptyListImplementation(ListImplementation): def make_list_with_one_item(self, w_item): space = self.space - if space.config.objspace.std.withfastslice: - return SliceTrackingListImplementation(space, [w_item]) - w_type = space.type(w_item) - if space.is_w(w_type, space.w_str): - strlist = [space.str_w(w_item)] - return StrListImplementation(space, strlist) - return RListImplementation(space, [w_item]) + return make_implementation_with_one_item(space, w_item) def length(self): return 0 @@ -834,6 +828,27 @@ else: return RListImplementation(space, list_w) +def make_implementation_with_one_item(space, w_item): + if space.config.objspace.std.withfastslice: + return SliceTrackingListImplementation(space, [w_item]) + if space.config.objspace.std.withsmartresizablelist: + from pypy.objspace.std.smartresizablelist import \ + SmartResizableListImplementation + impl = SmartResizableListImplementation(space) + impl.append(w_item) + return impl + if space.config.objspace.std.withchunklist: + return ChunkedListImplementation(space, [w_item]) + if space.config.objspace.std.withblist: + from pypy.objspace.std.blistimplementation import \ + BListImplementation + return BListImplementation(space, [w_item]) + w_type = space.type(w_item) + if space.is_w(w_type, space.w_str): + strlist = [space.str_w(w_item)] + return StrListImplementation(space, strlist) + return RListImplementation(space, [w_item]) + def convert_list_w(space, list_w): if not list_w: impl = space.fromcache(State).empty_impl @@ -884,7 +899,7 @@ if w_iterable is not EMPTY_LIST: list_w = space.unpackiterable(w_iterable) if list_w: - w_list.implementation = RListImplementation(space, list_w) + w_list.implementation = make_implementation(space, list_w) return w_list.implementation = space.fromcache(State).empty_impl @@ -1304,9 +1319,8 @@ sorterclass = CustomKeySort else: sorterclass = SimpleSort - impl = w_list.implementation.to_rlist() - w_list.implementation = impl - items = impl.list_w + impl=w_list.implementation + items = impl.get_list_w() sorter = sorterclass(items, impl.length()) sorter.space = space sorter.w_cmp = w_cmp @@ -1349,8 +1363,7 @@ mucked = w_list.implementation.length() > 0 # put the items back into the list - impl.list_w = sorter.list - w_list.implementation = impl + w_list.implementation = make_implementation(space, sorter.list) if mucked: raise OperationError(space.w_ValueError, Index: objspace/std/test/test_listmultiobject.py =================================================================== --- objspace/std/test/test_listmultiobject.py (revision 44498) +++ objspace/std/test/test_listmultiobject.py (working copy) @@ -144,3 +144,15 @@ def teardown_class(cls): _set_chunk_size_bits(cls.chunk_size_bits) + def test_chunklist_is_implementation_used(self): + import __pypy__ + l = ["1", "2", "3", "4", "5"] + assert "ChunkedListImplementation" in __pypy__.internal_repr(l) + l = list(["1", "2", "3", "4", "5"]) + assert "ChunkedListImplementation" in __pypy__.internal_repr(l) + l=[] + l.append('a') + assert "ChunkedListImplementation" in __pypy__.internal_repr(l) + l = ["6", "8", "3", "1", "5"] + l.sort() + assert "ChunkedListImplementation" in __pypy__.internal_repr(l) From cfbolz at gmx.de Mon Jun 25 15:57:10 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Mon, 25 Jun 2007 15:57:10 +0200 Subject: [pypy-dev] Some bugfixes to the multilistimplementation In-Reply-To: <467FC711.100@jippii.fi> References: <467FC711.100@jippii.fi> Message-ID: <348899050706250657u4b1261d4h41da130eac059546@mail.gmail.com> Hi Elmo! 2007/6/25, Elmo M?ntynen : > I've made the different [slow] list implementations even slower ! good! > While integrating the blist implementation, I found out that the chosen > implementation isn't even used when instantiating with "list()", and > neither in some other usecases. By correcting that, a bug in the chunked > implementation got uncovered, which I also fixed. I also added > "make_implementation_with_one_item()" to have the instantiation of the > implementations in one place so that it's easier to add new ones. The diff looks good to me. Note that there are still bugs in the multilist stuff, it wasn't heavily tested yet. > If they seem good, could someone commit these changes (is that diff > format good?). Would you trust me with commit rights, or should I still > send patches? I think you should get commit rights. Send a mail to michael ( micahel at gmail.com ) with your username and ssh key. Standard disclaimer: You have to agree to put your changes under the MIT license, you have to promise to write tests and you should follow the coding guide. Then you can just commit your work. > The test demonstrates the bugs, but I thougth of later adding a > BaseAppTest_ListMultiObject class with magic to test if the > implementations that the subclasses test are really used. Does that > sound good? yes, very good. I vaguely had something like this in mind, but never got around to it. Thanks for doing this! Carl Friedrich From rxe at ukshells.co.uk Mon Jun 25 22:01:35 2007 From: rxe at ukshells.co.uk (Richard Emslie) Date: Mon, 25 Jun 2007 13:01:35 -0700 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <467449FC.2040909@genesilico.pl> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> <467449FC.2040909@genesilico.pl> Message-ID: On Jun 16, 2007, at 1:37 PM, Maciek Fijalkowski wrote: >> > Just a sidenote - rffi supports (and uses) macros. Not sure how this > will look like in a ctypes-based solution. > if you are talking c macros - this could be very problematic for the llvm backend. cheers - Richard From simon at arrowtheory.com Tue Jun 26 20:40:26 2007 From: simon at arrowtheory.com (Simon Burton) Date: Tue, 26 Jun 2007 11:40:26 -0700 Subject: [pypy-dev] ep 2007 talks Message-ID: <20070626114026.3fb5760e.simon@arrowtheory.com> I'd like to synchronize with Maciek and Antonio with regards talking about rpython at EP this year. My talk comes right after your rpython talk - i think we can perhaps share the burden (joy?) of talking about rpython a bit. Simon. From arigo at tunes.org Wed Jun 27 13:40:49 2007 From: arigo at tunes.org (Armin Rigo) Date: Wed, 27 Jun 2007 13:40:49 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <20070616171054.GA695@code0.codespeak.net> <467449FC.2040909@genesilico.pl> Message-ID: <20070627114049.GA9686@code0.codespeak.net> Hi Richard, On Mon, Jun 25, 2007 at 01:01:35PM -0700, Richard Emslie wrote: > > Just a sidenote - rffi supports (and uses) macros. Not sure how this > > will look like in a ctypes-based solution. > > if you are talking c macros - this could be very problematic for the > llvm backend. Anton (which I add to the CC's of this mail) proposed a solution for calling C APIs from llvm bytecode in a portable way, which would work for functions whose argument types are not exactly specified by the standards as well as for some kind of macros. The idea is to generate stub "helper" functions in C and compile them with llvm-gcc. The llvm bytecode that genllvm produces would then call the helper functions instead of calling directly the external C API. Shouldn't be a performance problem as llvm will inline the helpers agressively. Of course, it points again to the question of whether it's really worthwhile to have an llvm backend in PyPy or if just producing C and using llvm-gcc on everything wouldn't work just as well. On the other hand, it's interesting to note that there are other reasons for which exactly the same kind of helper functions would be useful; for example, to wrap a C library full of macros and unspecified types and constants, ctypes alone is not enough. A bientot, Armin. From anto.cuni at gmail.com Wed Jun 27 14:01:53 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Wed, 27 Jun 2007 14:01:53 +0200 Subject: [pypy-dev] ep 2007 talks In-Reply-To: <20070626114026.3fb5760e.simon@arrowtheory.com> References: <20070626114026.3fb5760e.simon@arrowtheory.com> Message-ID: <468251B1.9020100@gmail.com> Simon Burton wrote: > I'd like to synchronize with Maciek and Antonio with regards > talking about rpython at EP this year. My talk comes right after > your rpython talk - i think we can perhaps share the burden (joy?) > of talking about rpython a bit. Hi Simon, I think that talking about rpython only once makes a lot of sense. Currently we already wrote some slides about it, that you can find in extradoc/talk/ep2007/rpython.txt. As you can see we did not write much, just few slides to give an idea of what rpython is; after all, our intent is to tell people why the might be interested in it, not to give a tutorial about rpython. Do you think is enough or you was thinking about a deeper introduction? ciao Anto From rxe at ukshells.co.uk Wed Jun 27 22:55:13 2007 From: rxe at ukshells.co.uk (Richard Emslie) Date: Wed, 27 Jun 2007 13:55:13 -0700 Subject: [pypy-dev] Fwd: Work plan for PyPy References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> Message-ID: waaaah - forget to cc everyone :-( Begin forwarded message: > From: Richard Emslie > Date: June 27, 2007 10:56:39 AM PDT > To: Armin Rigo > Subject: Re: [pypy-dev] Work plan for PyPy > > Hi Armin! > > On Jun 27, 2007, at 4:40 AM, Armin Rigo wrote: > >> Hi Richard, >> >> On Mon, Jun 25, 2007 at 01:01:35PM -0700, Richard Emslie wrote: >>>> Just a sidenote - rffi supports (and uses) macros. Not sure how >>>> this >>>> will look like in a ctypes-based solution. >>> >>> if you are talking c macros - this could be very problematic for the >>> llvm backend. >> >> Anton (which I add to the CC's of this mail) proposed a solution for >> calling C APIs from llvm bytecode in a portable way, which would work >> for functions whose argument types are not exactly specified by the >> standards as well as for some kind of macros. The idea is to >> generate >> stub "helper" functions in C and compile them with llvm-gcc. The >> llvm >> bytecode that genllvm produces would then call the helper functions >> instead of calling directly the external C API. Shouldn't be a >> performance problem as llvm will inline the helpers agressively. >> > > ok, that is somewhat what we have now - except the stub functions > are high level and shared *somewhat* with genc. > > Generating c code in llvm backend would sort of defeat the point of > it IMHO. We already have a backend that generates c. On the other > hand, if we hand write the c code - it may be an neverending task > of adding hacks to the backend. For instance just getting at errno > is particularly problematic right now - and would involve a > function call from a constant value. which means injecting a few > lines assembly and a temporary variable. Easy to do, but it is > nice right now that we pretty much have a direct mapping from our > our lli form to llvm assembly. > > >> Of course, it points again to the question of whether it's really >> worthwhile to have an llvm backend in PyPy or if just producing C and >> using llvm-gcc on everything wouldn't work just as well. On the >> other >> hand, it's interesting to note that there are other reasons for which >> exactly the same kind of helper functions would be useful; for >> example, >> to wrap a C library full of macros and unspecified types and >> constants, >> ctypes alone is not enough. > > > Like you say, llvm-gcc (well at least v4) would probably produce > equivalent results with genc. The way we are using llvm right now > (as a static compiler) i'm not sure of its worthiness either of > keeping the backend alive. Other than the potential of accurate > gc, potential of jit (which is sort of orthogonal to this), > potential of producing vectorized code from rpython... > > > Cheers, > Richard > > > > > > From simon at arrowtheory.com Thu Jun 28 04:07:15 2007 From: simon at arrowtheory.com (Simon Burton) Date: Wed, 27 Jun 2007 19:07:15 -0700 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <4673C374.2060806@genesilico.pl> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <4673C374.2060806@genesilico.pl> Message-ID: <20070627190715.3b35c12f.simon@arrowtheory.com> Do you think we could extend rffi to be able to expose function calls to outside callers, so that eg. we can make nifty cpython extension modules ? (i think this is number 3 in my list) Simon. On Sat, 16 Jun 2007 13:03:16 +0200 Maciek Fijalkowski wrote: > Simon Burton wrote: > > On Thu, 14 Jun 2007 18:45:54 +0200 > > Armin Rigo wrote: > > > > > >> Any comments or objections? Are there people that already make heavy > >> use of rctypes, or the extension compiler? > >> > > > > Indeed. Mainly rctypes. > > > > As well as an ever expanding use of libc, > > we recently have interfaced to libSDL and cairo for some > > very funky rpython graphics apps. And we use libpython > > so that we can extract stuff from python modules. > > Also, we do byte shuffling with rctypes: reading binary data > > to/from buffers, etc. > > > > I hope we can do all this with rffi, but there are two really > > cool things about rctypes: > > 1) it runs on cpython (our main app takes 30minutes+ to compile) > > 2) it has an automatic code generator (ctypes can autogenerate > > python wrappers from C header files) > > > Very valid points indeed. The 1st point sounds unlikely, also because > you might use C macros with rffi, which somehow invalidates possible > uses on top of ctypes. What I do is I test (by compiling) several parts > (some small code snippets involving ext functions). That make debugging > a bit harder (reading C code), but pretty straightforward - for example > memory management is done by hand in this part, which makes you > responsible of tracking C structures and how long they need to live. > > The second point is much more likely to happen and to be better than > ctypes code generation. (Because rffi is way more Cish than ctypes are). > > Also it makes stuff a bit harder when you don't submit your bindings back. > > If you plan to come to the post-EP pypy sprint, we can sit a bit on the > automatic code generator for rffi. > > Cheers, > fijal > From fijal at genesilico.pl Thu Jun 28 11:03:34 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 28 Jun 2007 11:03:34 +0200 Subject: [pypy-dev] ep 2007 talks In-Reply-To: <468251B1.9020100@gmail.com> References: <20070626114026.3fb5760e.simon@arrowtheory.com> <468251B1.9020100@gmail.com> Message-ID: <46837966.7070400@genesilico.pl> Antonio Cuni wrote: > Simon Burton wrote: > >> I'd like to synchronize with Maciek and Antonio with regards >> talking about rpython at EP this year. My talk comes right after >> your rpython talk - i think we can perhaps share the burden (joy?) >> of talking about rpython a bit. >> > > > Hi Simon, > I think that talking about rpython only once makes a lot of sense. > > Currently we already wrote some slides about it, that you can find in > extradoc/talk/ep2007/rpython.txt. > > As you can see we did not write much, just few slides to give an idea of > what rpython is; after all, our intent is to tell people why the might > be interested in it, not to give a tutorial about rpython. > > Do you think is enough or you was thinking about a deeper introduction? > I would extremely enjoy it, hence you give us some reason why we're talking about this at all :) There are a bit of orthogonal issues in our rpython talk (js backend and so on), but mostly it would be good to share a bit. When do you arrive to Vilnius? (I'm arriving on 5th and I might have no network at all till 5th). From fijal at genesilico.pl Thu Jun 28 11:08:43 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Thu, 28 Jun 2007 11:08:43 +0200 Subject: [pypy-dev] about breaking RPython, RCTypes In-Reply-To: References: Message-ID: <46837A9B.8090606@genesilico.pl> Martijn Faassen wrote: > Hi there, > > I posted this to a longer thread the other day but have received no > answers to it whatsoever. Perhaps it got lost, so I'll do a repost. I > spent a bit of time writing this post, and I ask some questions in it > that I think should be answered. Perhaps by telling me I don't > understand what I'm talking about or that my analysis is entirely > incorrect. Perhaps it was simply too long? > > Whatever it is, I do think the PyPy community needs to be aware of how > technical decisions have an impact on attracting or repelling potential > contributors. "answering by silence", which happened so far, is one way > to make potential contributors worried. :) > Sorry for a late reply. I think that discussing things live at the EP makes a lot of sense :) Also I think that several points regarding what is going to happen needs clarfication and so on. Let's talk about it live. Cheers, fijal From arigo at tunes.org Thu Jun 28 13:15:38 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 28 Jun 2007 13:15:38 +0200 Subject: [pypy-dev] Fwd: Work plan for PyPy In-Reply-To: References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> Message-ID: <20070628111538.GA27708@code0.codespeak.net> Hi Richard, On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote: > >>bytecode that genllvm produces would then call the helper functions > >>instead of calling directly the external C API. Shouldn't be a > >>performance problem as llvm will inline the helpers agressively. > >> > > > >ok, that is somewhat what we have now - except the stub functions > >are high level and shared *somewhat* with genc. Precisely, by contrast I'm thinking about having lower level stub functions that are generated automatically from the lltype declarations. For example, an external ll function ptr would generate a simple stub, always the same. GenC and GenLLVM would share the code to produce the stubs. Similarly, getting at an external variable (or variable-like macro) like errno should be expressable in an lltype way (it's not right now) that gets turned into """int _rpyget_errno(void) { return errno; }""". > >Generating c code in llvm backend would sort of defeat the point of > >it IMHO. The difference is that the stubs would always be generated from simple templates, not from genc's complex logic that turns flow graphs into C code. A bientot, Armin. From arigo at tunes.org Thu Jun 28 13:51:56 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 28 Jun 2007 13:51:56 +0200 Subject: [pypy-dev] Fwd: Work plan for PyPy In-Reply-To: <1183010847.19411.31.camel@asl.dorms.spbu.ru> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <1183010847.19411.31.camel@asl.dorms.spbu.ru> Message-ID: <20070628115156.GB27708@code0.codespeak.net> Hi Anton, On Thu, Jun 28, 2007 at 10:07:27AM +0400, Anton Korobeynikov wrote: > 1. Use more or less portable runtime, which will "unify" such calls. For > example, one can use APR (HLVM, in fact, will use it) > > 2. Delegate stubs resolution to specialised version of lli. I think, > LLVM's itself OS abstraction library (libSystem) can be partly used. Unless I'm missing something, it seems that neither of these approaches would work to call generic C APIs - only the ones that are re-exported from either the APR or libSystem. Doesn't sound too attractive from my point of view. A bientot, Armin. From arigo at tunes.org Thu Jun 28 14:04:44 2007 From: arigo at tunes.org (Armin Rigo) Date: Thu, 28 Jun 2007 14:04:44 +0200 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070627190715.3b35c12f.simon@arrowtheory.com> References: <20070614164554.GA654@code0.codespeak.net> <20070615174454.6e87fb1a.simon@arrowtheory.com> <4673C374.2060806@genesilico.pl> <20070627190715.3b35c12f.simon@arrowtheory.com> Message-ID: <20070628120444.GA400@code0.codespeak.net> Hi Simon, On Wed, Jun 27, 2007 at 07:07:15PM -0700, Simon Burton wrote: > Do you think we could extend rffi to be able to > expose function calls to outside callers, so that eg. > we can make nifty cpython extension modules ? It would be a simple matter to have a way to force genc to use specific names from functions and struct names so that they can be called from hand-written C code too. Is that what you mean? A bientot, Armin. From cfbolz at gmx.de Thu Jun 28 16:46:19 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Thu, 28 Jun 2007 16:46:19 +0200 Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: . test In-Reply-To: <20070628134930.7F1688101@code0.codespeak.net> References: <20070628134930.7F1688101@code0.codespeak.net> Message-ID: <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com> Hi Jacub! 2007/6/28, jlg at codespeak.net : > Author: jlg > Date: Thu Jun 28 15:49:30 2007 > New Revision: 44599 > > Modified: > pypy/dist/pypy/lang/scheme/TODO.txt > pypy/dist/pypy/lang/scheme/object.py > pypy/dist/pypy/lang/scheme/test/test_parser.py > Log: > RPython direction for object.py > > Modified: pypy/dist/pypy/lang/scheme/object.py > ============================================================================== > --- pypy/dist/pypy/lang/scheme/object.py (original) > +++ pypy/dist/pypy/lang/scheme/object.py Thu Jun 28 15:49:30 2007 > @@ -139,10 +139,16 @@ > raise NotImplementedError > > def add_lst(ctx, lst): > - return apply_lst(ctx, lambda x, y: x + y, lst) > + def adder(x, y): > + return x + y > + > + return apply_lst(ctx, adder, lst) > > def mul_lst(ctx, lst): > - return apply_lst(ctx, lambda x, y: x * y, lst) > + def multiplier(x, y): > + return x * y > + > + return apply_lst(ctx, multiplier, lst) > > def apply_lst(ctx, fun, lst): > acc = None > This doesn't make mul_lst and add_lst any more rpython than before. In fact, lambda is fully rpython. The problem with this code is rather that you cannot have local functions in RPython. As I said, don't worry about making your code RPython too much yet. Cheers, Carl Friedrich From jgustak at gmail.com Thu Jun 28 17:11:40 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Thu, 28 Jun 2007 17:11:40 +0200 Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: . test In-Reply-To: <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com> References: <20070628134930.7F1688101@code0.codespeak.net> <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com> Message-ID: On 6/28/07, Carl Friedrich Bolz wrote: > Hi Jacub! > > 2007/6/28, jlg at codespeak.net : > > Author: jlg > > Date: Thu Jun 28 15:49:30 2007 > > New Revision: 44599 > > > > Modified: > > pypy/dist/pypy/lang/scheme/TODO.txt > > pypy/dist/pypy/lang/scheme/object.py > > pypy/dist/pypy/lang/scheme/test/test_parser.py > > Log: > > RPython direction for object.py > > > > Modified: pypy/dist/pypy/lang/scheme/object.py > > ============================================================================== > > --- pypy/dist/pypy/lang/scheme/object.py (original) > > +++ pypy/dist/pypy/lang/scheme/object.py Thu Jun 28 15:49:30 2007 > > @@ -139,10 +139,16 @@ > > raise NotImplementedError > > > > def add_lst(ctx, lst): > > - return apply_lst(ctx, lambda x, y: x + y, lst) > > + def adder(x, y): > > + return x + y > > + > > + return apply_lst(ctx, adder, lst) > > > > def mul_lst(ctx, lst): > > - return apply_lst(ctx, lambda x, y: x * y, lst) > > + def multiplier(x, y): > > + return x * y > > + > > + return apply_lst(ctx, multiplier, lst) > > > > def apply_lst(ctx, fun, lst): > > acc = None > > > > This doesn't make mul_lst and add_lst any more rpython than before. In > fact, lambda is fully rpython. The problem with this code is rather > that you cannot have local functions in RPython. I that case error messages from pylint: E:142:add_lst.: Using unavailable keyword 'lambda' E:145:mul_lst.: Using unavailable keyword 'lambda' Causes only confusion. > As I said, don't worry about making your code RPython too much yet. I will try to refrain. Cheers, Jakub Gustak From anto.cuni at gmail.com Thu Jun 28 17:26:32 2007 From: anto.cuni at gmail.com (Antonio Cuni) Date: Thu, 28 Jun 2007 17:26:32 +0200 Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: . test In-Reply-To: References: <20070628134930.7F1688101@code0.codespeak.net> <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com> Message-ID: <4683D328.6030206@gmail.com> Jakub Gustak wrote: > I that case error messages from pylint: > E:142:add_lst.: Using unavailable keyword 'lambda' > E:145:mul_lst.: Using unavailable keyword 'lambda' I would say that in this case pylint is wrong, though I agree than in most cases lambda is used in a way that is not allowed in RPython. ciao Anto From simon at arrowtheory.com Thu Jun 28 21:20:45 2007 From: simon at arrowtheory.com (Simon Burton) Date: Thu, 28 Jun 2007 12:20:45 -0700 Subject: [pypy-dev] ep 2007 talks In-Reply-To: <46837966.7070400@genesilico.pl> References: <20070626114026.3fb5760e.simon@arrowtheory.com> <468251B1.9020100@gmail.com> <46837966.7070400@genesilico.pl> Message-ID: <20070628122045.caa702c9.simon@arrowtheory.com> On Thu, 28 Jun 2007 11:03:34 +0200 Maciek Fijalkowski wrote: > > > Do you think is enough or you was thinking about a deeper introduction? > > > I would extremely enjoy it, hence you give us some reason why we're > talking about this at all :) There are a bit of orthogonal issues in our > rpython talk (js backend and so on), but mostly it would be good to > share a bit. When do you arrive to Vilnius? (I'm arriving on 5th and I > might have no network at all till 5th). I will arrive on the 8th, in the afternoon. Simon. From rxe at ukshells.co.uk Fri Jun 29 01:40:37 2007 From: rxe at ukshells.co.uk (Richard Emslie) Date: Thu, 28 Jun 2007 16:40:37 -0700 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <20070628111538.GA27708@code0.codespeak.net> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <20070628111538.GA27708@code0.codespeak.net> Message-ID: <6AC2ADE5-7BA3-4C6E-BF40-8952284316BD@ukshells.co.uk> Hi Armin, On Jun 28, 2007, at 4:15 AM, Armin Rigo wrote: > Hi Richard, > > On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote: >>>> bytecode that genllvm produces would then call the helper functions >>>> instead of calling directly the external C API. Shouldn't be a >>>> performance problem as llvm will inline the helpers agressively. >>>> >>> >>> ok, that is somewhat what we have now - except the stub functions >>> are high level and shared *somewhat* with genc. > > Precisely, by contrast I'm thinking about having lower level stub > functions that are generated automatically from the lltype > declarations. > For example, an external ll function ptr would generate a simple stub, > always the same. GenC and GenLLVM would share the code to produce the > stubs. Similarly, getting at an external variable (or variable-like > macro) like errno should be expressable in an lltype way (it's not > right > now) that gets turned into """int _rpyget_errno(void) { return > errno; }""". > Ok - cool. I was hoping to propose something along these lines - but was unsure how to go about it. This would be ideal and work great for backends. I am guessing all memory management with this would be explicitly controlled from outside the stubs? >>> Generating c code in llvm backend would sort of defeat the point of >>> it IMHO. > > The difference is that the stubs would always be generated from simple > templates, not from genc's complex logic that turns flow graphs into C > code. > Yeah - this sounds fantastic. No clue how to do it - but great! :-) Well - I'm in the process of migrating back home... again. Once i get back though I can devote lots of time to this. I doubt i can make the sprint, but will try. Cheers, Richard From jgustak at gmail.com Fri Jun 29 01:40:52 2007 From: jgustak at gmail.com (Jakub Gustak) Date: Fri, 29 Jun 2007 01:40:52 +0200 Subject: [pypy-dev] scheme interpreter [status report] Message-ID: Hello. Scheme interpreter is starting look like its going to do some serious stuff in the near future, or maybe I am just too enthusiastic about it. What we have now: - We can execute simple expressions like: (+ 1 2 1.3 -2) Nested ones also works. - (define ) works with a given context, so: (define var 2) (+ var 40) => 42 - Conditional (if ) seem to work fine, is not arbitrary. pypy/lang/scheme/test/test_eval.py Gives you the best overview. What will be done in the nearest future: - comparison primitives: = < > eq? eqv? - (cons ) (car ) (cdr ) so we can play with lists. That kind of stuff. You can take a look at: pypy/lang/scheme/TODO.txt Will postpone quotations and symbols implementation for a while. I have to think about it, how much of it is parsing matter. Then i would like to work on (lambda ...). This will probably take some time to think again execution context, make a hard distinction between procedures and macros and a lot of stuff, that I don't even dare to think about now :-). Before EP I would like to make lambdas work and also have some tuning done (whatever it means), so on the sprint we could focus on really interesting stuff instead of refactoring and bug fixes. I would like to refine what I would like to do during the sprint, but not sure right now. Any ideas, what would be worth trying then? Cheers, Jakub Gustak From rxe at ukshells.co.uk Fri Jun 29 01:53:41 2007 From: rxe at ukshells.co.uk (Richard Emslie) Date: Thu, 28 Jun 2007 16:53:41 -0700 Subject: [pypy-dev] Work plan for PyPy In-Reply-To: <1183010847.19411.31.camel@asl.dorms.spbu.ru> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <1183010847.19411.31.camel@asl.dorms.spbu.ru> Message-ID: Hi Anton, On Jun 27, 2007, at 11:07 PM, Anton Korobeynikov wrote: > Hello, Richard. > >> Generating c code in llvm backend would sort of defeat the point of >> it IMHO. We already have a backend that generates c. On the other >> hand, if we hand write the c code - it may be an neverending task >> of adding hacks to the backend. For instance just getting at errno >> is particularly problematic right now - and would involve a >> function call from a constant value. which means injecting a few >> lines assembly and a temporary variable. Easy to do, but it is >> nice right now that we pretty much have a direct mapping from our >> our lli form to llvm assembly. ^^^ ahem, "ssi form" > I've talked a little bit with Chris Lattner about this (calling C API) > issue. Except stubs/wrappers it seems there are ways to call C API > > 1. Use more or less portable runtime, which will "unify" such > calls. For > example, one can use APR (HLVM, in fact, will use it) > > 2. Delegate stubs resolution to specialised version of lli. I think, > LLVM's itself OS abstraction library (libSystem) can be partly used. > thanks for these ideas. i guess Armin has replied already. I am guess the biggest concern is how to wrap libraries outside what these support. > I personally like the second variant more. Anyway, some specialised > lli > variant (or some application, which will call LLVM's execution engine) > will be needed, if pypy would want some extended features (like > accurate > GC, vectorisation, etc) from LLVM in the future. > > We never managed to run pypy under lli. In was one of the early goals, but it were waiting until gc to mature in the pypy infrastructure. Since that has happened, the llvm backend hasn't had much active development. Right now it only works with boehm, and not surprisingly the interaction of llvm's jit and boehm doesn't go very well in hand. I still think it would be fun and interesting to see pypy-llvm working under lli. Cheers, Ricahrd From cfbolz at gmx.de Fri Jun 29 12:52:54 2007 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Fri, 29 Jun 2007 12:52:54 +0200 Subject: [pypy-dev] scheme interpreter [status report] In-Reply-To: References: Message-ID: <348899050706290352l113a4b96y5cb2672f45c84842@mail.gmail.com> Hi Jakub (sorry for mis-spelling your name last time)! 2007/6/29, Jakub Gustak : > Scheme interpreter is starting look like its going to do some serious > stuff in the near future, or maybe I am just too enthusiastic about > it. > > What we have now: > - We can execute simple expressions like: (+ 1 2 1.3 -2) > Nested ones also works. > - (define ) works with a given context, so: > (define var 2) > (+ var 40) => 42 > - Conditional (if ) seem to work fine, is > not arbitrary. All sounds good! > pypy/lang/scheme/test/test_eval.py > Gives you the best overview. > > What will be done in the nearest future: > - comparison primitives: = < > eq? eqv? > - (cons ) (car ) (cdr ) so we can play with lists. > That kind of stuff. I would do cons, car, cdr first (they should be easy anyway). > You can take a look at: > pypy/lang/scheme/TODO.txt > > Will postpone quotations and symbols implementation for a while. I > have to think about it, how much of it is parsing matter. A lof of it is a matter of parsing, yes. I can help you there, if you want me to. > Then i would like to work on (lambda ...). This will probably take > some time to think again execution context, make a hard distinction > between procedures and macros and a lot of stuff, that I don't even > dare to think about now :-). Simple lambdas should be kind of easy, I would hope. > Before EP I would like to make lambdas work and also have some tuning > done (whatever it means), so on the sprint we could focus on really > interesting stuff instead of refactoring and bug fixes. > > I would like to refine what I would like to do during the sprint, but > not sure right now. Any ideas, what would be worth trying then? Good things to do on the sprint are things were you need the help of other PyPy people. Two ideas: - thinking about how to implement call/cc with the primitives stackless provides - making the interpreter rpython Cheers, Carl Friedrich From fijal at genesilico.pl Sat Jun 30 13:12:57 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 30 Jun 2007 13:12:57 +0200 Subject: [pypy-dev] Fwd: Work plan for PyPy In-Reply-To: <20070628111538.GA27708@code0.codespeak.net> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <20070628111538.GA27708@code0.codespeak.net> Message-ID: <46863AB9.1030102@genesilico.pl> Armin Rigo wrote: > Hi Richard, > > On Wed, Jun 27, 2007 at 01:55:13PM -0700, Richard Emslie wrote: > >>>> bytecode that genllvm produces would then call the helper functions >>>> instead of calling directly the external C API. Shouldn't be a >>>> performance problem as llvm will inline the helpers agressively. >>>> >>>> >>> ok, that is somewhat what we have now - except the stub functions >>> are high level and shared *somewhat* with genc. >>> > > Precisely, by contrast I'm thinking about having lower level stub > functions that are generated automatically from the lltype declarations. > For example, an external ll function ptr would generate a simple stub, > always the same. GenC and GenLLVM would share the code to produce the > stubs. Similarly, getting at an external variable (or variable-like > macro) like errno should be expressable in an lltype way (it's not right > now) that gets turned into """int _rpyget_errno(void) { return errno; }""". > Just a little correction - you can express c-level constant in lltype. There is CConstant class for that. Cheers, fijal From fijal at genesilico.pl Sat Jun 30 13:21:26 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 30 Jun 2007 13:21:26 +0200 Subject: [pypy-dev] [pypy-svn] r44599 - in pypy/dist/pypy/lang/scheme: . test In-Reply-To: References: <20070628134930.7F1688101@code0.codespeak.net> <348899050706280746i26bdbcbfif7da2c423c792a12@mail.gmail.com> Message-ID: <46863CB6.3010403@genesilico.pl> Jakub Gustak wrote: > I that case error messages from pylint: > E:142:add_lst.: Using unavailable keyword 'lambda' > E:145:mul_lst.: Using unavailable keyword 'lambda' > > Causes only confusion. > > I would not trust pylint too much. being rpython is quite complex issue. and trying to be rpython-compatible in pylint sense makes little sense to me (ie pylint will complain about all the surrounding initialization code, which is perfectly fine and even much more elegant than writing it all by hand). Making scheme more-rpythonish sounds like a nice task for a sprint. Cheers, fijal From arigo at tunes.org Sat Jun 30 13:49:21 2007 From: arigo at tunes.org (Armin Rigo) Date: Sat, 30 Jun 2007 13:49:21 +0200 Subject: [pypy-dev] Fwd: Work plan for PyPy In-Reply-To: <46863AB9.1030102@genesilico.pl> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <20070628111538.GA27708@code0.codespeak.net> <46863AB9.1030102@genesilico.pl> Message-ID: <20070630114921.GC1568@code0.codespeak.net> Hi Maciek, On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote: > Just a little correction - you can express c-level constant in lltype. > There is CConstant class for that. True, although I'm not sure it works for non-integer constants (e.g. static data). Also, I suppose that it could be abused to represent "errno" but it looks fragile. A bientot, Armin. From fijal at genesilico.pl Sat Jun 30 13:56:03 2007 From: fijal at genesilico.pl (Maciek Fijalkowski) Date: Sat, 30 Jun 2007 13:56:03 +0200 Subject: [pypy-dev] Fwd: Work plan for PyPy In-Reply-To: <20070630114921.GC1568@code0.codespeak.net> References: <30D05AC9-1A9E-4E90-9E79-C587AA81978C@ukshells.co.uk> <20070628111538.GA27708@code0.codespeak.net> <46863AB9.1030102@genesilico.pl> <20070630114921.GC1568@code0.codespeak.net> Message-ID: <468644D3.9040908@genesilico.pl> Armin Rigo wrote: > Hi Maciek, > > On Sat, Jun 30, 2007 at 01:12:57PM +0200, Maciek Fijalkowski wrote: > >> Just a little correction - you can express c-level constant in lltype. >> There is CConstant class for that. >> > > True, although I'm not sure it works for non-integer constants (e.g. > static data). Also, I suppose that it could be abused to represent > "errno" but it looks fragile. > ? The primary reason for creating this class was to use it for errno, so I don't think it's an abuse. Anyway, I think it should be extended to present some non-primitive types (it works for any pre-defined primitive type right now). Cheers, fijal