From jspaans at thehealthagency.com Tue Apr 1 17:23:22 2008 From: jspaans at thehealthagency.com (Jasper Spaans) Date: Tue, 1 Apr 2008 17:23:22 +0200 Subject: [py-dev] svnwc.auth patchlet Message-ID: Hi py-dev, I'm currently exploring the py.path module, and am trying to get it svnwc to work with svn locking. The following would be nice to have, as the current version doesn't provide a method to find out if a file has been locked: --- path/svn/wccommand.py~ 2008-04-01 17:13:09.000000000 +0200 +++ path/svn/wccommand.py 2008-04-01 17:13:09.000000000 +0200 @@ -644,7 +644,10 @@ if 'lastchangeddate' in d: self.mtime = parse_wcinfotime(d['lastchangeddate']) self.time = self.mtime * 1000000 - + if 'lockowner' in d: + self.lock = { 'owner': d['lockowner'], + 'mtime': parse_wcinfotime(d['lockcreated']) } + def __eq__(self, other): return self.__dict__ == other.__dict__ Proof of concept of the output: >>> import py >>> x = py.path.svnwc('/tmp/svnstoragetest-TMUVVD/root.xml') >>> dir(x.info()) ['__doc__', '__eq__', '__init__', '__module__', 'created_rev', 'kind', 'last_author', 'lock', 'mtime', 'path', 'rev', 'size', 'time', 'url'] >>> x.info().lock {'owner': 'foo', 'mtime': 1207067232} Cheers, -- Jasper Spaans http://www.thehealthagency.com/ From cfbolz at gmx.de Tue Apr 1 18:32:31 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 01 Apr 2008 18:32:31 +0200 Subject: [py-dev] svnwc.auth patchlet In-Reply-To: References: Message-ID: <47F2639F.8000609@gmx.de> Hi Jasper, Jasper Spaans wrote: > I'm currently exploring the py.path module, and am trying to get it > svnwc to work with svn locking. The following would be nice to have, > as the current version doesn't provide a method to find out if a file > has been locked: > > --- path/svn/wccommand.py~ 2008-04-01 17:13:09.000000000 +0200 > +++ path/svn/wccommand.py 2008-04-01 17:13:09.000000000 +0200 > @@ -644,7 +644,10 @@ > if 'lastchangeddate' in d: > self.mtime = parse_wcinfotime(d['lastchangeddate']) > self.time = self.mtime * 1000000 > - > + if 'lockowner' in d: > + self.lock = { 'owner': d['lockowner'], > + 'mtime': parse_wcinfotime(d['lockcreated']) } > + > def __eq__(self, other): > return self.__dict__ == other.__dict__ > > > Proof of concept of the output: > >>> import py > >>> x = py.path.svnwc('/tmp/svnstoragetest-TMUVVD/root.xml') > >>> dir(x.info()) > ['__doc__', '__eq__', '__init__', '__module__', 'created_rev', 'kind', > 'last_author', 'lock', 'mtime', 'path', 'rev', 'size', 'time', 'url'] > >>> x.info().lock > {'owner': 'foo', 'mtime': 1207067232} Could you write a few tests for that feature too? Otherwise I can't really apply the patch. Thanks, Carl Friedrich From jspaans at thehealthagency.com Tue Apr 1 18:59:45 2008 From: jspaans at thehealthagency.com (Jasper Spaans) Date: Tue, 1 Apr 2008 18:59:45 +0200 Subject: [py-dev] svnwc.auth patchlet In-Reply-To: <47F2639F.8000609@gmx.de> References: <47F2639F.8000609@gmx.de> Message-ID: Hi Carl Friedrich, > Could you write a few tests for that feature too? Otherwise I can't > really apply the patch. See the attached file for a reimplementation and some tests. Cheers, -- Jasper Spaans http://www.thehealthagency.com/ -------------- next part -------------- A non-text attachment was scrubbed... Name: svn.info.lock.patch Type: application/octet-stream Size: 2500 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20080401/4037d53d/attachment.obj From jopython at gmail.com Mon May 5 17:01:44 2008 From: jopython at gmail.com (Joe Python) Date: Mon, 5 May 2008 11:01:44 -0400 Subject: [py-dev] Building greenlet 0.9.1 in Solaris 10 Message-ID: Building greenlet 0.9.1 in Solaris 10 While trying to build the c extension within Solaris 10, i get the following error message: bash-3.00# /usr/local/bin/python setup.py build running build running build_ext building 'greenlet' extension creating build creating build/temp.solaris-2.10-sun4u-2.5 gcc -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -I/usr/local/include/python2.5 -c greenlet.c -o build/temp.solaris-2.10-sun4u-2.5/greenlet.o In file included from slp_platformselect.h:16, from greenlet.c:251: switch_sparc_sun_gcc.h: In function `slp_switch': switch_sparc_sun_gcc.h:56: error: `_cst' undeclared (first use in this function) switch_sparc_sun_gcc.h:56: error: (Each undeclared identifier is reported only once switch_sparc_sun_gcc.h:56: error: for each function it appears in.) switch_sparc_sun_gcc.h:52: error: PIC register `%l7' clobbered in `asm' error: command 'gcc' failed with exit status 1 bash-3.00# gcc -v Reading specs from /usr/sfw/lib/gcc/sparc-sun-solaris2.10/3.4.3/specs Configured with: /gates/sfw10/builds/sfw10-gate/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++ --enable-shared Thread model: posix gcc version 3.4.3 (csl-sol210-3_4-branch+sol_rpath) Python build is : Python 2.5.1 (r251:54863, May 16 2007, 19:58:05) [GCC 3.4.6] on sunos5 bash-3.00# cat /etc/release Solaris 10 3/05 s10_74L2a SPARC -------------- next part -------------- An HTML attachment was scrubbed... URL: http://codespeak.net/pipermail/py-dev/attachments/20080505/e7614ba3/attachment.htm From lijssies at gp-net.nl Tue May 6 08:34:44 2008 From: lijssies at gp-net.nl (Gerard Petersen) Date: Tue, 06 May 2008 08:34:44 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet Message-ID: <200805060834.44919.lijssies@gp-net.nl> Hi All, I write this email as a follow up to a chat on #pylib with cfbolz. Bare with me in the code example and explanation because I'm only into python for six months or so. The reason for this request is that I'm building an application with wx.Python that does remote file handling based on execnet. I was told that it's good practice to have exceptions raised in your code as low as possible, and handle them as high in your code as possible. A brief setup of my app (just to illustrate) gui/mainapp (the gui) dwlib/remote (the remote module) In the app (the gui) I call a function from dwlib that gets a remote dirlisting. If this fails the gui pops up a dialog with advice on how to continu. As the docs of execnet state. Errors on the remote side are raised as OSErrors and contain a textual version of the remote exception. Two that I forced (simulated): OSError: [Errno 13] Permission denied: '/some/where' OSError: [Errno 2] No such file or directory: '/some/whe' This is correct behaviour but I would like to have them as actual exceptions back up to the interface. In this case I have to parse the string in the dwlib module with something like "if value.startswith(OSError')" and raise another exception my self. So the request is: Is it possible to actually have the Exception from the remote side passed on as an exception object to the local side (or recreate one?!?). Attached is a script that simulates the problem. There are SSH keys in place, so (local)host access is functioning. As said I'm not a python guru (yet). so please don't shoot my code ... :-) I hope all is clear. Questions on the mather (and tips) are very welcome. Kind regards, Gerard. -- $Grtz =~ Gerard; ~ :wq! -------------- next part -------------- A non-text attachment was scrubbed... Name: fileread.py Type: application/x-python Size: 939 bytes Desc: not available Url : http://codespeak.net/pipermail/py-dev/attachments/20080506/036d395d/attachment-0001.bin From johnny at johnnydebris.net Tue May 6 09:38:20 2008 From: johnny at johnnydebris.net (Guido Wesdorp) Date: Tue, 06 May 2008 09:38:20 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <200805060834.44919.lijssies@gp-net.nl> References: <200805060834.44919.lijssies@gp-net.nl> Message-ID: <48200AEC.40505@johnnydebris.net> Gerard Petersen wrote: > Errors on the remote side are raised > as OSErrors and contain a textual version of the remote exception. They are actually raised as RemoteError (py.__.execnet.channel.RemoteError) instances, but I understand the frustration. The RemoteError instance doesn't hold a reference to the original error, which can be annoying. I think one of the problems could be that the remote exception may not be valid on the local machine: for instance if you import a library 'foo' that raises a (self-invented) FooException on the remote side, that library may not be available locally so it may be that FooException can not be instantiated/raised there. I'm not sure how to deal with this problem. I do know execnet has the possibility to pickle (?) and send over exceptions, though... I think what you could do in your code is catch the specific exceptions you're interested in on the remote side yourself, and send those over the channel if something goes wrong, rather than the data you would have sent if it would have succeeded. Of course this means that both the client and the server (never sure which is which, though ;) will become more complex, but I don't think it's too bad: g = py.execnet.PopenGateway() g.execute("""\ import os try: l = os.listdir('/some/dir/that/may/have/problems') except OSError, e: # could even be 'Exception' instead of 'OSError' channel.send(e) else: channel.send(l) """) ret = channel.receive() if isinstance(ret, Exception): raise ret # raise the remote exception again assert isinstance(ret, list) # here you know 'ret' is a list I understand this isn't optimal, and doesn't solve the problem entirely (just for the exceptions you deal with explicitly), but at least you can now get back to work again... ;) Cheers, Guido From cfbolz at gmx.de Tue May 6 09:45:03 2008 From: cfbolz at gmx.de (Carl Friedrich Bolz) Date: Tue, 06 May 2008 09:45:03 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <48200AEC.40505@johnnydebris.net> References: <200805060834.44919.lijssies@gp-net.nl> <48200AEC.40505@johnnydebris.net> Message-ID: <48200C7F.3030202@gmx.de> Guido Wesdorp wrote: > Gerard Petersen wrote: >> Errors on the remote side are raised >> as OSErrors and contain a textual version of the remote exception. > They are actually raised as RemoteError > (py.__.execnet.channel.RemoteError) instances, but I understand the > frustration. The RemoteError instance doesn't hold a reference to the > original error, which can be annoying. > > I think one of the problems could be that the remote exception may not > be valid on the local machine: for instance if you import a library > 'foo' that raises a (self-invented) FooException on the remote side, > that library may not be available locally so it may be that FooException > can not be instantiated/raised there. I'm not sure how to deal with this > problem. I do know execnet has the possibility to pickle (?) and send > over exceptions, though... Execnet is using only marshal, afaik. However, it should be possible to be at least a bit more informative and give RemoteError a name attribute or so that gives the name of the remote exception class. Just giving some message with traceback and everything makes it annoying to do anything with the exception object programmatically. Cheers, Carl Friedrich From johnny at johnnydebris.net Tue May 6 12:54:02 2008 From: johnny at johnnydebris.net (Guido Wesdorp) Date: Tue, 06 May 2008 12:54:02 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <48200C7F.3030202@gmx.de> References: <200805060834.44919.lijssies@gp-net.nl> <48200AEC.40505@johnnydebris.net> <48200C7F.3030202@gmx.de> Message-ID: <482038CA.4090504@johnnydebris.net> Carl Friedrich Bolz wrote: > Execnet is using only marshal, afaik. However, it should be possible to > be at least a bit more informative and give RemoteError a name attribute > or so that gives the name of the remote exception class. Just giving > some message with traceback and everything makes it annoying to do > anything with the exception object programmatically. > > Yes, I agree... Although I still think it's not easy to find a good way to send all the information (exception instance + traceback would be nice obviously) back over the channel, it would certainly be useful to do the minimal (set seperate instance name + message on the RemoteError instance for instance) compared to having a single explanation string... Cheers, Guido From faassen at startifact.com Tue May 6 16:34:45 2008 From: faassen at startifact.com (Martijn Faassen) Date: Tue, 06 May 2008 16:34:45 +0200 Subject: [py-dev] MAX_PATH on windows Message-ID: Hi there, Apparently the Windows API has a limitation in how long paths can get, set to 256 characters!(!!) This can break applications on Windows, including Py. Here are some relevant resources: http://mail.python.org/pipermail/python-bugs-list/2006-June/033754.html http://mail.python.org/pipermail/python-bugs-list/2007-March/037810.html http://msdn.microsoft.com/en-us/library/aa365247.aspx There is some discussion that this could be fixed in Python 2.5. It's also however not 100% clear to me, looking at the discussions, whether it really is. I'm on Python 2.4 myself... I think at the very least it'd be good to have a test case in Py that tries to trigger this error. Apparently a workaround is to prefix windows paths by "\\?\". Paths can then be a lot longer. I'm not sure how this interacts with the Python use of these APIs, however. If the workaround would really work, py could generate its windows paths with that prefix. Regards, Martijn From lijssies at gp-net.nl Wed May 7 08:30:29 2008 From: lijssies at gp-net.nl (Gerard Petersen) Date: Wed, 07 May 2008 08:30:29 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <482038CA.4090504@johnnydebris.net> References: <200805060834.44919.lijssies@gp-net.nl> <48200C7F.3030202@gmx.de> <482038CA.4090504@johnnydebris.net> Message-ID: <200805070830.29349.lijssies@gp-net.nl> Guido, Thanx for the code snippet. Initially I wanted to keep the code that goes remote as clean as possible (somebody told me ... ;-) but the code you came up with does seem good. The actual issue is somewhat more elaborate. I also have remote functions that read files, and a predicatble situation is that a file might not be there remotely. Since this is a possibiltity and not an exception as such, (next to a "permission denied" which is). There is also the possibility to return an error value along all the time. You then would still need some error handling remotely but the last line in your function would be something like "return contents, error". "Error" containing a level from zero upwards. I'll see if I can combine a setup like that with the "isinstance" function. More on the way, Thanx guys!! Regards, Gerard. On Tuesday 06 May 2008 12:54:02 Guido Wesdorp wrote: > Carl Friedrich Bolz wrote: > > Execnet is using only marshal, afaik. However, it should be possible to > > be at least a bit more informative and give RemoteError a name attribute > > or so that gives the name of the remote exception class. Just giving > > some message with traceback and everything makes it annoying to do > > anything with the exception object programmatically. > > Yes, I agree... Although I still think it's not easy to find a good way > to send all the information (exception instance + traceback would be > nice obviously) back over the channel, it would certainly be useful to > do the minimal (set seperate instance name + message on the RemoteError > instance for instance) compared to having a single explanation string... > > Cheers, > > Guido -- $Grtz =~ Gerard; ~ :wq! From walter at livinglogic.de Wed May 7 12:04:29 2008 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Wed, 07 May 2008 12:04:29 +0200 Subject: [py-dev] Request for enhancing Exception handling in execnet In-Reply-To: <200805070830.29349.lijssies@gp-net.nl> References: <200805060834.44919.lijssies@gp-net.nl> <48200C7F.3030202@gmx.de> <482038CA.4090504@johnnydebris.net> <200805070830.29349.lijssies@gp-net.nl> Message-ID: <48217EAD.60905@livinglogic.de> Gerard Petersen wrote: > Guido, > > Thanx for the code snippet. Initially I wanted to keep the code that goes > remote as clean as possible (somebody told me ... ;-) but > the code you came up with does seem good. > > The actual issue is somewhat more elaborate. I also have remote functions that > read files, and a predicatble situation is that a file might not be there > remotely. Since this is a possibiltity and not an exception as such, (next to > a "permission denied" which is). There is also the possibility to return an > error value along all the time. You then would still need some error handling > remotely but the last line in your function would be something like "return > contents, error". > "Error" containing a level from zero upwards. I'll see if I can combine a > setup like that with the "isinstance" function. > > More on the way, Thanx guys!! You might take a look at the ll.url module which is part of XIST (available from: http://www.livinglogic.de/Python/Download.html ll.url implements an ssh URL scheme which supports all file operations remotely: >>> from ll import url >>> u = url.URL("ssh://root at www.example.org/~/") >>> u.listdir(context=url.Context()) [URL('.bash_history'), URL('.profile'), URL('.bashrc'), ... >>> u = url.URL("ssh://root at www.example.org/~/.bash_history") >>> f = u.open("rb", context=url.Context()) >>> f.readline() '/etc/init.d/tomcat stop\n' >>> f.readline() 'ps waux | grep tomcat\n' It mirrors exceptions locally: >>> u = url.URL("ssh://root at www.example.org/~/does-not-exist") >>> f = u.open("rb", context=url.Context()) [...] IOError: [Errno 2] No such file or directory: '/root/does-not-exist' (of course this only works for builtin exceptions) And of course ll.url uses py.execnet for communicating with the remote host. Hope that helps! Servus, Walter From blep at users.sourceforge.net Fri Jun 27 10:45:37 2008 From: blep at users.sourceforge.net (Baptiste Lepilleur) Date: Fri, 27 Jun 2008 10:45:37 +0200 Subject: [py-dev] py.test bug: large stack trace produced with full source dump on import failure Message-ID: I have run into an issue with py.test which make it pretty much unsable with the following scenario: - a test module import another module - the import fails due to an exception thrown by the other module This scenario raise two bugs (hopefully this is the scenario to reproduce this bug): - Py.test produces a stack trace with a full source dump of the imported module source - Information about the exception that caused the failure is missing. The source does not end line 2269, so my guess is that py.test dump the source until the source line where the exception was raised. I believe py.test should truncate the source to show only the relevant part, that is ignore the 2000 lines until the global variable declaration SCHEMA_SCHEMA in this case. To illustrate the issue, I've put excerpt from both the standard python stacktrace and the py.test failure below: *** Standard python traces: ********************** --- Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import test_sbuildformat Traceback (most recent call last): File "", line 1, in File "E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\testing\test_sbuildformat.py", line 4, in import sbuilder.sbuildfileformat as sbuildfileformat File "E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\sbuildfileformat.py", line 2507, in SCHEMA_SCHEMA = getSchemaSchema() File "E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\sbuildfileformat.py", line 2269, in getSchemaSchema error_log.makeErrorReport( tokenizer.getNormalizedInput(), 13 ) sbuilder.sbuildfileformat.SchemaTypeParameterError >>> --- **** Py.test traces (abrievated, full source dump is 120KO) ********************************************** ============================= test process starts ============================= executable: c:\Python25\python.exe (2.5.2-final-0) using py lib: c:\Python25\lib\site-packages\py test_sbuilder.py[1] . test_sbuildformat.py - FAILED TO LOAD MODULE _______________________________________________________________________________ ________________________________ entrypoint: _________________________________ from __future__ import generators import types import re > import sbuilder.sbuildfileformat as sbuildfileformat [E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\testing\test_sbuildformat.py:4] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ import types import sets import weakref [....] 120KO later: > SCHEMA_SCHEMA = getSchemaSchema() [E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\sbuildfileformat.py:2507] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ def getSchemaSchema(): """Returns a document representing a schema of a schema. This can be used to validate a schema. Exceptions: - DocumentParsingError if failed to parse the schema text document (should never occurs, it is a bug introduced when updating the schema). - """ error_log = ErrorLog( synchronous_display = False ) error_memo = error_log.errorMemo() tokenizer = Tokenizer( getSchemaTextDocument(), 'schema.schema', error_log ) parser = Parser( tokenizer, error_log ) schema_value = parser.parse() if schema_value is None: raise DocumentParsingError, \ error_log.makeErrorReport( tokenizer.getNormalizedInput(), 13 ) try: schema_schema = SchemaBuilder(error_log).getSchema( schema_value ) if schema_schema and not schema_schema.check( error_log ): schema_schema = None except SchemaTypeParameterError, error: error.log( error_log ) schema_schema = None if schema_schema is None or error_log.hasNewErrorsSinceMemo( error_memo ): raise SchemaTypeParameterError, \ E error_log.makeErrorReport( tokenizer.getNormalizedInput(), 13 ) > SchemaTypeParameterError [E:\prg\vc\Lib\jsoncpp.roadmap\trunk\jsoncpp\sbuilder\src\sbuilder\sbuildfileformat.py:2269] _______________________________________________________________________________ ============= tests finished: 1 passed, 1 failed in 0.09 seconds ============== Regards, Baptiste.