.. include:: crossreferences.asc Potential impact --------------------------------------------------------------------------- The successful execution of the PyPy project will deliver a highly productive and flexible implementation of an Object-Oriented, Open Source Very-High-Level programming language (VHLL). This will impact software development in several important ways. - The cost of software development will diminish. - The time to market will be reduced. - The cost of software maintenance will be reduced. - The barriers to marketing a product will be lowered. The development methods of the PyPy project will prove that using Sprints, pair programming and test-driven development results in : - Broader understanding of the code base among developers - Rapid developments from ideas to working code - Sustainable project progress through unit testing The cost of software development is essentially labor costs, time to market, and costs of software tools. Having a (free) flexible VHLL means problems are solved closer to the abstraction level at which they are formulated, thereby improving the productivity of the individual programmer. Higher productivity, obviously, will decrease labor costs and time to market. However, the greater impact we expect to have is to produce a version of the language which, being superior to anything which has gone before, will be extremely widely used. To that end, our first target group is the existing community of Python/Jython programmers. See the section on Exploitation where this is discussed at length. Contributions to Standards +++++++++++++++++++++++++++ There are currently two implementations of Python in common use. The first one, which we will call CPython, (what the world knows as Python), is a C implementation of the Python Programming language which compiles to its own virtual machine. The second one is Jython, a pure-Python implementation which compiles to the Java virtual machine. There is no ANSI standard (or similar standard) for Python. Right now the de-facto standard for the programming language is 'whatever CPython does'. This is far from ideal, and questions arise, especially from the developers of Jython as to which CPython language behaviors are essential to the Python language itself, and which are mere accidents of this particular implementation. For example, the garbage-collection behavior of CPython is implemented by reference-counting, which ensures that an object is finalized as soon as the last reference to it goes away. That would be extremely inconvenient (close to impossible) to implement on standard Java Virtual Machines, which have a deliberately under-specified garbage collector (it can collect anything it pleases whenever it pleases...). In this case, the Jython designers had to obtain an explicit ruling from Guido van Rossum, Python's designer -- who ruled that the behavior of CPython was 'accidental' in this case, and not intrinsic to the Python language specification. Guido van Rossum has expressed interest in giving PyPy the status of 'implementation standard' (executable specification) of the Python programming language. PyPy's Object Space flexibility will be crucial in distinguishing "accidental" from "designed-in" characteristics. Here is the relevant mail from Guido van Rossum:: Having participated in one PyPy Sprint, I am very happy with this project, and hope to see it going forward. The PyPy team includes some of the best minds in the Python community. Unlike Perl or Tcl, Python is not a "one-implementation" language, and consequently the formal language specification should have priority over the behavior of a particular implementation. The PyPy project will reinforce this idea, and can be useful in sorting out ambiguities in the specification. It is even possible that PyPy will eventually serve as an "executable specification" for the language, and the behavior of one of PyPy's object spaces will be deemed the correct one. --Guido van Rossum (home page: http://www.python.org/~guido/) In order to do this we will have to do something which is called 'Submitting a PEP'. A PEP - Python Enhancement Proposal - is a design document providing information to the Python community, or describing a new feature for Python. The PEP process is designed for Community involvement and participation. There are two kinds of PEPs. A Standards Track PEP describes a new feature or implementation for Python. An Informational PEP describes a Python design issue, or provides general guidelines or information to the Python community, but does not propose a new feature. If we proposed to make PyPy the reference standard of the Python language, we would, obviously, have to submit a Standards Track PEP. The complete details of how to write a PEP are themselves an Informational PEP -- PEP #1 [PEP1]_ in fact. After circulating through the community, PEPs are reviewed by Guido van Rossum, the language author, and his chosen consultants, who may accept or reject a PEP or send it back to the author(s) for revision. Thus the expression of interest from Guido van Rossum is of extreme significance. It is very likely, if all the goals described in the project are completed, that PyPy will become the standard reference implementation of the Python language. In other words, our problems are technical, and not political. The political will is already there to make PyPy the reference language. We merely need to create it. Strategic impact +++++++++++++++++ PyPy will have a significant strategic impact throughout the IT sector. It produces immensely useful, practical results which shall be immediately exploited by Python developers world-wide. While it addresses issues which have hitherto mostly remained the special province of academia, and significantly enhances the State-of-the-Art, it is not a project that will only satisfy intellectual curiosity. We intend to make a new reference version of the Python Programming Language, which is faster, more flexible, more extensible, and which gives more control to the individual programmer as to how it is deployed. For instance, you will be able to build a Python interpreter customized for a very small amount of available memory. Or one with speed enhancements only possible because there is a huge amount of memory available. Or a PyPy interpreter which will be executed on several machines but offer a single distributed computation space and balances the load by moving execution threads around. We can produce Object Spaces which implement Logic Programming, Aspect Oriented Programming and Design By Contract, hot new topics in computer science research which are rarely seen in industry because existing popular languages do not support them. An implementation of the language with substantial improvements will have an immediate direct effect on European competitiveness. Moreover, the planned improvements directly target the handheld, mobile, and embedded device sectors, where Europe is the acknowledged world leader. People working in such industries have long desired a high level language with a very small footprint. The new innovative concept of Object Spaces, pioneered by PyPy makes possible the construction of tiny Object Spaces, suitable for running on the smallest devices. Indeed, it will be possible for application programmers to configure Python runtime environments to suit their particular hardware characteristics -- for instance, a version that runs in a minimal amount of memory, or a version that can exploit a huge amount of memory to achieve the highest performance speed. Python with greater speed will seamlessly improve the offerings of those European Companies who already develop using Python. Moreover, some companies resist using Python because of speed concerns. If execution speed, rather than development speed is of paramount importance, then Python is currently not a very good language choice. A faster Python would thus have more appeal. Strategic Impact of Open Source ================================ Open Source has now reached an installed application base sufficient to become widely recognized as a viable business standard, especially in Europe. This has no doubt been aided by the proprietary alternatives being intellectual properties of large USA based companies and the growing awareness of the benefits that Open Source can provide. The European Union, comprising heterogeneous distinct regions with different availability of economic resources, is well positioned to take advantage of the open source momentum. One reason that Open Source is becoming so appealing is due to its equal suitability for projects based on diverse capital budgets. Python is an Open Source, easy to learn, easy to use, programming language that is readily accessible to a broad user base, estimated at 175,000 programmers world-wide, ranging from education establishments through government to commercial enterprises of all sizes. This project aims at building upon the inherent strengths of Python to ensure its longevity in the commercial and research marketplace. This will maximize the return of the existing and future capital investment in this technology and ensure Python's widespread acceptance as one of the most cost-effective technology platforms available. Potential Impact on the Balance of Trade ======================================== One of the greatest threats to European competitiveness is its dependence upon proprietary closed source software, mostly made in the United States. This is not only an issue of money being spent in the United States is money that is not being spent here, although that affects matters as well. There are two more serious risks. The first is a threat in the present. Any company which writes its software in a proprietary, closed source language is dependent upon its software provider. If you have a bug, you must wait for them to fix it. If this bug is not a high priority for them, you can wait a long time. If you have access to the source you always have the option of fixing it yourself, or hiring somebody else to do that. But this is not the greatest of your worries. You are at constant risk of having your software provider discontinue support for your platform. This is a real threat, not a theoretical one. In 2002, Microsoft announced that it would no longer be supporting Visual Basic 6.0 after the year 2005. All Visual Basic Developers have been told to convert their code to run under Microsoft's new .NET framework. Before that, in 2001, Microsoft suddenly stopped supporting its Visual J++ language platform, meant to be a direct competitor for Java, after settling a lawsuit with Sun Microsystems. No migration path was specified. Microsoft is making these decisions because they make business sense for Microsoft, regardless of the effects on European software developers. Right now Python is the sixth most popular programming language in the world. Java and Visual Basic (VB), ranked 1 and 2, are closed source proprietary American products. The number one reason that is cited by Java users as to 'why they don't use Python' is that it is too slow. PyPy will fix this. The Visual Basic programmers are in a more interesting position. Microsoft, in its wisdom, has decided to end support of their current platform, Visual Basic 6.0. After the year 2005, these programmers will have to move to Microsoft's .NET. They're understandably unhappy, and tempted to move, not to .NET, but to an Open Source language, just so that they can have control over their own destiny and indicate their displeasure with Microsoft at the same time. This is a tremendous opportunity for us. Every Java and VB user that switches to Python does not pay license fees to Sun or Microsoft and helps the European balance of Trade. Of course, all the PBF members are predicting that an improved Python will improve their sales, both domestic and foreign. The dominance of the PBF by European companies means this can only improve Europe's trade balance. European Dimension =================== PyPy is an extremely high-profile project, as is only right since we intend to utilize the estimated 175,000 Python users as our user-base. Consequently, it is essential that we have the full support of the International Python community. Our success in that regard was more fully spoken about in B3.1, International Standards, but suffice to say that the only possible dimensions for this project is 'Europe wide' or 'World Wide'. We could not find the expertise needed in a single country, and nor could we attain international acceptance without involving the top members of the Python community. Working at a smaller (single-country) scale would only 'fork the project' -- make a local version which is of limited use, while the main line project continued to develop in another direction. At best we would have a ghetto. At worst, we would have two hostile camps slinging insults at each other over which 'was the real Python'. We have avoided all of these problems by carefully making the PyPy project sufficiently international, by discussing this proposal for nearly a year at international conferences, and by inviting non-European Python luminaries to our week-long intensive code-writing meetings called 'Sprints'. By producing a prototype in four coding Sprints (spread out over half a year) held in locations in Hildesheim, Germany; Gothenburg, Sweden; Louvain-La-Neuve, Belgium; and Berlin, Germany, while discussing the project on mailing lists and on our website, we have made certain that we have the proper dimension for our project to succeed, and become the new reference language for Python. Innovation Related Activities ++++++++++++++++++++++++++++++ This is an Open Source Project. Thus complicated issues involving intellectual property simply do not arise. The knowledge produced by this project, every deliverable, is listed at Dissemination Level Public. We are utterly committed to transparency and open dissemination, and as such will have all of our code available for nightly downloads, our papers freely available on the PyPy websites, and freely available for other people to link to. All Consortium members will have signed a consortium agreement, asserting that all code will be released under the MIT Open Source License, approved by both the Open Source Initiative and the Free Software Foundation. The MIT license is here. :: The MIT License Copyright (c) Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. This license is concise and clearly permits any person obtaining our product to use it without limitation. The open, clear language removes any chance of licensing controversy. Management of this intellectual property consists of a posting the license on the website or wherever source code is available, and periodically running a program, especially before software releases, attaching a copy of the license at the top of every file released. The only other intellectual property which we will produce are scientific papers, talks, and the like. They will all be freely available. Copyright shall rest in the authors, unless somebody gives a paper to one of the scientific journals that keeps all copyright to itself. In either case, no management is necessary. The current version of Python is licensed under the Python Software License. Our project will in no way conflict with this license. This is authoritative. Tim Peters, a member of the PyPy project, is also director of the Python Software Foundation, and responsible for the Intellectual Property of the existing Python language. Dissemination ++++++++++++++ To successfully disseminate knowledge from the PyPy project, the project needs to have good steadfast routines for documenting and interacting with the project stakeholders. The project management team (project manager and assisting project manager) will be responsible for overseeing dissemination tasks and activities. Dissemination will consist of, but not be limited to * The key activities in the project and the development process are the "Sprints". These are open forums to which we actively invite members from both commercial and research oriented organizations to actively participate * since PyPy as a project has goal of a high transparency (see B5), documentation and information as well as project member contact information will be easy to locate for external interested potential stakeholders * the PyPy project will gather on a regular basis, during Sprints, knowledge about the the development process and results. Every six months, the project will go through a project review workshop in which specific emphasis will be put on knowledge gathering and dissemination strategies * a PyPy newsletter will be produced for external organizations/commercial enterprises that will be sent out after each Sprint, keeping them updated on project process and development progress. The assistant project manager will be responsible for this. * to ensure that interested external parties will be able to use knowledge acquired in the PyPy project we will host two workshops during the project, one after 6 months and the first review workshop and one at the end of the project in which we actively work with interested parties to analyze potential usage of PyPy process and prototype We will also partake in the following official events, forum, conferences to spread information about the ongoing project, its unique process and technological impact: EuroPython/EuroZope (European Python Conference), ACM/IFIP/USENIX International Middleware Conference, OSCON (Open Source Convention), OOPSLA (Object-Oriented programming, systems, languages and applications) PyCon (Python Developers conference), FOSDEM (Free and Open Source Developer European Meeting), ECOOP (European Conference for Object Oriented Programming) More conferences will be added once the project gets started. Exploitation +++++++++++++ Our first goal in exploitation will be to make PyPy the reference Python language. Our market, then is all existing Python programmers. _Just how many are we?_ It is always difficult to measure how many people are using a programming language. Python is generally ranked the sixth most popular computer language in the world. Only Java, Visual Basic, C, C++, and Perl are believed to have more users. The Python FAQ, available at the Python language home site of python.org says:: 2.1. How many people are using Python? Certainly thousands, and quite probably tens of thousands of users. More are seeing the light each day. The comp.lang.python newsgroup is very active, but overall there is no accurate estimate of the number of subscribers or Python users. Jacek Artymiak has created a Python Users Counter; you can see the current count by visiting http://www.wszechnica.safenet.pl/cgi-bin/checkpythonuserscounter.py Jacek's counter has more than 43 000 registered users. Googling for 'python programming' gives 3,020,000 hits. ('Python' gives ten million, but many of those Pythons are probably actual reptiles.) Compared to the most common languages these days, C, C++, Java and Visual Basic, there is certainly less 'market penetration', and in some niches, languages such as PHP, Perl and SQL are popular, but Python is vastly bigger than the more obscure languages such as Haskell, OCaml, Smalltalk, ADA, Ruby etc. Some statistics from python.org may be relevant. There were 47,751 Python 2.3 downloads in the first 10 days of September. These are the 'bleeding edge' developers, who were interested in the new release of Python first made available at the end of August. At the other end of the spectrum, there are many more who wait for Python to become available as a Red Hat, or Debian package, or simply use the version of Python that came installed with their machine when they bought it. For what it's worth, there are about 600,000 web visitors a month to python.org. About one third of them are using the documentation, which is a fairly good indicator that they are trying to use the language. Number of unique IP addresses per month is about 350,000 at this point, and has been rising steadily from about 250,000 this time last year. Converting this into an actual estimate of number of users is a difficult, because: 1) One user may use multiple machines (e.g. home & work & cafe) 2) Many sites use a firewall that hides the actual IP (and thus combines multiple users into one) 3) Not all Python users go to python.org every month (or at least not the Documentation page), because they have already downloaded the documentation for local browsing. A conservative estimate would be that there are at least 175,000 active Python users in the world. At least half of them are in Europe, if O'Reilly's sales statistics for their popular books *Learning Python*, *Python in a Nutshell* and *The Python Cookbook* are to be considered relevant. Another piece of data, sales statistics of *The Essential Jython* targeted at users of the Python version that compiles to the Java Virtual Machine indicate that there are at least 10 thousand Jython users worldwide. 175,000. That's a lot of Python users. Greatly improving the language which they all use will have an enormous impact. Since the PyPy development team is in constant contact with the Python community, and its world-wide leadership, there is no friction or political resistance to PyPy. The individual members of the consortium are among the most well-known and well-respected members of the Python community. We have taken special care to include prominent stakeholders in the existing language from the very beginning, and have constantly invited members of the community to a series of development Sprints whereby we produced a prototype which served as a proof-of-concept. We have also budgeted for extensive communication and dissemination with this community. Thus when we deliver a working product, they will be ready to adopt it. Beyond this, there are the users of other languages. There are something like 12 million programmers world-wide and roughly 50% of those use Visual Basic (according to International Data Corp). In March 2002, Borland said Java had about 1.5 million developers. How do we go about getting them to use PyPy? First of all, we must give them something that they find useful, and useful enough to switch. While Python users might be satisfied at getting a faster language, the users of other languages, who will need to make a greater effort in switching need a greater reason to switch. Fortunately, this is the technical goal of the project. The current State-of-the-Art, both in Python and in programming languages in general, does not best serve the new needs of the creators of embedded, networked and distributed software sectors. They need a more flexible language which is easier to reduce to its 'bare-bones' for embedding, and which can dynamically reconfigure itself and optimize execution speed at the interpreter level. Consequently, the PyPy project is a collaboration between academic researchers and SME software and service providers. The former have the skill and vision to produce a new run-time language architecture for the twenty-first century and the latter wish to deploy PyPy in their innovative new business ventures. Furthermore, an important aspect of the current proposal is that we will make use of the new opportunities that the Object Spaces give us to extend the reach of the Python to areas where programmers currently have to resort to other languages. In particular, we will bring the Stackless version of Python into the mainstream. This allows Python to handle thousands of threads inexpensively, making simulation software easy to program. The makers of computer games have already shown interest in this, and have funded its early development. The Stackless Python implementation also supports mobility of running code, allowing it to migrate from one machine to another. Other programming areas addressed by the project by building on Object Spaces are Aspect Oriented Programming, Design by Contract, and Constraint Solving. All these will significantly increase the attractiveness of Python as a programming language, and we will work with people in the Python community who are interested in simulation, software engineering, and the semantic web to ensure that they can take full advantage of the new platform we are offering. Here is a selection of endorsements of the project from stakeholders, members of the Python Business Forum. .. include:: endorsements.asc The results of the PyPy project are also expected to be utilized by established industrial users. We have expressions of interest from *Bang and Olufsen*, the Danish manufacturer of high-end stereo equipment, *Vodaphone* and *Ericsson* the mobile industrialists, *Siemens*, the German conglomerate, and *Axis* the Swedish-based multinational market leader in in-house developed chip technology for network video and print servers. References: .. [PEP1] http://www.python.org/peps/pep-0001.html