*Note: this things are experimental and are being implemented on the
`io-improvements`_ branch*

.. _`io-improvements`: http://codespeak.net/svn/pypy/branch/io-improvements

=============
GC operations
=============

This document tries to gather gc-related issues which are very recent
or in-developement. Also, it tries to document needed gc refactorings
and expected performance of certain gc-related operations.

Problem area
============

Since some of our gcs are moving, we at some point decided to simplify
the issue of having care of it by always copying the contents of
data that goes to C level. This yields a performance penalty, also
because some gcs does not move data around anyway.

So we decided to introduce new operations which will simplify issues
regarding this.

Pure gc operations
==================

(All available from rlib.rgc)

* can_move(p) - returns a flag telling whether pointer p will move.
  useful for example when you want to know whether memcopy is safe.

* malloc_nonmovable(TP, n=None) - tries to allocate non-moving object.
  if it succeeds, it return an object, otherwise (for whatever reasons)
  returns null pointer. Does not raise! (never)

Usage patterns
==============

Usually those functions are used via helpers located in rffi. For things like
os.write - first get_nonmovingbuffer(data) that will give you a pointer
suitable of passing to C and finally free_nonmovingbuffer.

For os.read like usage - you first call alloc_buffer (that will allocate a
buffer of desired size passable to C) and afterwards create str_from_buffer,
finally calling keep_buffer_alive_until_here.

String builder
==============

In Python strings are immutable by design. In RPython this still yields true,
but since we cooperate with lower (C/POSIX) level, which has no notion of
strings, we use buffers. Typical use case is to use list of characters l and
than ''.join(l) in order to get string. This requires a lot of unnecessary
copying, which yields performance penalty for such operations as string
formatting. Hence the idea of string builder. String builder would be an
object to which you can append strings or characters and afterwards build it
to a string. Ideally, this set of operations would not contain any copying
whatsoever.

Low level gc operations for string builder
------------------------------------------

* alloc_buffer(T, size) - allocates Array(nolength=True) with possibility
  of later becoming of shape T

* realloc_buffer(buf, newsize) - tries to shrink or enlarge buffer buf. Returns
  new pointer (since it might involve copying)

* build_buffer(T, buf) - creates a type T (previously passed to alloc_buffer)
  from buffer.

Depending on a gc, those might be implemented dumb (realloc always copies)
or using C-level realloc. Might be implemented also in whatever clever way
comes to mind.