[z3-five] unicodes and strings in Zope 2.9's ZPT with Zope 3's i18n

[Martijn Faassen]

Chris Withers wrote:
> Martijn Faassen wrote:
>> What the right way should be for existing Zope 2 applications I do not 
>> know.
> 
> Yes, the particular problem I see is with persistent objects that have 
> encoded strings as attributes. How did you guys manage the upgrading of 
> existing Silva instances which were created when you still used encoded 
> strings everywhere?

Pain, lots of sustained pain. Lots of upgrade scripts. Luckily lots of 
Silva content is stored in XML which does get encoding right from th 
start. Anyway, we had so much pain that in retrospect I sometimes think 
we could've stuck to UTF-8 and just lived with that, even though it 
wasn't the proper way for Python. When the pain was finally over, we 
started using PlacelessTranslationService and we got new, more subtle 
pain...

We did this relatively early in the life of Silva, which reduced the 
pain somewhat as there were less instances to worry about. First half of 
2003 I think -- we were there early. I suspect it'd be *incredibly* 
painful for Plone to do any transition in its current stage of 
development...

Anyway, we did get some gain from the pain in that we can use Five i18n 
without unicode issues. We also know that text stored in Silva is going 
to be in unicode which helps with the XML export logic and such.

>> For *new* Zope 2 applications, my strong recommendation would be to 
>> store human-readable text as unicode, and if your text is stored 
>> externally (file, rdb), to decode it into unicode as soon as possible 
>> when it enters your application.
> 
> Totally agreed, although I'm not sure Zope 2's publisher helps as much 
> as it should do, and I doubt any of the Zope DA's are focused this way :-S

I haven't had much issue with the publisher - you need to set the 
response header to UTF-8 and it will automatically translate unicode 
output to UTF-8. The DA's are more troubling, though we've been 
exploring relational/object mappers (SQLObject, SQL Alchemy), which have 
mechanisms to deal with this. Zope 3's DA's also take care of unicode at 
the DA level, so we could use those in Zope 2 that might take care of that.

Regards,

Martijn