[py-dev] py.log -- annotating logs

[Ian Bicking]

I'm guessing you meant to copy the list...

Grig Gheorghiu wrote:
> Maybe the easiest way would be to just define a consumer per
> "application domain", i.e. for your Web app you'd have a
> web_app_consumer() function that would know how to print all the info
> you need. 
> 
> The way it works right now is that you need to associate a consumer
> with one or more keywords. Then you have the flexibility of changing
> consumers for those keywords dynamically via configuration files etc.
> 
> So I'd do something like this in a configuration file:
> 
> log = py.log.Producer("mywebapp")
> def my_consumer(message):
>     remote_ip = get_current_remote_ip()
>     crt_time = get_current_time()
>     message.args = [crt_time, remote_ip] + message.args
>     print str(message) # prints "[mywebapp] crt_time remote_ip
> <rest_of_message>"

This particular example doesn't work well with other consumers.  I'd 
want to change the message and then let another consumer deal with that 
message.

Unfortunately this kind of composition through dispatching is very 
configuration-unfriendly -- that's something we're hitting with WSGI 
middleware configuration right now.  But what makes it unfriendly is 
also what makes it very flexible.

The wrapping consumer I gave before could, potentially, be inserted in a 
number of places.  It could be put in place globally, or for a single 
keyword, or whatever.  This presumes, of course, that it's possible to 
find consumers without running them, so a new consumer can be inserted 
instead of just overwriting the old one.

This can be challenging in itself.  There might be one root consumer. 
Let's say by default it's the consumer that dispatches based on keyword. 
  If we try to insert another consumer in the root consumer, and the 
dispatching consumer has been replaced, the replacement doesn't 
necessarily have the same API.  Should that replacement consumer send 
all unknown attributes to its sub-consumer?  But the keyword dispatching 
consumer doesn't have a single sub-consumer to dispatch to, so that 
doesn't work there.  And other dispatchers are also possible -- for 
instance, this application supports multiple clients, and if a client is 
associated with a request I'd like to put the log file in a location 
specific to that client.  But that's more programmatic than keywords, 
and I can create the sub-consumers dynamically.  Or maybe some component 
is generating annoying log messages, and I want to filter them out based 
on some really arbitrary expression.  There's lots of possibilities. 
But the setup can become hard.

But then, it's clear that the logging module doesn't have easy setup 
either.

> py.log.set_consumer("mywebapp", my_consumer)
> 
> Then you would just use log(message) in your code.
> 
> Does this help or maybe I'm way off the mark?


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org