generator及在vgod的TurboGears comet decorator..

时间：2007-04-14 来源：chenhouwu

一. generator与yield语句

generator function的定义是通过yield 语句（yield_stmt ::="yield" expression_list）来实现的。

Python Reference Manual 6.8节，有如下的话：

The yield statement is only used when defining a generator function, and is only used in the body of the generator function. Using a yield statement in a function definition is sufficient to cause that definition to create a generator function instead of a normal function.

When a generator function is called, it returns an iterator known as a generator iterator, or more commonly, a generator. The body of the generator function is executed by calling the generator's next() method repeatedly until it raises an exception.

When a yield statement is executed, the state of the generator is frozen and the value of expression_list is returned to next()'s caller. By ``frozen'' we mean that all local state is retained, including the current bindings of local variables, the instruction pointer, and the internal evaluation stack: enough information is saved so that the next time next() is invoked, the function can proceed exactly as if the yield statement were just another external call.

The yield statement is not allowed in the try clause of a try ... finally construct. The difficulty is that there's no guarantee the generator will ever be resumed, hence no guarantee that the finally block will ever get executed.

注意蓝色的句子。

另外：generator类型是指generator function调用的返回结果，generator function仍然是function类型的。见以下代码：

>>> def f():
... yield 'df'
...
>>> f
<function f at 0xb7d351ec>
>>> isinstance(f,types.GeneratorType)
False
>>> type(f) is types.GeneratorType
False
>>> type(f()) is types.GeneratorType
True
>>> f()
<generator object at 0xb7d3942c>

二. vgod的TurboGears comet decorator代码分析
vgod写的comet很短，comet.py内容如下

import cherrypy
import time

class comet:
   def __init__(self, content_type):
      self.type = content_type
      self.delim = 'NextPart-' + str(time.time())
      cherrypy.response.headerMap['Content-Type'] = \
         'multipart/x-mixed-replace;boundary="%s"' % (self.delim)
   def __call__(self, func):
      def wrapper():
         for part in func():
            yield ("--%(delim)s\r\n" + \
                  "Content-type: %(type)s\r\n\r\n" + \
                  "%(part)s" + \
                  "--%(delim)s\r\n") % \
                  {'delim':self.delim, 'part':str(part), 'type':self.type}
      return wrapper

在controller中，按如下使用：

@expose()
def comet_wait(self):

      @comet(content_type='text/plain')
      def _waiting():
         yield "waiting..."
         time.sleep(5) # replace me with real code
         yield "start"
      return _waiting()
cherrypy.config.update({'/comet_wait': {'stream_response': True}})

而通过读TurboGears的controller源代码知，在 expose 函数中，如果是被修饰的函数返回的是list类型的对象，则不进行处理，如果是generator 或者basestring（basestring其实是一个tuple（str,unicode））和dict类型的，则进入下一步处理函数，在此函数中只对dict 类型的数据调用expose decorator中指定的模板进行处理。在expose的调用方，依次输出list和generator的每个元素的字串表达。

cherry response的内容有两种方式：flatten content和stream content，见Return versus Yield
（http://docs.cherrypy.org/return-versus-yield）。stream方式每次发送一个generator.next()的内容，而flatten方式则把所有的generator.next()的内容都拼接在一起，然后才输出。在大多数情况下，都应该用flatten content, 但此处实现server push的精要就是用stream 方式。

上面的代码的最后一行就是指定对此URL的访问采用stream 方式的response，从cherry的测试代码
（http://www.cherrypy.org/browser/trunk/cherrypy/test/test_core.py?rev=1271）
可知，用@cherrypy.config.wrap(stream_response=True)去修饰comet_wait函数更好，无需考虑url traversal。不过这个decorator是cherrypy 3.0中的新特性，在目前TurboGears 中的cherrypy2.2.1中无法使用。