pykafka.handlers¶
Author: Keith Bourgoin, Emmett Butler
-
class
pykafka.handlers.
ResponseFuture
(handler)¶ Bases:
object
A response which may have a value at some point.
-
__init__
(handler)¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
get
(response_cls=None, timeout=None, **response_kwargs)¶ Block until data is ready and return.
Raises an exception if there was an error.
-
set_error
(error)¶ Set error and trigger get method.
-
set_response
(response)¶ Set response data and trigger get method.
-
-
class
pykafka.handlers.
Handler
¶ Bases:
object
Base class for Handler classes
-
__weakref__
¶ list of weak references to the object (if defined)
-
spawn
(target, *args, **kwargs)¶ Create the worker that will process the work to be handled
-
-
class
pykafka.handlers.
ThreadingHandler
¶ Bases:
pykafka.handlers.Handler
A handler that uses a
threading.Thread
to perform its work-
Event
(**kwargs)¶ A factory function that returns a new event.
Events manage a flag that can be set to true with the set() method and reset to false with the clear() method. The wait() method blocks until the flag is true.
-
GaiError
¶ alias of
socket.gaierror
-
Lock
()¶ allocate_lock() -> lock object (allocate() is an obsolete synonym)
Create a new lock object. See help(LockType) for information about locks.
-
class
Queue
(maxsize=0)¶ Create a queue object with a given maximum size.
If maxsize is <= 0, the queue size is infinite.
-
empty
()¶ Return True if the queue is empty, False otherwise (not reliable!).
-
full
()¶ Return True if the queue is full, False otherwise (not reliable!).
-
get
(block=True, timeout=None)¶ Remove and return an item from the queue.
If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until an item is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Empty exception if no item was available within that time. Otherwise (‘block’ is false), return an item if one is immediately available, else raise the Empty exception (‘timeout’ is ignored in that case).
-
get_nowait
()¶ Remove and return an item from the queue without blocking.
Only get an item if one is immediately available. Otherwise raise the Empty exception.
-
join
()¶ Blocks until all items in the Queue have been gotten and processed.
The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate the item was retrieved and all work on it is complete.
When the count of unfinished tasks drops to zero, join() unblocks.
-
put
(item, block=True, timeout=None)¶ Put an item into the queue.
If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until a free slot is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Full exception if no free slot was available within that time. Otherwise (‘block’ is false), put an item on the queue if a free slot is immediately available, else raise the Full exception (‘timeout’ is ignored in that case).
-
put_nowait
(item)¶ Put an item into the queue without blocking.
Only enqueue the item if a free slot is immediately available. Otherwise raise the Full exception.
-
qsize
()¶ Return the approximate size of the queue (not reliable!).
-
task_done
()¶ Indicate that a formerly enqueued task is complete.
Used by Queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.
If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).
Raises a ValueError if called more times than there were items placed in the queue.
-
-
class
Semaphore
(value=1)¶ Bases:
object
This class implements semaphore objects.
Semaphores manage a counter representing the number of release() calls minus the number of acquire() calls, plus an initial value. The acquire() method blocks if necessary until it can return without making the counter negative. If not given, value defaults to 1.
Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 Python Software Foundation. All rights reserved.
-
__enter__
(blocking=True, timeout=None)¶ Acquire a semaphore, decrementing the internal counter by one.
When invoked without arguments: if the internal counter is larger than zero on entry, decrement it by one and return immediately. If it is zero on entry, block, waiting until some other thread has called release() to make it larger than zero. This is done with proper interlocking so that if multiple acquire() calls are blocked, release() will wake exactly one of them up. The implementation may pick one at random, so the order in which blocked threads are awakened should not be relied on. There is no return value in this case.
When invoked with blocking set to true, do the same thing as when called without arguments, and return true.
When invoked with blocking set to false, do not block. If a call without an argument would block, return false immediately; otherwise, do the same thing as when called without arguments, and return true.
When invoked with a timeout other than None, it will block for at most timeout seconds. If acquire does not complete successfully in that interval, return false. Return true otherwise.
-
__init__
(value=1)¶ x.__init__(…) initializes x; see help(type(x)) for signature
-
__weakref__
¶ list of weak references to the object (if defined)
-
acquire
(blocking=True, timeout=None)¶ Acquire a semaphore, decrementing the internal counter by one.
When invoked without arguments: if the internal counter is larger than zero on entry, decrement it by one and return immediately. If it is zero on entry, block, waiting until some other thread has called release() to make it larger than zero. This is done with proper interlocking so that if multiple acquire() calls are blocked, release() will wake exactly one of them up. The implementation may pick one at random, so the order in which blocked threads are awakened should not be relied on. There is no return value in this case.
When invoked with blocking set to true, do the same thing as when called without arguments, and return true.
When invoked with blocking set to false, do not block. If a call without an argument would block, return false immediately; otherwise, do the same thing as when called without arguments, and return true.
When invoked with a timeout other than None, it will block for at most timeout seconds. If acquire does not complete successfully in that interval, return false. Return true otherwise.
-
release
()¶ Release a semaphore, incrementing the internal counter by one.
When the counter is zero on entry and another thread is waiting for it to become larger than zero again, wake up that thread.
-
-
SockErr
¶ alias of
socket.error
-
Socket
= <module 'socket' from '/home/docs/.pyenv/versions/2.7.16/lib/python2.7/socket.pyc'>¶
-
spawn
(target, *args, **kwargs)¶ Create the worker that will process the work to be handled
-
-
class
pykafka.handlers.
RequestHandler
(handler, connection)¶ Bases:
object
Uses a Handler instance to dispatch requests.
Bases:
tuple
Return self as a plain tuple. Used by copy and pickle.
Exclude the OrderedDict from pickling
Create new instance of Shared(connection, requests, ending)
Return a nicely formatted representation string
Return a new OrderedDict which maps field names to their values
Make a new Shared object from a sequence or iterable
Return a new Shared object replacing specified fields with new values
Alias for field number 0
Alias for field number 2
Alias for field number 1
-
class
Task
(request, future)¶ Bases:
tuple
-
__getnewargs__
()¶ Return self as a plain tuple. Used by copy and pickle.
-
__getstate__
()¶ Exclude the OrderedDict from pickling
-
static
__new__
(_cls, request, future)¶ Create new instance of Task(request, future)
-
__repr__
()¶ Return a nicely formatted representation string
-
_asdict
()¶ Return a new OrderedDict which maps field names to their values
-
classmethod
_make
(iterable, new=<built-in method __new__ of type object>, len=<built-in function len>)¶ Make a new Task object from a sequence or iterable
-
_replace
(**kwds)¶ Return a new Task object replacing specified fields with new values
-
future
¶ Alias for field number 1
-
request
¶ Alias for field number 0
-
-
__init__
(handler, connection)¶
-
__weakref__
¶ list of weak references to the object (if defined)
-
_start_thread
()¶ Run the request processor
-
request
(request, has_response=True)¶ Construct a new request
Parameters: has_response – Whether this request will return a response Returns: pykafka.handlers.ResponseFuture
-
start
()¶ Start the request processor.
-
stop
()¶ Stop the request processor.