pykafka.client¶
Author: Keith Bourgoin, Emmett Butler
-
class
pykafka.client.
KafkaClient
(hosts='127.0.0.1:9092', zookeeper_hosts=None, socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, use_greenlets=False, exclude_internal_topics=True, source_address='')¶ Bases:
object
A high-level pythonic client for Kafka
NOTE: KafkaClient holds weak references to Topic instances via
pykafka.cluster.TopicDict
. To perform operations directly on these topics, such as examining their partition lists, client code must hold a strong reference to the topics it cares about. If client code doesn’t need to examine Topic instances directly, no strong references are necessary.-
__init__
(hosts='127.0.0.1:9092', zookeeper_hosts=None, socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, use_greenlets=False, exclude_internal_topics=True, source_address='')¶ Create a connection to a Kafka cluster.
Documentation for source_address can be found at https://docs.python.org/2/library/socket.html#socket.create_connection
Parameters: - hosts (bytes) – Comma-separated list of kafka hosts to which to connect.
- zookeeper_hosts (bytes) – KazooClient-formatted string of ZooKeeper hosts to which to connect. If not None, this argument takes precedence over hosts
- socket_timeout_ms (int) – The socket timeout (in milliseconds) for network requests
- offsets_channel_socket_timeout_ms (int) – The socket timeout (in milliseconds) when reading responses for offset commit and offset fetch requests.
- use_greenlets (bool) – Whether to perform parallel operations on greenlets instead of OS threads
- exclude_internal_topics (bool) – Whether messages from internal topics (specifically, the offsets topic) should be exposed to the consumer.
- source_address (str ‘host:port’) – The source address for socket connections
-
__weakref__
¶ list of weak references to the object (if defined)
-
update_cluster
()¶ Update known brokers and topics.
Updates each Topic and Broker, adding new ones as found, with current metadata from the cluster.
-