pykafka.client

Author: Keith Bourgoin, Emmett Butler

class pykafka.client.KafkaClient(hosts='127.0.0.1:9092', socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, ignore_rdkafka=False, exclude_internal_topics=True, source_address='')

Bases: object

A high-level pythonic client for Kafka

__init__(hosts='127.0.0.1:9092', socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, ignore_rdkafka=False, exclude_internal_topics=True, source_address='')

Create a connection to a Kafka cluster.

Documentation for source_address can be found at https://docs.python.org/2/library/socket.html#socket.create_connection

Parameters:
  • hosts (bytes) – Comma-separated list of kafka hosts to used to connect.
  • socket_timeout_ms (int) – The socket timeout (in milliseconds) for network requests
  • offsets_channel_socket_timeout_ms (int) – The socket timeout (in milliseconds) when reading responses for offset commit and offset fetch requests.
  • ignore_rdkafka (bool) – Don’t use rdkafka, even if installed.
  • exclude_internal_topics (bool) – Whether messages from internal topics (specifically, the offsets topic) should be exposed to the consumer.
  • source_address (str ‘host:port’) – The source address for socket connections
__weakref__

list of weak references to the object (if defined)

update_cluster()

Update known brokers and topics.

Updates each Topic and Broker, adding new ones as found, with current metadata from the cluster.