class pykafka.cluster.Cluster(hosts, handler, socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, exclude_internal_topics=True)

Bases: object

A Cluster is a high-level abstraction of the collection of brokers and topics that makes up a real kafka cluster.

__init__(hosts, handler, socket_timeout_ms=30000, offsets_channel_socket_timeout_ms=10000, exclude_internal_topics=True)

Create a new Cluster instance.

  • hosts (str) – Comma-separated list of kafka hosts to used to connect.
  • handler (pykafka.handlers.Handler) – The concurrency handler for network requests.
  • socket_timeout_ms (int) – The socket timeout (in milliseconds) for network requests
  • offsets_channel_socket_timeout_ms (int) – The socket timeout (in milliseconds) when reading responses for offset commit and offset fetch requests.
  • exclude_internal_topics (bool) – Whether messages from internal topics (specifically, the offsets topic) should be exposed to consumers.

list of weak references to the object (if defined)


Get fresh cluster metadata from a broker.


Should this topic be excluded from the list shown to the client?


Update brokers with fresh metadata.

Parameters:broker_metadata (Dict of {name: metadata} where metadata is pykafka.protocol.BrokerMetadata and name is str.) – Metadata for all brokers.

Update topics with fresh metadata.

Parameters:metadata (Dict of {name, metadata} where metadata is pykafka.protocol.TopicMetadata and name is str.) – Metadata for all topics.

The dict of known brokers for this cluster


Get the broker designated as the offset manager for this consumer group.

Based on Step 1 at

Parameters:consumer_group (str) – The name of the consumer group for which to find the offset manager.

The concurrency handler for network requests


The dict of known topics for this cluster


Update known brokers and topics.