pykafka.topic

Author: Keith Bourgoin, Emmett Butler

class pykafka.topic.Topic(cluster, topic_metadata)

A Topic is an abstraction over the kafka concept of a topic. It contains a dictionary of partitions that comprise it.

__init__(cluster, topic_metadata)

Create the Topic from metadata.

Parameters:
  • cluster (pykafka.cluster.Cluster) – The Cluster to use
  • topic_metadata (pykafka.protocol.TopicMetadata) – Metadata for all topics.
earliest_available_offsets()

Get the earliest offset for each partition of this topic.

fetch_offset_limits(offsets_before, max_offsets=1)

Get earliest or latest offset.

Use the Offset API to find a limit of valid offsets for each partition in this topic.

Parameters:
  • offsets_before (int) – Return an offset from before this timestamp (in milliseconds)
  • max_offsets (int) – The maximum number of offsets to return
get_balanced_consumer(consumer_group, **kwargs)

Return a BalancedConsumer of this topic

Parameters:consumer_group (str) – The name of the consumer group to join
get_producer(**kwargs)

Create a pykafka.producer.Producer for this topic.

For a description of all available kwargs, see the Producer docstring.

get_simple_consumer(consumer_group=None, **kwargs)

Return a SimpleConsumer of this topic

Parameters:consumer_group (str) – The name of the consumer group to join
latest_available_offsets()

Get the latest offset for each partition of this topic.

name

The name of this topic

partitions

A dictionary containing all known partitions for this topic

update(metadata)

Update the Partitions with metadata about the cluster.

Parameters:metadata (pykafka.protocol.TopicMetadata) – Metadata for all topics