Author: Keith Bourgoin, Emmett Butler

pykafka.partitioners.random_partitioner(partitions, key)

Returns a random partition out of all of the available partitions.

class pykafka.partitioners.BasePartitioner

Bases: object

Base class for custom class-based partitioners.

A partitioner is used by the pykafka.producer.Producer to decide which partition to which to produce messages.


list of weak references to the object (if defined)

class pykafka.partitioners.HashingPartitioner(hash_func=None)

Bases: pykafka.partitioners.BasePartitioner

Returns a (relatively) consistent partition out of all available partitions based on the key.

Messages that are published with the same keys are not guaranteed to end up on the same broker if the number of brokers changes (due to the addition or removal of a broker, planned or unplanned) or if the number of topics per partition changes. This is also unreliable when not all brokers are aware of a topic, since the number of available partitions will be in flux until all brokers have accepted a write to that topic and have declared how many partitions that they are actually serving.

__call__(partitions, key)
  • partitions (sequence of pykafka.base.BasePartition) – The partitions from which to choose
  • key (Any hashable type if using the default hash() implementation, any valid value for your custom hash function) – Key used for routing

A partition

Return type:


Parameters:hash_func (function) – hash function (defaults to hash()), should return an int. If hash randomization (Python 2.7) is enabled, a custom hashing function should be defined that is consistent between interpreter restarts.