Quantcast
Channel: Topic messages with partitions=1 stored on multiple brokers
Browsing all 9 articles
Browse latest View live

Topic messages with partitions=1 stored on multiple brokers

My current setup has 2 brokers, multiple topics, each topic with 1 partition, kafka 0.7.1 Some topics are logged onto multiple brokers inspite of partitions set to 1. Kafka 0.7.1 does not have...

View Article



Topic messages with partitions=1 stored on multiple brokers

Seems to me you are confusing partitions and brokers. Partition count has nothing to do with the number of brokers to which a message a sent -- just the number of partitions into which that message...

View Article

Topic messages with partitions=1 stored on multiple brokers

Hello Suchi, As Philip said in 0.7 the number of partitions is per broker. As from 0.8 the number of partitions will be global, meaning the total number of partitions across the broker cluster....

View Article

Topic messages with partitions=1 stored on multiple brokers

Yes I understand that. I am letting the producer/consumer use zookeeper to discover brokers. I can clearly see in the logs(brokers) that both the brokers create a new topic log for the same topic. The...

View Article

Topic messages with partitions=1 stored on multiple brokers

Thx for your replies. Can u please clarify this further for me? In 0.7, if the partition count is per broker, shouldn't each broker have a log directory for every topic? I don't see that though... Only...

View Article


Topic messages with partitions=1 stored on multiple brokers

No, it does not matter, since that has no bearing on TCP connectivity. Philip -- Philip O'Toole

View Article

Topic messages with partitions=1 stored on multiple brokers

It may something with your configuration. Try it with a simpler setup of one broker, see that both topics appear on the broker, and build on your understanding and experimentation with that setup....

View Article

Topic messages with partitions=1 stored on multiple brokers

Hi Suchi, In 0.7 a producer would send to one broker within a session, hence you may want to send enough messages to see them shown up on both brokers. Guozhang -- -- Guozhang -- Guozhang Wang

View Article


Topic messages with partitions=1 stored on multiple brokers

This could be related to https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-IamusingtheZKbasedproducerin0.7andIseedataonlyproducedonsomeofthebrokers%2Cbutnotall%2Cwhy%3F Thanks, Jun -- Jun Rao

View Article

Browsing all 9 articles
Browse latest View live




Latest Images