Q31-Q35(Kafka)

Q26-Q30(Kafka)

Q26. Offset is set by us or does it set by kafka?

Q27. What are the benefits of using Kafka?

Q28. Difference between Kafka and Web API?

==============================================================================

Q26. Offset is set by us or does it set by kafka?

Answer:

Kafka stores offset commits.

==============================================================================

Q27. What are the benefits of using Kafka?

Answer:

Kafka is a distributed public subscribe messaging service. 

Distributed - 

Highly scalable

Low latency

Enormous messaging streaming

Durability - , durability refers to the persistence of data/messages on disk. Also, messages replication is one of the reasons behind durability, hence messages are never lost.

Real-Time Handling

Kafka can handle real-time data pipeline. Since we need to find a technology piece to handle real-time messages from applications, it is one of the core reasons for Kafka as our choice.

Kafka doesn’t try to track which message was actually read by what consumer and just hold on to unread messages. Instead, it holds all of the messages for a pre-specified amount of time, and consumers are charged with tracking their location within each log

1. Kafka is highly scalable. Kafka is a distributed system, which is able to be scaled quickly and easily without incurring any downtime. Apache Kafka is able to handle many terabytes of data without incurring much at all in the way of overhead.
2. Kafka is highly durable. Kafka persists the messages on the disks, which provides intra-cluster replication. This makes for a highly durable messaging system.
3. Kafka is Highly Reliable. Kafka replicates data and is able to support multiple subscribers. Additionally, it automatically balances consumers in the event of failure. That means that it’s more reliable than similar messaging services available.
4. Kafka Offers High Performance. Kafka delivers high throughput for both publishing and subscribing, utilizing disk structures that are capable of offering constant levels of performance, even when dealing with many terabytes of stored messages.

https://data-flair.training/blogs/advantages-and-disadvantages-of-kafka/

==============================================================================

Q28. Difference between Kafka and Web API?

Answer:

WebApi:

1. Request/ Response model.

Kafka:

1. Publish/ Subscribe model. 

https://stackoverflow.com/questions/57852689/kafka-msg-vs-rest-calls

https://medium.com/capital-one-tech/choosing-between-rest-web-apis-and-message-streaming-8e2f4813a058

==============================================================================

==============================================================================

Q21-Q25(Kafka)

Q21. Give one limitation of working with Kafka?

Q22. Multiple consumers can read from same Topic in Kafka?

Q23. I have to ensure once I post the data in kafka, the consumers must its own set of messages. lets say we have 10 consumers and all are subscribing to same topic? 

Q24. What is partition in Kafka?

Q25. In kafka once you have read the message how you make sure that this message is not read again?

=============================================================================

Q21. Give one limitation of working with Kafka?

Answer:

Like any technology, Kafka has its limitations - one of them is the maximum package size of 1 MB. This is only a default setting, but should not be changed easily.

=============================================================================

Q22. Multiple consumers can read from same Topic in Kafka?

Answer:

Yes, multiple consumer can read from same topic also multiple publisher can publish on same topic. 

=============================================================================
Q23. I have to ensure once I post the data in kafka, the consumers must its own set of messages. lets say we have 10 consumers and all are subscribing to same topic? 

Answer:

Topics can be further divided into partitions. each consumer must be associated wtih different partition. 

=============================================================================
Q24. What is partition in Kafka?

Answer:

A topic might have multiple partition. Partitions can be used to handle arbitrary amount of data. 
=============================================================================

Q25. In kafka once you have read the message how you make sure that this message is not read again?

Answer:

Offset