Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-18452: Implemented batch size in acquired records #18459

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from

Conversation

apoorvmittal10
Copy link
Collaborator

The PR implements a way to divide acquired batches into batch size as desirable by client.

The BatchSize is the soft limit and should allign the batches in response and cached state in broker at the log batch boundaries i.e.

say produce log batch is from 0-50, 51-100

for batch size from client - 60: then response will be 0-100
for batch size from client - 40: then response will be 0-50, 51-100

This means that broker will not break the batches rather align them in response as per desirable batch size from client so complete batches are acknowledged which shall avoid offset based tracking on broker.

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@github-actions github-actions bot added triage PRs from the community core Kafka Broker KIP-932 Queues for Kafka labels Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Kafka Broker KIP-932 Queues for Kafka triage PRs from the community
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant