Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: AutoMQ/automq
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: 1.3.2-rc0
Choose a base ref
...
head repository: AutoMQ/automq
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: main
Choose a head ref

Commits on Nov 1, 2024

  1. perf: limit the inflight requests (#2100)

    * docs: add todos
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(network): limit the inflight requests by size
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(ReplicaManager): limit the queue size of the `fetchExecutor`s
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(KafkaApis): limit the queue size of async request handlers
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor(network): make "queued.max.requests.size.bytes" configurable
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * style: fix lint
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * fix(network): limit the min queued request size per queue
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 1, 2024
    Copy the full SHA
    a7250cd View commit details

Commits on Nov 3, 2024

  1. fix(issue2108): avoid blocking at the end of a compaction iteration w…

    …hen there are un-uploaded data (#2109)
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 3, 2024
    Copy the full SHA
    412a99b View commit details
  2. fix(s3stream): wait force upload complete before return (#2112)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    Co-authored-by: Shichao Nie <niesc@automq.com>
    superhx and SCNieh authored Nov 3, 2024
    Copy the full SHA
    8338f18 View commit details

Commits on Nov 5, 2024

  1. fix(compaction): prevent double release on compaction shutdown (#2115)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 5, 2024
    Copy the full SHA
    04b53f4 View commit details
  2. fix(s3stream): fix available bandwidth metrics (#2119)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 5, 2024
    Copy the full SHA
    4f93b92 View commit details

Commits on Nov 6, 2024

  1. fix(checkstyle): fix checkstyle (#2122)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 6, 2024
    Copy the full SHA
    0443529 View commit details
  2. fix(e2e): remove unstable autobalancer tests (#2124)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 6, 2024
    Copy the full SHA
    e53c837 View commit details
  3. Copy the full SHA
    08ca9ec View commit details

Commits on Nov 7, 2024

  1. perf(log): avoid too many checkpoint at the same time (#2130)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 7, 2024
    Copy the full SHA
    9c51a61 View commit details
  2. perf(tools/perf): assuming all partitions have the same offset at the…

    … same time (#2127) (#2128)
    
    * feat(tools/perf): log progress on resetting offsets
    
    
    
    * fix: reset timeouts
    
    
    
    * feat: increase the log interval
    
    
    
    * perf(tools/perf): assuming all partitions have the same offset at the same time
    
    
    
    * feat: limit the min of --backlog-duration
    
    
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 7, 2024
    Copy the full SHA
    42debe7 View commit details
  3. refactor(tools/perf): retry sending messages in when waiting topics r…

    …eady (#2133)
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 7, 2024
    Copy the full SHA
    8f5fcc5 View commit details

Commits on Nov 8, 2024

  1. fix(s3stream/storage): correct if condition on awaitTermination (#2138

    )
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 8, 2024
    Copy the full SHA
    73e8901 View commit details
  2. feat(tools/perf): run benchmark without consumer (#2135)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 8, 2024
    Copy the full SHA
    cbaf3a2 View commit details
  3. fix(issue2139): prevent read object info from closed ObjectReader (#2141

    )
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 8, 2024
    Copy the full SHA
    90516c7 View commit details
  4. fix(issue2139): add computeIfAbsent atomic operation to AsyncLRUCache (

    …#2144)
    
    close #2139
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 8, 2024
    Copy the full SHA
    a74cf69 View commit details
  5. fix(issue2140): remove override equals and hashCode method for Object…

    …Reader (#2147)
    
    close #2140
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 8, 2024
    Copy the full SHA
    351e1e0 View commit details

Commits on Nov 13, 2024

  1. fix(issue2151): avoid using stale broker IPs for AutoBalancer consumer (

    #2152)
    
    close #2151
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Nov 13, 2024
    Copy the full SHA
    ab249e0 View commit details
  2. chore(github): update code owners (#2155)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 13, 2024
    Copy the full SHA
    5eea51d View commit details

Commits on Nov 14, 2024

  1. feat(quota): support to update broker request rate quota (#2158)

    * refactor(quota): refactor `maybeRecordAndGetThrottleTimeMs`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * fix(quota): throttle the produce request whatever the acks is
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor(quota): separate `Request` in `ClientQuotaManager` and `RequestRate` in `BrokerQuotaManager`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * sytle: fix lint
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat(quota): support to update broker request rate quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test(quota): test update quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 14, 2024
    Copy the full SHA
    54ea5d4 View commit details
  2. feat(quota): support broker quota for slow fetch (#2160)

    * feat(quota): introduce `SLOW_FETCH` broker quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat(quota): add slow fetch quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test(quota): test broker slow fetch quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test(quota): test zero quota value
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 14, 2024
    Copy the full SHA
    4597b74 View commit details

Commits on Nov 15, 2024

  1. feat(backpressure): back pressure by system load (#2161)

    * feat(backpressure): init backpressure module
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat(backpressure): implement `DefaultBackPressureManager`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test(backpressure): test `DefaultBackPressureManager`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 15, 2024
    Copy the full SHA
    bcdd7e1 View commit details
  2. refactor(backpressure): introduce interface Checker (#2162)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 15, 2024
    Copy the full SHA
    7d6c2c9 View commit details
  3. feat(quota): support to get current quota by type (#2163)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 15, 2024
    Copy the full SHA
    e2f0e95 View commit details

Commits on Nov 20, 2024

  1. feat(tools/perf): create topics in batch (#2166)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 20, 2024
    Copy the full SHA
    ff3b68e View commit details
  2. chore(backpressure): log it on back pressure (#2164)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 20, 2024
    Copy the full SHA
    3073399 View commit details
  3. feat(table): table topic aspect (#2167)

    * feat(table): table topic aspect
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    
    * chore(table): fix PR review
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    
    * fix(table): fix unit test
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    
    ---------
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 20, 2024
    Copy the full SHA
    2e3d94a View commit details
  4. docs: Introducing AutoMQ Guru on Gurubase.io (#2159)

    Introducing AutoMQ Guru on Gurubase.io
    
    Signed-off-by: Kursat Aktas <kursat.ce@gmail.com>
    kursataktas authored Nov 20, 2024
    Copy the full SHA
    ef1520a View commit details

Commits on Nov 21, 2024

  1. chore(workflow): add spotless check (#2168)

    chore(workflow): spotless check
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 21, 2024
    Copy the full SHA
    1e217ee View commit details

Commits on Nov 22, 2024

  1. fix(stream): release FetchResults if the subsequent fetch fails (#2172

    )
    
    * fix(stream): release `FetchResult`s if the subsequent fetch fails
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * revert: "fix(stream): release `FetchResult`s if the subsequent fetch fails"
    
    This reverts commit 5836a6a.
    
    * refactor: add the `FetchResult` into the list in order rather than in reverse order
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * fix: release `FetchResult`s if failed to fetch
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 22, 2024
    Copy the full SHA
    e9bbf7a View commit details
  2. chore(stream): move asyncsemaphore to util (#2173)

    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 22, 2024
    Copy the full SHA
    d7b73f0 View commit details

Commits on Nov 25, 2024

  1. feat(quota): support to get current quota metric value... (#2170)

    * fix: fix logs
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat(quota): support to get current quota metric value
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor(backpressure): remove `Regulator#minimize`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(quota): increase the max of broker quota throttle time
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(backpressure): decrease cooldown time
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf(quota): increase the max of broker quota throttle time
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * docs: update comments
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 25, 2024
    Copy the full SHA
    89f2c6b View commit details
  2. feat(quota): exclude internal client IDs from broker quota (#2179)

    * feat(quota): exclude internal client IDs from broker quota
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat(autobalancer): mark producers and consumers internal clients
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 25, 2024
    Copy the full SHA
    018833e View commit details

Commits on Nov 26, 2024

  1. fix(quota): limit the max throttle time (#2180)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 26, 2024
    Copy the full SHA
    bfc0e61 View commit details
  2. fix(quota): check whether the client in white list before fetch (#2181)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 26, 2024
    Copy the full SHA
    57d8e5f View commit details
  3. chore(table): set table max.message.bytes to 20MiB (#2182)

    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 26, 2024
    Copy the full SHA
    64b3865 View commit details

Commits on Nov 28, 2024

  1. fix: use the "adjusted" maxSize in ElasticLogSegment#readAsync (#…

    …2184)
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 28, 2024
    Copy the full SHA
    dab96d7 View commit details
  2. fix: release PooledMemoryRecords if it's dropped in the fetch sessi…

    …on (#2185)
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Nov 28, 2024
    Copy the full SHA
    2583617 View commit details
  3. feat(table): auto create table topic control topic (#2186)

    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 28, 2024
    Copy the full SHA
    fba35b6 View commit details

Commits on Nov 29, 2024

  1. fix(issues2193): retry 2 times to cover most of BlockNotContinuousExc…

    …eption (#2194)
    
    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Nov 29, 2024
    Copy the full SHA
    d6641d7 View commit details

Commits on Dec 2, 2024

  1. feat(core): reuse unregistered node when requesting for next node id (#…

    …2200)
    
    * feat(core): reuse unregistered node when requesting for next node id (#2183)
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    
    * fix(core): fix getting duplicated node id
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    
    ---------
    
    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Dec 2, 2024
    Copy the full SHA
    6217fb1 View commit details
  2. chore(bin): increase the gc log file count (#2202)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 2, 2024
    Copy the full SHA
    504ce4e View commit details
  3. feat(backpressure): add metrics (#2198)

    * feat(backpressure): log it on recovery from backpressure
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: add metric fetch_limiter_waiting_task_num
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: add metric fetch_limiter_timeout_count
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: add metric fetch_limiter_time
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: add metric back_pressure_state
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: add metric broker_quota_limit
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * fix(backpressure): run checkers with fixed delay
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * style: fix lint
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * perf: drop too large values
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: record -1 for other states
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test: fix tests
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 2, 2024
    Copy the full SHA
    b470ba4 View commit details

Commits on Dec 3, 2024

  1. feat(backpressure): support dynamic configs (#2204)

    * feat(backpressure): make back pressure manager configurable
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test: test diabled
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: move backpressure from s3stream to kafka.core
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: init `BackPressureManager` in `BrokerServer`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: introduce `BackPressureConfig`
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * feat: make `BackPressureManager` reconfigurable
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * test: test reconfigurable
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: rename config key
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    * refactor: move metric "back_pressure_state" from s3stream to core
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    
    ---------
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 3, 2024
    Copy the full SHA
    52eb245 View commit details
  2. fix(core): write next node id into image (#2206)

    Signed-off-by: Shichao Nie <niesc@automq.com>
    SCNieh authored Dec 3, 2024
    Copy the full SHA
    46c0e76 View commit details
  3. feat(backpressure): stop and remove all scheduled tasks on shutdown (#…

    …2207)
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 3, 2024
    Copy the full SHA
    1cdce66 View commit details
  4. fix(backpressure): start before registering to dynamic configs (#2208)

    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 3, 2024
    Copy the full SHA
    eab1c07 View commit details
  5. fix(backpressure): fix metric value of back pressure state (#2209)

    fix: fix metric value of back pressure state
    
    Signed-off-by: Ning Yu <ningyu@automq.com>
    Chillax-0v0 authored Dec 3, 2024
    Copy the full SHA
    89ccf10 View commit details
  6. feat(config): add table topic conversion type configuration (#2203)

    * feat(config): add table topic conversion type configurations
    
    * feat(config): rename table topic type to schema type and update related configurations
    
    * feat(config): add table topic schema registry URL configuration and validation
    
    * test(config): add unit tests for ControllerConfigurationValidator table topic schema configuration
    
    * fix(tests): update exception type in ControllerConfigurationValidatorTableTest for schema validation
    
    * feat(config): polish code
    Gezi-lzq authored Dec 3, 2024
    Copy the full SHA
    af4dd1f View commit details

Commits on Dec 5, 2024

  1. chore(gradle): update aws version to 2.29.26 (#2210)

    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Dec 5, 2024
    Copy the full SHA
    e95f608 View commit details
  2. fix(docker): fix docker compose quick start (#2213)

    Signed-off-by: Robin Han <hanxvdovehx@gmail.com>
    superhx authored Dec 5, 2024
    Copy the full SHA
    35522a8 View commit details
Loading