Provide guidance for setting readers and writers

Description

The following page gives some guidance on readers and writers: https://docs.couchbase.com/server/6.5/manage/manage-settings/general-settings.html#data-settings

It needs to be enhanced to provide the following guidance for the 'Fixed Value' selection:

" A good rule of thumb is to set the readers and writers to the queue depth of the underlying IO subsystem." 

 

Environment

None

Release Notes Description

None

Activity

Show:

Shivani Gupta June 17, 2020 at 2:02 PM

Thanks .

So here is the guidance for the documentation (for the Fixed Value bullet point):

"A good rule of thumb is set each of readers and writers equal to the queue depth of the underlying IO subsystem (i.e. readers = queue_depth and writers = queue_depth). However, for best performance it is recommended to benchmark with different settings and pick the one that best meets your throughput and latency requirements in your environment"

Dave Rigby June 17, 2020 at 10:53 AM

Dave Rigby one clarification needed is whether the (readers + writers) = queue depth or is it (readers = queue depth) and (writers = queue depth)?

As a basic rule for the documentation I would suggest the latter - for a couple of reasons:

  1. Both readers (bgFetch, DCP backfill) and writers (flushing, compaction) are bursty - as such you probably want utilise the available queue depth for each task when it's running. If we only gave readers half the queue depth then when not flushing / compacting you'd likely not make use of the available disk bandwidth.

  2. Writes arn't actually blocking (until we issue the final fsync in a batch) - as such you're not necessarily consuming any disk IO (and queue depth) all the time.

Ultimately, for optimal performance in a given workload / environment the user would need to experiment and benchmark the different settings and pick the one which meets their requirements (typically highest OP/s while still within their latency SLAs). I think a reasonable starting point is reader=writers=queue_depth.

Shivani Gupta June 16, 2020 at 11:54 PM

 one clarification needed is whether the (readers + writers) = queue depth or is it (readers = queue depth) and (writers = queue depth)?

Shivani Gupta June 16, 2020 at 11:53 PM

.  FYI

Unresolved
Pinned fields
Click on the next to a field label to start pinning.

Details

Assignee

Reporter

Story Points

Components

Affects versions

Priority

Instabug

Open Instabug

PagerDuty

Sentry

Zendesk Support

Created June 16, 2020 at 11:53 PM
Updated March 15, 2022 at 5:18 PM
Instabug