Provide guidance for setting readers and writers
Description
Environment
Release Notes Description
Activity

Shivani Gupta June 17, 2020 at 2:02 PM
Thanks .
So here is the guidance for the documentation (for the Fixed Value bullet point):
"A good rule of thumb is set each of readers and writers equal to the queue depth of the underlying IO subsystem (i.e. readers = queue_depth and writers = queue_depth). However, for best performance it is recommended to benchmark with different settings and pick the one that best meets your throughput and latency requirements in your environment"

Dave Rigby June 17, 2020 at 10:53 AM
Dave Rigby one clarification needed is whether the (readers + writers) = queue depth or is it (readers = queue depth) and (writers = queue depth)?
As a basic rule for the documentation I would suggest the latter - for a couple of reasons:
Both readers (bgFetch, DCP backfill) and writers (flushing, compaction) are bursty - as such you probably want utilise the available queue depth for each task when it's running. If we only gave readers half the queue depth then when not flushing / compacting you'd likely not make use of the available disk bandwidth.
Writes arn't actually blocking (until we issue the final
fsync
in a batch) - as such you're not necessarily consuming any disk IO (and queue depth) all the time.
Ultimately, for optimal performance in a given workload / environment the user would need to experiment and benchmark the different settings and pick the one which meets their requirements (typically highest OP/s while still within their latency SLAs). I think a reasonable starting point is reader=writers=queue_depth.

Shivani Gupta June 16, 2020 at 11:54 PM
one clarification needed is whether the (readers + writers) = queue depth or is it (readers = queue depth) and (writers = queue depth)?

Shivani Gupta June 16, 2020 at 11:53 PM
. FYI
Details
Details
Assignee

Reporter

Story Points
Components
Affects versions
Priority
Instabug
PagerDuty
PagerDuty Incident
PagerDuty

Sentry
Linked Issues
Sentry
Zendesk Support
Linked Tickets
Zendesk Support

The following page gives some guidance on readers and writers: https://docs.couchbase.com/server/6.5/manage/manage-settings/general-settings.html#data-settings
It needs to be enhanced to provide the following guidance for the 'Fixed Value' selection:
" A good rule of thumb is to set the readers and writers to the queue depth of the underlying IO subsystem."