mirror of
https://github.com/katanemo/plano.git
synced 2026-04-25 16:56:24 +02:00
b30ad791f7
* Fix link issues and add icons * Improve Doc * fix test * making minor modifications to shuguangs' doc changes --------- Co-authored-by: Salman Paracha <salmanparacha@MacBook-Pro-261.local> Co-authored-by: Adil Hafeez <adil@katanemo.com>
21 lines
896 B
ReStructuredText
21 lines
896 B
ReStructuredText
.. _arch_overview_threading:
|
|
|
|
Threading Model
|
|
===============
|
|
|
|
Arch builds on top of Envoy's single process with multiple threads architecture.
|
|
|
|
A single *primary* thread controls various sporadic coordination tasks while some number of *worker*
|
|
threads perform filtering, and forwarding.
|
|
|
|
Once a connection is accepted, the connection spends the rest of its lifetime bound to a single worker
|
|
thread. All the functionality around prompt handling from a downstream client is handled in a separate worker thread.
|
|
This allows the majority of Arch to be largely single threaded (embarrassingly parallel) with a small amount
|
|
of more complex code handling coordination between the worker threads.
|
|
|
|
Generally, Arch is written to be 100% non-blocking.
|
|
|
|
.. tip::
|
|
|
|
For most workloads we recommend configuring the number of worker threads to be equal to the number of
|
|
hardware threads on the machine.
|