dask · jacobtomlinson · Feb 23, 2023 · Feb 1, 2023 · Feb 8, 2023 · Feb 8, 2023
diff --git a/docs/source/best-practices.rst b/docs/source/best-practices.rst
@@ -369,3 +369,5 @@ Compute related results with shared computations in a single :func:`dask.compute
 
 This allows Dask to compute the shared parts of the computation (like the
 ``dd.read_csv`` call above) only once, rather than once per ``compute`` call.
+For more guidance on when to call ``compute`` when working with Dask DataFrames
+or Arrays see `this blog post <https://www.coiled.io/blog/dask-compute>`_.
diff --git a/docs/source/deploying-cloud.rst b/docs/source/deploying-cloud.rst
@@ -14,9 +14,10 @@ You may want to consider the following options:
     and `Dask-Yarn <https://yarn.dask.org>`_.
 
     Specific documentation for the popular Amazon EMR service can be found
-    `here <https://yarn.dask.org/en/latest/aws-emr.html>`_
+    `here <https://yarn.dask.org/en/latest/aws-emr.html>`_.
 3.  Directly launching cloud resources such as VMs or containers via a cluster manager with
-    `Dask Cloud Provider <https://cloudprovider.dask.org/en/latest/>`_
+    `Dask Cloud Provider <https://cloudprovider.dask.org/en/latest/>`_.
+4. A commercial Dask deployment option like `Coiled <https://coiled.io/>`_ to handle the creation and management of Dask clusters on a cloud computing environment (AWS and GCP).
 
 Cloud Deployment Example
 ------------------------
@@ -57,7 +58,7 @@ Data Access
 -----------
 
 You may want to install additional libraries in your Jupyter and worker images
-to access the object stores of each cloud:
+to access the object stores of each cloud (see :doc:`how-to/connect-to-remote-data`):
 
 -  `s3fs <https://s3fs.readthedocs.io/>`_ for Amazon's S3
 -  `gcsfs <https://gcsfs.readthedocs.io/>`_ for Google's GCS

diff --git a/docs/source/deploying.rst b/docs/source/deploying.rst
@@ -19,18 +19,15 @@ The ``dask.distributed`` scheduler works well on a single machine and scales to
 in a cluster. We recommend using ``dask.distributed`` clusters at all scales for the following
 reasons:
 
-1.  It provides access to asynchronous API, notably :doc:`Futures <../../futures>`
+1.  It provides access to asynchronous APIs, notably :doc:`Futures <../../futures>`.
 2.  It provides a diagnostic dashboard that can provide valuable insight on
-    performance and progress
+    performance and progress (see :doc:`dashboard`).
 3.  It handles data locality with sophistication, and so can be more
     efficient than the multiprocessing scheduler on workloads that require
-    multiple processes
+    multiple processes.
 
 This page describes various ways to set up Dask clusters on different hardware, either
-locally on your own machine or on a distributed cluster.  If you are just
-getting started then you can save this page for later as Dask runs perfectly well on a single machine
-without a distributed scheduler. But once you start using Dask in anger you'll find a lot of benefit
-both in terms of scaling and debugging by using the distributed scheduler.
+locally on your own machine or on a distributed cluster.
 
 You can continue reading or watch the screencast below:
 
@@ -45,24 +42,23 @@ You can continue reading or watch the screencast below:
            allowfullscreen></iframe>
 
 If you import Dask, set up a computation, and call ``compute``, then you
-will use the single-machine scheduler by default.  To use the ``dask.distributed``
-scheduler you must set up a ``Client``.
+will use the single-machine scheduler by default.
 
 .. code-block:: python
 
    import dask.dataframe as dd
    df = dd.read_csv(...)
    df.x.sum().compute()  # This uses the single-machine scheduler by default
 
+To use the ``dask.distributed`` scheduler you must set up a ``Client``.
+
 .. code-block:: python
 
    from dask.distributed import Client
    client = Client(...)  # Connect to distributed cluster and override default
    df.x.sum().compute()  # This now runs on the distributed system
 
-There are many ways to start the distributed scheduler and worker components that your client
-needs to connect to. You can run them manually using :doc:`command line tools <deploying-cli>`
-but often the most straight forward way is to use a *cluster manager* utility class.
+There are many ways to start the distributed scheduler and worker components, however, the most straight forward way is to use a *cluster manager* utility class.
 
 .. code-block:: python
 
@@ -71,12 +67,15 @@ but often the most straight forward way is to use a *cluster manager* utility cl
    client = Client(cluster)  # Connect to distributed cluster and override default
    df.x.sum().compute()  # This now runs on the distributed system
 
-There are a number of different *cluster managers* available, so you can use
-Dask distributed with a range of platforms. These *cluster managers* deploy a scheduler
+These *cluster managers* deploy a scheduler
 and the necessary workers as determined by communicating with the *resource manager*.
-All *cluster managers* follow the same interface but have platform specific configuration
-options. This makes it convenient to switch from your local machine to a remote multi-node
-cluster without sacrificing the flexibility of the platform you are deploying on.
+All *cluster managers* follow the same interface, but with platform-specific configuration
+options, so you can switch from your local machine to a remote cluster with very minimal code changes.
+
+.. figure:: images/dask-cluster-manager.svg
+   :scale: 50%
+
+   An overview of cluster management with Dask distributed.
 
 `Dask Jobqueue <https://github.com/dask/dask-jobqueue>`_, for example, is a set of
 *cluster managers* for HPC users and works with job queueing systems
@@ -93,62 +92,82 @@ Those workers are then allocated physical hardware resources.
    client = Client(cluster)  # Connect to distributed cluster and override default
    df.x.sum().compute()  # This now runs on the distributed system
 
-.. figure:: images/dask-cluster-manager.svg
-   :scale: 50%
-
-   An overview of cluster management with Dask distributed.
-
 .. _deployment-options:
 
-To summarize, you can use the default, single-machine scheduler to use Dask
-on your local machine. If you'd like use a cluster *or* simply take advantage
-of the :doc:`extensive diagnostics <../diagnostics-distributed>`,
-you can use Dask distributed. The following resources explain
-in more detail how to set up Dask on a variety of local and distributed hardware:
+The following resources explain how to set up Dask on a variety of local and distributed hardware.
 
 .. _deployment-single-machine:
 
-- Single Machine:
-    - :doc:`Default Scheduler <scheduling>`: The no-setup default.
-      Uses local threads or processes for larger-than-memory processing
-    - :doc:`dask.distributed <deploying-python>`: The sophistication of
-      the newer system on a single machine.  This provides more advanced
-      features while still requiring almost no setup.
+Single Machine
+--------------
+
+Dask runs perfectly well on a single machine with or without a distributed scheduler.
+But once you start using Dask in anger you’ll find a lot of benefit both in terms of scaling
+and debugging by using the distributed scheduler.
+
+- :doc:`Default Scheduler <scheduling>`
+   The no-setup default. Uses local threads or processes for larger-than-memory processing
+
+- :doc:`dask.distributed <deploying-python>`
+   The sophistication of the newer system on a single machine.  This provides more advanced features while still requiring almost no setup.
 
 .. _deployment-distributed:
 
-- Distributed computing:
-    - `Beginner's Guide to Configuring a Dask distributed Cluster <https://blog.dask.org/2020/07/30/beginners-config>`_
-    - `Overview of cluster management options <https://blog.dask.org/2020/07/23/current-state-of-distributed-dask-clusters>`_
-    - :doc:`Manual Setup <deploying-cli>`: The command line interface to set up
-      ``dask-scheduler`` and ``dask-worker`` processes.  Useful for IT or
-      anyone building a deployment solution.
-    - :doc:`SSH <deploying-ssh>`: Use SSH to set up Dask across an un-managed
-      cluster.
-    - :doc:`High Performance Computers <deploying-hpc>`: How to run Dask on
-      traditional HPC environments using tools like MPI, or job schedulers like
-      SLURM, SGE, TORQUE, LSF, and so on.
-    - :doc:`Kubernetes <deploying-kubernetes>`: Deploy Dask with the
-      popular Kubernetes resource manager using either Helm or a native deployment.
-    - `YARN / Hadoop <https://yarn.dask.org/en/latest/>`_: Deploy
-      Dask on YARN clusters, such as are found in traditional Hadoop
-      installations.
-    - `Dask Gateway <https://gateway.dask.org/>`_ provides a secure,
-      multi-tenant server for managing Dask clusters and allows users to launch
-      and use Dask clusters in a shared cluster environment.
-    - :doc:`Python API (advanced) <deploying-python-advanced>`: Create
-      ``Scheduler`` and ``Worker`` objects from Python as part of a distributed
-      Tornado TCP application.  This page is useful for those building custom
-      frameworks.
-    - :doc:`Docker <deploying-docker>` images are available and may be useful
-      in some of the solutions above.
-    - :doc:`Cloud <deploying-cloud>` for current recommendations on how to
-      deploy Dask and Jupyter on common cloud providers like Amazon, Google, or
-      Microsoft Azure.
-- Hosted / managed Dask clusters (listed in alphabetical order):
-    - `Coiled <https://coiled.io/>`_ handles the creation and management of
-      Dask clusters on cloud computing environments (AWS, Azure, and GCP).
-    - `Domino Data Lab <https://www.dominodatalab.com/>`_ lets users create
-      Dask clusters in a hosted platform.
-    - `Saturn Cloud <https://saturncloud.io/>`_ lets users create
-      Dask clusters in a hosted platform or within their own AWS accounts.
+Distributed Computing
+---------------------
+
+There are a number of ways to run Dask on a distributed cluster (see the `Beginner's Guide to Configuring a Distributed Dask Cluster <https://blog.dask.org/2020/07/30/beginners-config>`_).
+
+High Performance Computing
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+See :doc:`deploying-hpc` for more details.
+
+- `Dask-Jobqueue <https://jobqueue.dask.org>`_
+    Provides cluster managers for PBS, SLURM, LSF, SGE and other resource managers.
+- `Dask-MPI <http://mpi.dask.org/en/latest/>`_
+    Deploy Dask from within an existing MPI environment.
+- `Dask Gateway for Jobqueue <https://gateway.dask.org/install-jobqueue.html>`_
+    Multi-tenant, secure clusters. Once configured, users can launch clusters without direct access to the underlying HPC backend.
+
+Kubernetes
+~~~~~~~~~~
+
+See :doc:`deploying-kubernetes` for more details.
+
+- :doc:`Helm <deploying-kubernetes-helm>`
+   An easy way to stand up a long-running Dask cluster.
+- `Dask Kubernetes <https://kubernetes.dask.org/en/latest/>`_
+   For native Kubernetes integration for fast moving or ephemeral deployments.
+- `Dask Gateway for Kubernetes <https://gateway.dask.org/install-kube.html>`_
+    Multi-tenant, secure clusters. Once configured, users can launch clusters without direct access to the underlying Kubernetes backend.
+
+Cloud
+~~~~~
+
+See :doc:`deploying-cloud` for more details.
+
+- `Dask-Yarn <https://yarn.dask.org>`_
+    Deploy Dask on YARN clusters, such as are found in traditional Hadoop installations.
+- `Dask Cloud Provider <https://cloudprovider.dask.org/en/latest/>`_
+    Constructing and managing ephemeral Dask clusters on AWS, DigitalOcean, GCP, Azure, and Hertzner
+- You can use `Coiled <https://coiled.io/>`_, a commercial Dask deployment option, to handle the creation and management of Dask clusters on cloud computing environments (AWS and GCP).
+
+Ad-hoc deployments
+~~~~~~~~~~~~~~~~~~
+
+- :doc:`Manual Setup <deploying-cli>`
+    The command line interface to set up ``dask-scheduler`` and ``dask-worker`` processes.
+- :doc:`deploying-ssh`
+    Use SSH to set up Dask across an un-managed cluster.
+- :doc:`Python API (advanced) <deploying-python-advanced>`
+    Create ``Scheduler`` and ``Worker``   objects from Python as part of a distributed Tornado TCP application.
+
+.. _managed-cluster-solutions:
+
+Managed Solutions
+~~~~~~~~~~~~~~~~~
+
+- You can use `Coiled <https://coiled.io/>`_ to handle the creation and management of Dask clusters on cloud computing environments (AWS and GCP).
+- `Domino Data Lab <https://www.dominodatalab.com/>`_ lets users create Dask clusters in a hosted platform.
+- `Saturn Cloud <https://saturncloud.io/>`_ lets users create Dask clusters in a hosted platform or within their own AWS accounts.
diff --git a/docs/source/ecosystem.rst b/docs/source/ecosystem.rst
@@ -74,3 +74,8 @@ There are many different implementations of the Dask distributed cluster.
   cluster backend.
 - `dask-cuda <https://github.com/rapidsai/dask-cuda>`_: Construct a Dask cluster which resembles ``LocalCluster``  and is specifically
   optimized for GPUs.
+
+Commercial Dask Deployment Options
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- You can use `Coiled <https://coiled.io/>`_ to handle the creation and management of Dask clusters on cloud computing environments (AWS and GCP).
diff --git a/docs/source/faq.rst b/docs/source/faq.rst
@@ -184,6 +184,12 @@ those on Hadoop, HPC, Kubernetes, and Cloud clusters.
 
     For more information see :doc:`deploying-kubernetes`
 
+4. **Commercial Dask deployment:**
+
+   - You can use `Coiled <https://coiled.io/>`_ to handle the creation and management of Dask clusters on cloud computing environments (AWS and GCP).
+   - `Domino Data Lab <https://www.dominodatalab.com/>`_ Lets users create Dask clusters in a hosted platform.
+   - `Saturn Cloud <https://saturncloud.io/>`_ Lets users create Dask clusters in a hosted platform or within their own AWS accounts.
+
 
 Is Dask secure?
 ~~~~~~~~~~~~~~~
@@ -227,6 +233,8 @@ In these cases some level of explicit indirection may be required.  For this, we
 recommend the `Dask Gateway project <https://gateway.dask.org>`_, which uses IT-level
 permissions to properly route authenticated users into secure resources.
 
+You may also want to consider a managed cluster solution (see :ref:`managed-cluster-solutions`).
+
 
 How do I manage software environments?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~