@ -1314,13 +1314,19 @@ used by the redis result backend.
.. _conf-cassandra-result-backend:
Cassandra backend settings
--------------------------
Cassandra/AstraDB backend settings
----------------------------------
.. note ::
This Cassandra backend driver requires :pypi: `cassandra-driver` .
This backend can refer to either a regular Cassandra installation
or a managed Astra DB instance. Depending on which one, exactly one
between the :setting: `cassandra_servers` and
:setting: `cassandra_secure_bundle_path` settings must be provided
(but not both).
To install, use :command: `pip` :
.. code-block :: console
@ -1339,10 +1345,32 @@ This backend requires the following configuration directives to be set.
Default: `` [] `` (empty list).
List of `` host `` Cassandra servers. For example::
List of `` host `` Cassandra servers. This must be provided when connecting to
a Cassandra cluster. Passing this setting is strictly exclusive
to :setting: `cassandra_secure_bundle_path` . Example::
cassandra_servers = ['localhost']
.. setting :: cassandra_secure_bundle_path
`` cassandra_secure_bundle_path ``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Default: None.
Absolute path to the secure-connect-bundle zip file to connect
to an Astra DB instance. Passing this setting is strictly exclusive
to :setting: `cassandra_servers` .
Example::
cassandra_secure_bundle_path = '/home/user/bundles/secure-connect.zip'
When connecting to Astra DB, it is necessary to specify
the plain-text auth provider and the associated username and password,
which take the value of the Client ID and the Client Secret, respectively,
of a valid token generated for the Astra DB instance.
See below for an Astra DB configuration example.
.. setting :: cassandra_port
`` cassandra_port ``
@ -1359,7 +1387,7 @@ Port to contact the Cassandra servers on.
Default: None.
The key- space in which to store the results. For example::
The keyspace in which to store the results. For example::
cassandra_keyspace = 'tasks_keyspace'
@ -1446,18 +1474,85 @@ Named arguments to pass into the ``cassandra.cluster`` class.
'protocol_version': 3
}
Example configuration
~~~~~~~~~~~~~~~~~~~~~
Example configuration (Cassandra)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block :: python
result_backend = 'cassandra://'
cassandra_servers = ['localhost']
cassandra_keyspace = 'celery'
cassandra_table = 'tasks'
cassandra_read_consistency = 'ONE '
cassandra_write_consistency = 'ONE '
cassandra_read_consistency = 'QUORUM '
cassandra_write_consistency = 'QUORUM '
cassandra_entry_ttl = 86400
Example configuration (Astra DB)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block :: python
result_backend = 'cassandra://'
cassandra_keyspace = 'celery'
cassandra_table = 'tasks'
cassandra_read_consistency = 'QUORUM'
cassandra_write_consistency = 'QUORUM'
cassandra_auth_provider = 'PlainTextAuthProvider'
cassandra_auth_kwargs = {
'username': '<<CLIENT_ID_FROM_ASTRA_DB_TOKEN>>',
'password': '<<CLIENT_SECRET_FROM_ASTRA_DB_TOKEN>>'
}
cassandra_secure_bundle_path = '/path/to/secure-connect-bundle.zip'
cassandra_entry_ttl = 86400
Additional configuration
~~~~~~~~~~~~~~~~~~~~~~~~
The Cassandra driver, when estabilishing the connection, undergoes a stage
of negotiating the protocol version with the server(s). Similarly,
a load-balancing policy is automatically supplied (by default
`` DCAwareRoundRobinPolicy `` , which in turn has a `` local_dc `` setting, also
determined by the driver upon connection).
When possible, one should explicitly provide these in the configuration:
moreover, future versions of the Cassandra driver will require at least the
load-balancing policy to be specified (using `execution profiles <https://docs.datastax.com/en/developer/python-driver/3.25/execution_profiles/> `_ ,
as shown below).
A full configuration for the Cassandra backend would thus have the
following additional lines:
.. code-block :: python
from cassandra.policies import DCAwareRoundRobinPolicy
from cassandra.cluster import ExecutionProfile
from cassandra.cluster import EXEC_PROFILE_DEFAULT
myEProfile = ExecutionProfile(
load_balancing_policy=DCAwareRoundRobinPolicy(
local_dc='datacenter1', # replace with your DC name
)
)
cassandra_options = {
'protocol_version': 5, # for Cassandra 4, change if needed
'execution_profiles': {EXEC_PROFILE_DEFAULT: myEProfile},
}
And similarly for Astra DB:
.. code-block :: python
from cassandra.policies import DCAwareRoundRobinPolicy
from cassandra.cluster import ExecutionProfile
from cassandra.cluster import EXEC_PROFILE_DEFAULT
myEProfile = ExecutionProfile(
load_balancing_policy=DCAwareRoundRobinPolicy(
local_dc='europe-west1', # for Astra DB, region name = dc name
)
)
cassandra_options = {
'protocol_version': 4, # for Astra DB
'execution_profiles': {EXEC_PROFILE_DEFAULT: myEProfile},
}
.. _conf-s3-result-backend:
S3 backend settings
Stefano Lottini
4 months ago
committed by