QGIS RPC services
RPC services runs QGIS server processes and expose gRPC interfaces. for requesting and managing the QGIS processes.
Workers may be grouped by pools that share the exact same configuration and network address.
For examples, scaling a docker container with a running rpc-server in a docker compose stack automatically create a pool of workers.
That is, a pool is addressed by a gRPC client as a single endpoint. (i.e qgis-rpc like in the Docker compose setup example.
QGIS processes
A worker may run a configurable number of QGIS processes. Incoming QGIS requests to the gRPC service are distributed with a fair-queuing dispatching algorithm to the child QGIS server processes.
You may increase or decrease the number of processes but another strategy is to scale the number of worker services while keeping the number of sub-processes relatively small.
Depending of the situation it may be better to choose one or another strategy.
Life cycle and pressure conditions
If a processes crash, the worker is then in a degraded state that can be monitored.
In degraded state, the RPC service will try to restore dead workers so as to keep the number of live QGIS processes constante.
In some situation the number of dead process exceed some limite will stop with an error code.
There is one condition for a worker to deliberatly exit with a error condition: the process failure pressure.
The process failure pressure is the ratio of failed processes over the initial number of configured processes. If this ratio raise above some configured limit, then the service will exit with critical error condition.
Process timeout
A process may be deliberatly killed (and thus increase the pressure) on long running requests.
If the response time exceed the request server.timeout then the process processing the request is considered as stalled and asked to abort gracefully the request. The grace timeout is controlled by the worker.cancel_timeout; if the process fail to abort the request then process is killed, which will increase the failure pressure.
Note
Configuration
When reading configuration from file, the format is TOML by default.
Check the configuration json schema.
TOML configuration
[logging]
level = "INFO"
[server]
#
# Use admin services
enable_admin_services = true
#
# Timeout for requests in seconds
timeout = 20
#
# Request timeout
#
# The maximum amount of time to wait in seconds before
# closing connections. During this period,
# no new connections are allowed.
shutdown_grace_period = 10
#
# The maximum allowed failure pressure.
# If the failure pressure exceed this value then
# the service will exit with critical error condition,
max_failure_pressure = 0.9
#
[server.listen]
#
# Socket address
address = "127.0.0.1:23456"
#
# Enable TLS
#
# Enable TLS, require certificat and key
enable_tls = false
#
# Path to TLS key file
#tls_key_file = # Optional
#
# Path to TLS cert PEM file
#tls_cert_file = # Optional
[worker]
#
# Name of the worker instance
name = ""
#
# Number of simultanous workers
num_processes = 1
#
# Timeout for starting child process
process_start_timeout = 5
#
# Cancel timeout
#
# The grace period to apply on worker timeout
# when attempting to cancel the actual request
# This number should be kept small (a few seconds) since it
# will be used after the response timeout.
#
cancel_timeout = 3
#
# Maximum queued requests
#
# The maximum number of requests that can be
# queued. If the number of waiting requests reach the limit,
# the subsequent requests will be returned with a `service unavailable`
# error.
max_waiting_requests = 50
#
# Max failure pressure
#
# The maximum allowed failure pressure.
# If the failure pressure exceed this value then
# the service will exit with critical error condition.
max_failure_pressure = 0.5
#
# Startup projects
#
# Projects to restore at startup
restore_projects = []
#
# Qgis configuration
#
[worker.qgis]
#
# Max number of projects in cache
#
# The maximum number of projects allowed in cache.
# The default value is set to 50 projects.
max_projects = 50
#
# Load project in cache when requested
#
# Load project in cache at request.
# If set to 'false', project not loaded in cache will
# return a 403 HTTP code when requested.
# Thus, adding project's to cache will require a specific
# action from another service or admininstrative
# management tools.
load_project_on_request = true
#
# Reload outdated project when requested
#
# Reload outdated project at request.
# If set to 'false', outdated project in cache will
# not be refreshed when requested.
# Thus, refreshing project's to cache will require a specific
# action from another service or admininstrative
# management tools.
reload_outdated_project_on_request = false
#
# Allow python embedded macros
#
# Set authorization to run Python Embedded in projects.
# If enabled, it will use the QGIS settings value defined in the
# QGIS settings options.
# If disabled, Python Embedded is completely disabled and QGIS defined
# settings will be ignored.
# For security reason this is disabled by default.
enable_python_embedded = false
#
# Maximum chunk size
#
# Set the maximum chunk size for streamed responses.
max_chunk_size = 1048576
#
# Qgis settings
#
# Qgis settings override.
# Use the syntax '<section>/<path>' for keys.
# Not that values defined here will override those
# from QGIS3.ini file.
qgis_settings = {}
#
# Ignore INT signal in worker
#
# Ignore INT signal in workers.
# This is useful when you don't want
# propagating signal from parent process.
ignore_interrupt_signal = true
#
# Projects configuration
#
# Projects and cache configuration
#
[worker.qgis.projects]
#
# Trust layer metadata
#
# Trust layer metadata.
# Improves layer load time by skipping expensive checks
# like primary key unicity, geometry type and
# srid and by using estimated metadata on layer load.
# Since QGIS 3.16
#
trust_layer_metadata = false
#
# Disable GetPrint requests
#
# Don't load print layouts.
# Improves project read time if layouts are not required,
# and allows projects to be safely read in background threads
# (since print layouts are not thread safe).
#
disable_getprint = false
#
# Force read only mode
#
# Force layers to open in read only mode
force_readonly_layers = true
#
# Ignore bad layers
#
# Allow projects to be loaded with event if it contains
# layers that cannot be loaded.
# Note that the 'dont_resolve_layers flag' trigger automatically
# this option.
#
ignore_bad_layers = false
#
# Disable OWS advertised urls
#
# Disable ows urls defined in projects.
# This may be necessary because Qgis projects
# urls override proxy urls.
disable_advertised_urls = false
#
# Scheme mapping definitions
#
# Defines mapping betweeen location base path and storage handler root url.
# Resource path relative to location will be joined the the root url path.
# In the case of Qgis storage, the handler is responsible for transforming
# the result url into a comprehensive format for the corresponding
# QgsProjectStorage implementation.
# This is handled by the default storage implementation for Qgis native
# project storage.
# In case of custom QgsProjectStorage, if the scheme does not allow passing
# project as path component, it is possible to specify a custom resolver function.
#
search_paths = {}
#
# Allow direct path resolution
#
# Allow direct path resolution if there is
# no matching from the search paths.
# Uri are directly interpreted as valid Qgis project's path.
# WARNING: allowing this may be a security vulnerabilty."
#
allow_direct_path_resolution = false
#
# Project storage Handler configurations
#
# Configure storage handlers.
# The name will be used as scheme for project's search path
# configuration.
#
#
[worker.qgis.projects.handlers.'key']
#handler = # Required
config = {}
#
# Plugins configuration
#
[worker.qgis.plugins]
#
# Plugin paths
#
# The list of search paths for plugins.
# Qgis plugins found will be loaded according to
# the 'install' list.
# If the list is empty, the 'QGIS_PLUGINPATH'
# variable will be checked.
paths = []
#
# Installable plugins
#
# The list of installable plugins.
# Note: if the plugin directory contains other plugins
# plugins not in the list will NOT be loaded !
# The Plugins will be installed at startup
# if the 'install_mode' is set to 'auto'.
# Note that an empty list means what it is:
# i.e, *no* installed plugins.
#install = # Optional
#
# Plugin installation mode
#
# If set to 'auto', plugins installation
# will be checked at startup. Otherwise,
# Installation will be done from already available
# plugins.
install_mode = "external"
#
# Enable processing scripts
#
# Enable publication of processing scripts
enable_scripts = true
#
# Extra builtins providers
#
# Load extra builtin processing providers
# such as 'grass' and 'otb'.
extra_builtin_providers = []
#
# Path to plugin manager executable
#
# The absolute path to the qgis-plugin_manager executable
# that will be used for installing plugin in automatic mode.
plugin_manager = "/usr/local/bin/qgis-plugin-manager"
#
# QGIS Network configuration
#
[worker.qgis.network]
#
# Transfer timeout in ms
#
# Transfers are aborted if no bytes are transferred before
# the timeout expires.
# If set to 0, the timeout is disobled.
# Default value is set to 10000 milliseconds.
#
transfer_timeout = 10000
#
# Trace network activity
trace = false
#
# Global cache policy
#
# Set a global cache policy for all requests"
# If set, this will override requests cache policy".
#
#cache_policy = # Optional
#
# Domain policies
#
# Set per domain policy
#
[worker.qgis.network.domain_policy.'key']
#
# Cache load control
#
# Override QNetworkRequest::CacheLoadControl for request.
#cache_policy = # Optional
#
# Transfer timeout in ms
#transfer_timeout = # Optional
Dynamic configuration setup
This is only available if you are running the service throught the official docker image.
The configuration may be set dynamically at startup by running an executable returning a configuration in JSon format.
The executable is controlled by the QJAZZ_CONFIG_EXEC variable.
The default settings allow you to define a remote URL for downloading the configuration at startup. See the basic-with-config-server example for an example of remote configuration setup.
Note
Qgis project’s cache overview
Each processes manage its own cache. This is due to a limitation in Qgis that prevent sharing resources between differents processes and the fact that Qgis server runtime is essentialy single threaded.
The cache in Qgis services do not use the default internal cache of Qgis server but its own caching system based on QgisProjectStorage objects. This ensure that any storage backends implemented or added in Qgis with plugins is supported.
Project’s access is uniform: the cache configuration define search paths which are indirection to the corresponding backends:
[worker.projects.search_paths]
'/a_path' = "/path/to/projects/" # Path to files volume
'/another/path' = "file:///other/projects/" # With explicit scheme
'/path/to/postgres' = "postgres://?service=name" # projects stored in postgres
Any following subpath to a search path is considered as the relative project’s path or the projects name user for url resolution:
/path/to/postgres/projname
will be resolved to:
postgres://?service=name&project=projname
From client perspective, a project is always refered by its search path followed by the (relative) project’s path or name:
/<search_path>/<project_path>
Dynamic paths
Dynamic paths allows to define templated path stems that will by be substitued in the final path resolution:
"/{user}/{theme}" = "/path/to/{user}/projects/{theme}"
and /alice/forests/coolmap.qgs will be resolved to:
"/path/to/alice/projects/forests/coolmap.qgis"
Note
Managing cache
Note
Cache can be managed with the service cli command:
Usage: qjazz-server-cli cache [OPTIONS] COMMAND [ARGS]...
Commands for cache management
Options:
--help Show this message and exit.
Commands:
catalog List available projects from search paths
checkout CheckoutProject PROJECT from cache
clear Clear cache
drop Drop PROJECT from cache
info Return info from PROJECT in cache
list List projects from cache
update Synchronize cache between processes
Project checkout
Whenever a project is checked out from cache, a cache status is returned
- NEW:
Project exists and is not loaded in cache
- NEEDUPDATE:
Project is already loaded and need to be updated because source storage has been modified
- UNCHANGED:
Project is loaded and is up to date
- REMOVED:
Project is loaded but has been removed from storage
- NOTFOUND:
Project does not exists
You may pull the project to make it change state depending on its inital state:
Initial state |
State after pull |
Action |
|---|---|---|
NEW |
UNCHANGED |
Project is loaded in cache |
NEEDUPDATE |
UNCHANGED |
Cached project is updated with new version |
UNCHANGED |
UNCHANGED |
No action |
REMOVED |
NOTFOUND |
Project is removed from cache |
NOTFOUND |
NOTFOUND |
No action |
Cache restoration
Projects loaded with the cache managment api are pinned in the cache: they cannot leave the cache except if removed explicitely.
Operations on cache are recorded internally and a when a QGIS processes is restored, all pinned projects are loaded.
The cache may be restored at startup either from the worker.restore_projects setting (or the CONF_WORKER__RESTORE_PROJECTS env variables).
Dynamic cache restoration
Dynamic restoration is only available with the official Docker image.
The cache restoration configuration may be set from remote location and indepedantly of the remote configuration setup using the QJAZZ_RESTORE_PROJECTS_EXEC env varible. The executable must return a comma separated list of projects to load at startup (internally it use the CONF_WORKER__RESTORE_PROJECTS).
Note
Note