2.2 Configuration

Configuration files aren't mandatory but they provide a convenient way to specify options that you use regularly when running EventQL programs, so you don't have to enter them on the command line each time you run the program.

Configuration File Format

EventQL uses the INI file format, that is a simple text file grouped into sections that consists of key-value pairs.

[client]
host = prod.example.com ; execute queries on this server

You can override every option set in the configuration file by using the command line option -C followed by the corresponding section.key=value pair.

$ evql -C client.host=localhost

Configuration File Path

If no explicit config path is provided using the --config option, EventQL will search for the configuration file at the following locations:

Binaries	Config Search Paths
evqld	/etc/evqld.conf
evql, evqlctl	/etc/evql.conf ~/.evql.conf

Configuration Options

The EventQL configuration options are grouped in three sections: client, server and cluster.

Option	Default Value	Description
cluster.*
cluster.name	—	The name of the cluster
cluster.coordinator	—	The cluster coordinator service. Legal values: "zookeeper"
cluster.zookeeper_hosts	—	A comma-separated list of zookeeper hosts (only used when cluster.coordinator=zookeeper)
cluster.rebalance_interval	60000000
cluster.allowed_hosts	—	A comma-separated list of CIDR network ranges that are allowed to connect as internal nodes to the cluster. This setting does not affect which hosts are allowed to connect as a client. You can set this option to "0.0.0.0/0" to allow all hosts to connect as internal nodes.
cluster.allow_anonymous	true	Allow anonymous users to connect to the cluster Note: this does not circumvent client auth or any other ACLs. It merely controls if an anonymous user is even allowed to connect, let alone execute an operation.
cluster.allow_drop_table	true	If false, DROP TABLE is globally forbidden, regardless ACLs.
cluster.allow_create_database	true	If false, CREATE DATABASE is globally forbidden, regardless of ACLs.
server.*
server.datadir	—	The location of the EvenQL data directory (mandatory)
server.listen	—	The address (host:port) on which the server should listen. NOTE that this address is published to the coordinator service and must be a reachable by all other servers in the cluster. I.e. you can't use localhost or 0.0.0.0. (mandatory)
server.name	—	The name of the server (optional)
server.pidfile	—	If set, the server will write a pidfile to the provided path and aquire an exclusive lock on the pidfile. If the exclusive lock fails, the server will exit.
server.daemonize	false
server.indexbuild_threads	2	The number of background compaction threads to start
server.replication_threads_max	4	The max number of background replication threads to start
server.client_auth_backend	—
server.internal_auth_backend	—
server.noleader	false	If set to true, this server will not partake in leader election
server.noalloc	false	If set to true, no partitions/data will be allocated to this server
server.gc_mode	MANUAL
server.gc_interval	30000000
server.cachedir_maxsize	68719476736	Unit: Bytes
server.disk_capacity		The maximum number of bytes that the server is allowed to write/use on disk. Unit is Bytes. This is an optional limit, if it is unset, the server will use the actual number of free bytes on disk as the limit. Even if the limit is set and allows using more disk space than is actually available, the server will use the (smaller) real limit.
server.loadinfo_publish_interval	15m	How often should the server publish it's current load info (i.e disk usage and other stats) to the cluster. Unit is microseconds. The load info is used when deciding on which server to allocate new chunks, so a shorter interval and therefore more up-to-date load info is usually better. However, making the interval smaller will increase the load on the coordination service (e.g. ZooKeeper). The QPS to to the coordination service can be calculated using "num_servers / interval_in_s". So with the default value of 15 minutes and 1,000 servers we will have roughly 1 write QPS to Zookeeper (good). With 10,000 servers we have 10 write QPS (still okay).
server.load_limit_soft	0.95	This value controls the upper limit on a servers disk utilization. If the soft load limit was reached, now new partitions will be allocated on the server. (Default: 95%)
server.load_limit_hard	0.98	This value controls the upper limit on a servers disk utilization. If the soft load limit was reached, now new partitions will be allocated on the server and the cluster will slowly start to remove existing partitions from the server until the disk usage falls below the soft limit.
server.partitions_loading_limit_soft	4	The maximum number of partitions that can be loading on a server at the same time before allocations with priorty "IDLE" will stop being placed on the server. Allocations with priorty "BEST_EFFORT" or "MUST_ALLOCATE" may still be placed on the server even after the soft limit was reached.
server.partitions_loading_limit_hard	64	The maximum number of partitions that can be loading on a server at the same time before allocations with priorty "IDLE" and "BEST_EFFORT" will stop being placed on the server. Allocations with priority "MUST_ALLOCATE" may still be placed on the server even after the soft limit was reached.
server.c2s_io_timeout	60s	How long should the server wait for data on a connection to a client when it expects the data to arrive immediately. (optional, unit: microseconds)
server.c2s_idle_timeout	30min	How long should the server wait for new data on an idle connection to a client. An idle connection is a connection where no data is expected to arrive immediately. (optional, unit: microseconds)
server.s2s_io_timeout	10s	How long should the server wait for data on a connection to another server when it expects the data to arrive immediately. (optional, unit: microseconds)
server.s2s_idle_timeout	10s	How long should the server wait for new data on an idle connection to another server. An idle connection is a connection where no data is expected to arrive immediately. (optional, unit: microseconds)
server.s2s_pool_max_connections	unlimited	Limit how many (internal) connections each server should keep in its connection pool. Set this value to zero to turn off connection pooling.
server.s2s_pool_max_connections_per_host	unlimited	Limit how many (internal) connections each server should keep in its connection pool per each distinct remote endpoint.
server.s2s_pool_linger_timeout	1s	Controls after which time an idle connection is removed (and closed) from the connection pool. Unit is microseconds.
server.http_io_timeout	1s	Configures the HTTP I/O timeout. The timeout controls how long the server will wait for the client to send the next byte of the request while reading the http request as well as how long the server will wait for the client to read the next byte of the response while writing the response. (optional, unit: microseconds)
server.heartbeat_interval	1s	How often should the server send a keepalive/heartbeat frame on a busy connection. Note that this value must be lower than the idle timeout and also puts a lower limit on the idle timeout that a connection client may choose. (optional, unit: microseconds)
server.query_progress_rate_limit	250ms	How often should the server send a progress event. (optional, unit: microseconds)
server.query_max_concurrent_shards	256	The default maximum number of shards to be executed in parallel/ concurrently for a single query. In other words this setting limits the maximum parallelism for a query. You should consider increasing the value if you're running on more than 64 machines.
server.query_max_concurrent_shards_per_host	6	The default maximum number of shards to be executed on any given host for a single query.
server.query_failed_shard_policy	tolerate	The failed shard policy can either be "tolerate" or "error". If the value is "tolerate" failed shards will be ignore/excluded from the query result (the percentage of 'missing data' will be returned with each result). If the value is "error" any failed shard will result in a query error. Valid values: "tolerate", "error"
client.*
client.host	localhost	The hostname of the EventQL server
client.port	9175	The port of the EventQL server
client.database		The database that should be used for following queries (optional)
client.user	$USER	Username to use when connecting to server (optional)
client.password		Password to use when connecting to server (optional)
client.auth_token		Auth-Token to use when connecting to server (optional)
client.timeout	60s	Timeout to use when connecting to server (unit is microseconds)
client.history_file	$HOME/.evql_history	Where to write the interactive shell history file
client.history_maxlen	1024	Maximum number of entries in the interactive shell history file