% This LaTeX document was generated using the LaTeX backend of PlDoc,
% The SWI-Prolog documentation system



\section{About the SWI-Prolog Redis client}

\label{sec:redis-overview}

\href{https://redis.io}{Redis} is an in-memory key-value store. Redis can be
operated as a simple store for managing (notably) volatile persistent
data. Redis can operate in serveral modes, ranging from a single server
to clusters organised in several different ways to provide high
availability, resilience and replicating the data close to the involved
servers. In addition to being a key-value store, Redis enables
additional communication between clients such as \textit{publish/subscribe} to
message channels, \textit{streams}, etc.

These features can be used to connect \textit{micro services}, both for sharing
state, notifications and distributing tasks.

\subsection{Redis and threads}

\label{sec:redis-threads}

The connection between the redis client and server uses a \textit{stream pair}.
Although SWI-Prolog stream I/O is thread-safe, having multiple threads
using this same connection will mixup writes and their replies.

At the moment, the following locking is in place.

\begin{itemize}
    \item Connections created using \predref{redis_connect}{3} are \textit{not} locked. This
implies the connection handle may be used from a single thread only,
or \predref{redis}{3} requests must be protected using \predref{with_mutex}{2}.
    \item Redis/3 request using a \textit{server name} established using \predref{redis_server}{3}
are locked using a mutex with the same name as the server name.
\end{itemize}

\subsection{Redis TLS support}

\label{sec:redis-tls}

If SWI-Prolog includes the \const{ssl} library, the Redis client can connect
to the server using TLS (SSL). Connecting requires the same three
files as \verb$redis-cli$ requires: the root certificate file, a client
certificate and the private key of the client certificate. Below is
an example call to \predref{redis_server}{3}:

\begin{code}
:- redis_server(swish, localhost:6379,
                [ user(bob),
                  password("topsecret"),
                  version(3),
                  tls(true),
                  cacert('ca.crt'),
                  key('client.key'),
                  cert('client.cert')
                ]).
\end{code}

\subsection{Using Redis sentinels}

\label{sec:redis-sentinels}

Redis sentinels is one of the two options to create a high
availability service. It consists of minimally three Redis servers
and mininally three sentinel servers. The sentinel servers monitor
the Redis servers and will initiate a fail-over when the master
becomes disfunctional and certain safety constraints are satisfied. A
client needs to be aware of this setup. It is given an initial list
with (a subset of) the known sentinels. The client attempts to
connect to one of the sentinels and ask it for the current Redis
master server. Details are described in \href{https://redis.io/docs/reference/sentinel-clients/}{Sentinel client
spec}. The
SWI-Prolog client maintains the actual list of sentinels dynamically
after successful discovery of the first sentinel. Below is an example
\predref{redis_server}{3} to connect to a sentinel network. The \textit{Address} specification
\verb$sentinel(swish)$ tells the library we want to connect to a sentinel
network that is monitored under the name \const{swish}.

\begin{code}
:- redis_server(swish_sentinel, sentinel(swish),
                [ user(janbob),
                  password("topsecret"),
                  version(3),
                  sentinels([ host1:26379,
                                  host2:26379,
                                  host3:26379
                                ])
                ]).
\end{code}

\subsection{About versions}

\label{sec:redis-versions}

The current stable version of Redis is 6. Many Linux distros still ship
with version 5. Both talk protocol version 2. Version 6 also supports
protocol version 3. The main differences are:

\begin{itemize}
    \item The version 3 protocol has several improvements that notably
improvement passing large objects using a \textit{streaming} protocol.
    \item Hashes (maps) in the version 3 protocol are exchanged as lists
of \textit{pairs} (\verb$Name-Value$), while version 2 exchanges hashes as
a list of alternating names and values. This is visible to the
user. High level predicates such as \predref{redis_get_hash}{3} deal with
both representations.
    \item The version 3 protocol supports \textit{push messages} to deal with
\textit{monitor} and \textit{subscribe} events on the same connection as used
for handling normal requests.
\end{itemize}

New projects are encouraged to use Redis version 6 with the version 3
protocol. See \predref{redis_server}{3}.

\subsection{Redis as a message brokering system}

\label{sec:redis-brokering}

Starting with Redis 5, redis supports \textit{streams}. A stream is a list of
messages. Streams can be used as a reliable alternative to the older
Redis PUB/SUB (Publish Subscribe) mechanism that has no memory, i.e., if
a node is down when a message arrives the message is missed. In
addition, they can be used to have each message processed by a
\textit{consumer} that belongs to a \textit{consumer group}. Both facilities are
supported by \verb$library(redis_streams)$ (\secref{redisstreams})

Redis streams provide all the low-level primitives to realise message
brokering. Putting it all together is non-trivial though. Notably:

\begin{itemize}
    \item We must take care of messages that have been sent to some
consumer but the consumer fails to process the message and (thus)
ACK it is processed. This is handled by \predref{xlisten_group}{5} using
several options. Good defaults for these options are hard to
give as it depends on the required processing time for a message,
how common failures are and an acceptable delay time in case
of a failure, what to do in case of a persistent failure, etc.
    \item Streams are independent from consumer groups and acknowledged
messages remain in the stream. \predref{xstream_set}{3} can be used to
limit the length of the stream, discarding the oldest messages.
However, it is hard to give a sensible default. The required
queue length depends on the the size of the messages, whether
messages come in more or less randomly or in bursts (that cause
the stream to grow for a while), available memory, how bad it is
if some messages get lost, etc.
\end{itemize}

The directory \verb$doc/packages/examples/redis$ in the installation provides
an example using streams and consumer groups to realise one or more
clients connected to one or more compute nodes.

\subsection{History}

\label{sec:redis-history}

This module is based on the \verb$gpredis.pl$ by Sean Charles for GNU-Prolog.
This file greatly helped me understanding what had to be done, although,
eventually, not much of the original interface is left. The main
difference to the original client are:

\begin{itemize}
    \item Replies are not wrapped by type in a compound term.
    \item String replies use the SWI-Prolog string type.
    \item Values can be specified as \verb$Value as prolog$, after which they
are returns as a (copy of) Value. This prefixes the value
using "\Sneg{}u0000T\Sneg{}u0000".
    \item Strings are in UTF-8 encoding to support full Unicode.
    \item Using \predref{redis_server}{3}, actual connections are established
lazily and when a connection is lost it is automatically
restarted.
    \item This library allows for using the Redis publish/subscribe
interface. Messages are propagated using \predref{broadcast}{1}.
\end{itemize}