\documentclass[11pt]{article}
\usepackage{times}
\usepackage{pl}
\usepackage{html}
\usepackage{plpage}
\sloppy
\makeindex

\onefile
\htmloutput{.}					% Output directory
\htmlmainfile{pldoc}				% Main document file
\bodycolor{white}				% Page colour

\renewcommand{\runningtitle}{SWI-Prolog documentation package}

\begin{document}

\title{SWI-Prolog Source Documentation \\
       Version 2}
\author{Jan Wielemaker \\
	VU University, Amsterdam \\
	The Netherlands \\
	E-mail: \email{J.Wielemaker@vu.nl}}

\maketitle

\begin{abstract}
This document presents PlDoc, the SWI-Prolog source-code documentation
infrastructure.  PlDoc is loosely based on JavaDoc, using structured
comments to mix documentation with source-code.  SWI-Prolog's PlDoc
is entirely written in Prolog and well integrated into the environment.
It can create HTML+CSS and \LaTeX{} documentation files as well as act
as a web-server for the loaded project during program development.

The SWI-Prolog website (\url{http:www.swi-prolog.org}) is written in
Prolog and integrates PlDoc to provide a comprehensive searchable
\href{http://www.swi-prolog.org/pldoc/index.html}{online manual}.

Version~2 of PlDoc extends the syntax with
\href{http://en.wikipedia.org/wiki/Markdown}{Markdown} markup as
specified by \href{http://www.stack.nl/~dimitri/doxygen/}{Doxygen}.
Based on experience with version~1, PlDoc~2 both tightens some rules
to avoid misinterpretations and relaxes others that were considered
too conservative.
\end{abstract}

\pagebreak
\tableofcontents

\vfill
\vfill

\newpage

\section{Introduction}
\label{sec:pldoc-intro}

When developing Prolog source that has to be maintained for a longer
period or is developed by a ---possibly distributed--- team some basic
quality mechanisms need to be adopted. A shared and well designed
codingstyle \cite{DBLP:dblpjournals/tplp/CovingtonBOWP12} is one of
them. In addition, documentation of source-files and their primary
interfaces as well as a testing framework must be established.

In our view, hitherto existing documentation and testing frameworks
fell short realising the basic needs in a lightweight and easy to
adopt system. To encourage consistent style, well commented code and
test-assisted development, we make sure that

\begin{shortlist}
    \item The documentation and testing framework requires a
	  minimum of work and learning.
    \item The framework is immediately rewarding to the individual
          programmer as well as the team,
\end{shortlist}

First, we describe the documentation system we developed for SWI-Prolog.
In \secref{motivation} we motivate our main choices.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Overview}
\label{sec:pldoc-overview}

Like JavaDoc, the PlDoc infrastructure is based on \emph{structured
comments}.  Using comments, no changes have to be made to Prolog to
load the documented source.  If the \pllib{pldoc} library is loaded,
Prolog will not only load the source, but also parse all structured
comments.  It processes the mode-declarations inside the comments and
stores these as annotations in the Prolog database to support the test
framework and other runtime and compiletime analysis tools that may be
developed in the future.

Documentation for all or some of the loaded files can be written to file
in either HTML+CSS or \LaTeX{} (see \secref{doclatex}) format. Each
source file is documented in a single file. In addition, the
documentation generator will generate an index file that can be used as
an index for a browser or input file for \LaTeX{} for producing nicely
typeset document.

To support the developer, the documentation system can be asked to start
a web-server that can be used to browse the documentation.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Structured comments}
\label{sec:pldoc-comments}

Structured comments come in two flavours, the line-comment (\%) based
one, seen mostly in the Prolog community and the block-comment
(\verb$/*$\ldots\verb$*/$) based one, commonly seen in the Java and C
domains.  As we cannot determine the argument names, type and modes
from following (predicate) source itself, we must supply this in the
comment.\footnote{See \secref{motivation:modes}.}  The overall
structure of the comment therefore is:

\begin{shortlist}
    \item Semi-formal type- and mode-description, see \secref{modes}
    \item Wiki-style documentation body, see \secref{wiki}
    \item JavaDoc style tags (\chr{@}keyword value, see \secref{tags})
\end{shortlist}

The \verb$/*$\ldots\verb$*/$ style comment starts with
\verb$/**$<white>. The type and mode declarations start at the first
non-blank line and are ended by a blank line.

The \chr{\%}-style line comments start with \verb$%!$<white> or, for
compatibility reasons, with \verb$%%$<white>.\footnote{The \texttt{\%\%}
leader was considered to give too many false positives on arbitrary
source code. It is still accepted, but invalid comments are silently
ignored, while invalid comments that start with \texttt{\%!} result in a
warning.} The type and mode declaration is ended by the first line that
starts with a single \%. E.g., the following two fragments are identical
wrt. PlDoc. Skipping blank-lines in \verb$/**$ comments allows to start
the comment on the second line.

\begin{code}
%!	predicate(-Arg:type) is nondet
%	Predicate ...
\end{code}

\begin{code}
/**
 * predicate(-Arg:type) is nondet
 *
 * Predicate ...
 */
\end{code}


The JavaDoc style keyword list starts at the first line starting
with @<word>.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{File (module) comments}
\label{sec:sectioncomments}

An important aspect is documentation of the file or module as a whole,
explaining its design, purpose and relation to other modules. In JavaDoc
this is the comment that preceeds the class definition. The Prolog
equivalent would be to put the module comment in front of the module
declaration. The module declaration itself however is an important index
to the content of the file and is therefore best kept first.

The general comment-structure for module comments is to use a type
identifier between angled brackets, followed by the title of the
section.  Currently the only type provided is \const{module}.  Other
types may be added later.

\subsection*{Example}

\begin{code}
/** <module> Prolog documentation processor

This module processes structured comments and generates both formal
mode declarations from them as well as documentation in the form of
HTML or LaTeX.

@author	Jan Wielemaker
@license GPL
*/
\end{code}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Type, mode and determinism declaration headers}
\label{sec:modes}

Many predicates can sensibly be called in different ways, e.g. with a
specific argument as input or as output. The header of the documentation
of a predicate consists of one or more \jargon{templates}, each
representing a specific way of calling the predicate.

A template can contain information about types, argument instantiation
patterns, determinism and more. The syntax is informally described below:

\begin{center}
\begin{tabular}{lrl}
\hline
<template>	\isa <head>['//'] 'is' <determinism> \\
		\ora <head>['//'] \\
<determinism>	\isa 'det' \\
		\ora 'semidet' \\
		\ora 'failure' \\
		\ora 'nondet' \\
		\ora 'multi' \\
		\ora 'undefined' \\
<head>		\isa <functor>'('<argspec> {',' <argspec>}')' \\
		\ora <functor> \\
<argspec>	\isa [<instantiation>]<argname>[':'<type>] \\
<instantiation> \isa '++' $\mid$ '+' $\mid$ '-' $\mid$ '--' $\mid$ '?'
		     $\mid$ ':' $\mid$ '@' $\mid$ '!' \\
<type>		\isa <term> \\
\hline
\end{tabular}
\end{center}

The \jargon{determinism} values originate from Mercury. Their meaning is
explained in the table below. Informally, \const{det} is used for
deterministic transformations (e.g.\ arithmetic), \const{semidet} for
tests, \const{nondet} and \const{multi} for \jargon{generators}.  The
\const{failure} indicator is rarely used.  It mostly appears in hooks
or the recovery goal of catch/3.

\begin{center}
\begin{tabular}{l|p{0.7\linewidth}}
\hline
\bf Determinism & \bf Predicate behaviour \\
\hline
\const{det}     & Succeeds exactly once without a choice point \\
\const{semidet} & Fails or Succeeds exactly once without a choice-point \\
\const{failure} & Always fails \\
\const{nondet}  & No constraints on the number of times the predicate
		  succeeds and whether or not it leaves choice-points
		  on the last success. \\
\const{multi}   & As \const{nondet}, but succeeds at least one time. \\
\const{undefined} & Well founded semantics third value.
		    See undefined/0. \\
\hline
\end{tabular}
\end{center}

The meanings of the \jargon{instantiation patterns} for individual
arguments are:

\begin{center}
\begin{tabular}{l|p{0.7\linewidth}}
\hline
++&	Argument is ground at call-time, i.e., the argument does not contain a
	variable anywhere. \\
+ &	Argument is fully instantiated at call-time, to a term that satisfies
	the type.  This is not necessarily \jargon{ground}, e.g., the term
	\exam{[_]} is a \jargon{list}, although its only member is
	unbound. \\
- &	Argument is an \emph{output} argument. It may be unbound at call-time,
	or it may be bound to a term. In the latter case, the predicate behaves
	as if the argument was unbound, and then unified with that term after
	the goal succeeds.
	For example, the goal \exam{findall(X, Goal, [T])} is good style and
	equivalent to \exam{findall(X, Goal, Xs), Xs = [T]}\footnote{The
	ISO standard dictates that \exam{findall(X, Goal, 1)} raises a
	\const{type_error} exception, breaking this semantic relation.
	SWI-Prolog does not follow the standard here.} Determinism
	declarations assume that the argument is a free variable at call-time.
	For the case where the argument is bound or involved in constraints,
	\const{det} effectively	becomes \const{semidet}, and \const{multi}
	effectively becomes \const{nondet}.\\
--&	Argument is unbound at call-time.  Typically used by predicates that
	create `something' and return a handle to the created object,
	such as open/3 which creates a \jargon{stream}. \\
? &	Argument is bound to a \emph{partial term} of the indicated
	type at call-time.  Note that a variable is a partial term for any
	type. \\
: &	Argument is a meta-argument.  Implies \chr{+}. \\
@ &	Argument will not be further instantiated than it is at call-time.
	Typically used for type tests. \\
! &	Argument contains a mutable structure that may be modified using
	setarg/3 or nb_setarg/3. \\
\hline
\end{tabular}
\end{center}

Users should be aware that calling a predicate with arguments instantiated
in a way other than specified by one of the templates may result in errors or
unexpected behavior.

Developers should ensure that predicates are \jargon{steadfast} with respect to
output arguments (marked - in the template). This means that instantiation of
output arguments at call-time does not change the semantics of the goal (it may
be used for optimization, though). If this steadfast behavior cannot be
guaranteed, -- should be used instead.

In the current version, argument \jargon{types} are represented by an arbitrary term
without formal semantics. In future versions we may adopt a formal type
system that allows for runtime verification and static type analysis
\cite{DBLP:conf/cl/Hermenegildo00,DBLP:journals/ai/MycroftO84,DBLP:conf/acsc/JefferyHS00}


\subsection*{Examples}

\begin{code}
%!	length(+List:list, -Length:int) is det.
%!	length(?List:list, -Length:int) is nondet.
%!	length(?List:list, +Length:int) is det.
%
%	True if List is a list of length Length.
%
%	@compat iso
\end{code}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Tags: \chr{@}see, etc.}
\label{sec:tags}

Optionally, the description may be followed by one or more \emph{tags}.
Our tag convention is strongly based on the conventions used by javaDoc.
It is adviced to place tags in the order they are described below.

\begin{description}
    \definition{\chr{@}arg {\it Name Description}}
Defines the predicate arguments. Each argument has its own \const{@arg}
tag. The first word is the name of the argument. The remainder of the
tag is the description. Arguments declarations normally appear in order
used by the predicate.

    \definition{\chr{@}param {\it Name Description}}
This is a synonym for \const{@arg}, using the JavaDoc tag name.

    \definition{\chr{@}throws {\it Term Description}}
Error condition. First Prolog term is the error term. Remainder is the
description.

    \definition{\chr{@}error  {\it Error Description}}
As \chr{@}throws, but the exception is embedded in \term{error}{Error,
Context}.

    \definition{\chr{@}author {\it Name}}
Author of the module or predicate. Multiple entries are used if there
are multiple authors.

    \definition{\chr{@}version {\it Version}}
Version of the module.  There is no formal versioning system.

    \definition{\chr{@}see {\it Text}}
Point to related material.  Often contains links to predicates or
files.

    \definition{\chr{@}deprecated {\it Alternative}}
The predicate or module is deprecated.  The description specifies what
to use in new code.

    \definition{\chr{@}compat {\it Standards and systems}}
When implementing libraries or externally defined interfaces this tag
describes to which standard the interface is compatible.

    \definition{\chr{@}copyright {\it Copyright holder}}
Copyright notice.

    \definition{\chr{@}license {\it License conditions}}
License conditions that apply to the source.

    \definition{\chr{@}bug {\it Bug description}}
Known problems with the interface or implementation.

    \definition{\chr{@}tbd {\it Work to be done}}
Not yet realised behaviour that is enticipated in future versions.
\end{description}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Wiki notation}
\label{sec:wiki}

Structured comments that provide part of the documentation are written
in Wiki notation, based on \href{http://www.twiki.org}{TWiki}, with some
Prolog specific additions.

\subsection{Structuring conventions}
\label{sec:pldoc-comment-structure}

\begin{description}
    \item [Paragraphs]
Paragraphs are separated by a blank line. Paragraphs that are indented
in the source-code \emph{after} normalising the left-margin with at
least 16 spaces are \emph{centered}. Paragraphs where all the lines
start with ``\verb$> $'' (greater than followed by a blank) are rendered
in the HTML backend using a \const{blockquote} element and in \LaTeX{}
using the \const{quote} environment.

    \item [General lists]
The wiki knows three types of lists: \emph{bullet lists} (HTML
\const{ul}), \emph{numbered lists} (HTML \const{ol}) and
\emph{description lists} (HTML \const{dl}). Each list environment is
headed by an empty line and each list-item has a special symbol at the
start, followed by a space.  Each subsequent item must be indented at
exactly the same column.  Lists may be nested by starting a new list at
a higher level of indentation.  The list prefixes are:

\begin{center}
\begin{tabular}{lp{0.7\linewidth}}
 \tt *	& Bulleted list item \\
 \tt 1.	& Numbered list item.  Any number from 1..9 is allowed, which
	  allows for proper numbering in the source.  Actual numbers
	  in the HTML or \LaTeX{} however are re-generated, starting
	  at 1. \\
 \tt \$ Title : Item & Description list item.
\end{tabular}
\end{center}

    \item [Term lists]
Especially when describing option lists or different accepted types,
it is common to describe the behaviour on different terms.  Such
lists must be written as below.  <Term1>, etc. must be valid Prolog
terms and end in the newline.  The Wiki adds \verb$' . '$ to the text
and reads it using the operator definitions also used to read
the mode terms.  See \secref{modes}.  Variable names encountered in
the \arg{Term} are used for indentifying variables in the following
\arg{Description}.  At least one \arg{Description} must be non-empty
to avoid confusion with a simple item list.

\begin{code}
   * Term1
     Description
   * Term2
     Description
\end{code}

    \item [Predicate description lists]
Especially for processing Wiki files, the Wiki notation allows for
including the description of a predicate `in-line', where the
documentation is extracted from a loaded source file. For example:

\begin{code}
The following predicates are considered Prolog's prime list processing
primitives:

    * [[member/2]]
    * [[append/3]]
\end{code}

    \item [Tables]
The Wiki provides only for limited support for tables.   A table-row
is started by a \chr{|} sign and the cells are separated by the same
character.  The last cell must be ended with \chr{|}.  Multiple lines
that parse into a table-row together form a table.  Example:

\begin{code}
	| Algorithm    | Time (sec) |
	| Depth first  | 1.0        |
	| Breath first | 0.7        |
	| A*           | 0.3        |
\end{code}

    \item [Section Headers]
Section headers are creates using one of the constructs below taken
from TWiki.  Section headers are normally not used in the source-code,
but can be useful inside README and TODO files.  See \secref{dirindex}.

\begin{code}
---+ Section level 1
---++ Section level 2
---+++ Section level 3
---++++ Section level 4
\end{code}

In addition, PlDoc recognises the \jargon{markdown} syntax, including
named sections as defined by \program{doxygen}. A section is named
(labeled) using an optional sequence \verb${\#$\textit{name}\verb$}$.
The three code sections below provide examples. Note that \texttt{\#}
section headers should be positioned at the left margin and the
\texttt{\#} must be followed by blank space. If the header is
underlined, the underline is a line that only contains \const{=} or
\const{-} characters. There must be a minimum of three\footnote{Markdown
demands two, but this results in ambiguities with the \texttt{==} fence
for code blocks.} of such characters.


\begin{code}
Section level 1
===============

Section level 2
---------------
\end{code}

\begin{code}
# Section level 1
## Section level 2
### Section level 3
#### Section level 4
\end{code}

\begin{code}
Section level 1		{#label}
===============

# Section level 1	{#label}
\end{code}

    \item [Code blocks]
There are two ways to write a code block. The first one is
\jargon{fenced}. Here, the block is preceeded and followed by a fence
line.  The traditional PlDoc fence line is \verb$==$.  Doxygen fence
lines are also accepted.  They contain at least three tilde (\chr{~})
characters, where the opening fence line may be followed by a file
extension between curly brackets.  In all cases, the code is indented
relative to the indentation of the fence line.  Below are two examples,
the first being the traditional PlDoc style.  The second is the Doxygen
style, showing a code block that is indented (because it is a body
fragment) and that is flagged as Prolog source. Note that the
\verb${.pl}$ is optional.

\begin{code}
  ==
  small(X) :-
     X < 2.
  ==
\end{code}

\begin{code}
  ~~~{.pl}
    ...,
    format('Hello ~w~n', [World]),
    ...,
  ~~~
\end{code}

The second form of code blocks are \jargon{indented blocks}. Such a
block must be indented between 4 and 8 characters, relative to the
indentation of the last preceeding non-blank line. The block is opened
with a blank line and closed by a blank line or a line that is indented
less than the indentation of the initial line. It is allowed to have a
single blank line in the middle of a code block, provided that the next
line is again indented at least as much as the initial line. The initial
line as well as a line that follows a blank line may not be a valid list
opening line or a table row, i.e., it may not start with one of
\verb$*-$ followed by a space or \verb$|$.

    \item [Rulers]
PlDoc accepts both the original PlDoc and markdown conventions for
rulers. A PlDoc ruler is a line with at least two dashes (-) that starts
at the left-most column.  A markdown ruler holds at least three ruler
characters and any number of spaces.  The ruler characters are the
dash (-), underscore (\verb$_$) or asterisk (\verb$*$).  Below are
three examples, the last two of which are valid markdown.

\begin{code}
--
***
		- - -
\end{code}

    \item [Line breaks]
A line break may be added by \emph{ending} the physical line with the
HTML linebreak, \verb$<br>$ or \verb$<br/>$.\footnote{The markdown
conventions are (original) two spaces at the of the physical line and
(GitHub) a physical line break. Neither fit well with source code.
Doxygen supports restricted HTML and allows for
\texttt{\string<br\string>}.}
\end{description}

\subsection{Text markup: fonts and links}
\label{sec:pldoc-markup}

\subsubsection{Emphasizing text}
\label{sec:emph}

Text emphasis is a combination of old plaintext conventions in Usenet
and E-mail and the doxygen version of markdown. \Tabref{fonts} shows the
font-changing constructions.  The phrase \jargon{limited context} means
that

\begin{itemize}
    \item The opening \chr{*} or \chr{_} must be preceeded by white space
          or a character from the set \verb$<{([,:;$ and must be
	  followed by an alphanumerical character.
    \item The closing \chr{*} or \chr{_} may not be followed by an
	  alphanumerical character and may not be preceeded by white
	  space or a character from the set \verb$({[<=+-\@$.
    \item The scope of these operations is always limited to the
	  identified structure (paragraph, list item, etc.)
\end{itemize}

Note that \verb$=$<identifier>\verb$=$ is limited to a an
\jargon{identifier}, such as a file name, XML name, etc. Identifiers
must start and end with an alphanumerical character, while characters
from the set \verb$.-/:$ may appear internally. Note that this set
explicitly does not allow for white space in code spans delimited by a
single \const{=}. This markup is specifically meant to deal with code
that is either not Prolog code or invalid Prolog code. Valid Prolog code
should use the backtick as described in \secref{inlinecode}.

\begin{table}
\begin{center}
\begin{tabular}{|l|p{0.7\linewidth}|}
\hline
\tt *bold*	& Typeset text in \textbf{bold} for
		  limited content (see running text). \\
\tt *|bold|*	& Typeset text in \textbf{bold}.
		  Content can be long. \\
\tt _emphasize_ & Typeset text as \emph{emphasize} for
		  limited content (see running text). \\
\tt _|emphasize|_ & Typeset text as \emph{emphasize}.
		  Content can be long. \\
\tt =code=      & Typeset text \texttt{fixed} font for
		  identifiers (see running text). \\
\tt =|code|=    & Typeset text \texttt{fixed} font.
		  Content can be long. \\
\tt Word	& Capitalised words that appear as argument-name
		  are written in \arg{Italic} \\
\hline
\end{tabular}
\end{center}
    \caption{Wiki constructs to change the font}
    \label{tab:fonts}
\end{table}

\subsubsection{Inline code}
\label{sec:inlinecode}

Inline code can be realised using the \verb$=$ switch described in
\secref{emph} or the markdown backtick. In addition, it can use the
mardown/Doxygen \jargon{backtick} (\verb$`$) convention: a string
that is delimited by backticks is considered code, provided:

    \begin{itemize}
    \item An internal double backtick is translated into a single
          backtick.
    \item Inline code is limited to the current structure (paragraph,
          table cell, list item, etc.
    \item The content of the code block is valid Prolog syntax.
	  Note that in Doxygen, the syntax is not validated and
	  a single quote cancels the recognition as code.  The
	  latter is a problematic in Prolog because single
	  quotes are often required.
    \end{itemize}

Currently, `Var` is typeset as a variable (italics) and other terms
are typeset using a fixed-width code font.

In addition, compound terms in canonical notation (i.e.,
\mbox{\textit{functor}\texttt{(},\textit{...args...}\texttt{)}} that can
be parsed are first verified as a file-specification for
absolute_file_name/3 and otherwise rendered as \jargon{code}.


\subsubsection{Links}
\label{sec:pldoc-links}

\Tabref{links} shows the constructs for creating links.

\begin{table}
\begin{center}
\begin{tabular}{|l|p{0.7\linewidth}|}
\hline
\tt name/arity  & Create a link to a predicate \\
\verb$`name/arity`$  & Create a link to a predicate \\
\verb$``name/arity``$  & Predicate indicator that does not create a
			 link \\
\tt name//arity & Create a link to a DCG rule \\
\verb$`name//arity`$ & Create a link to a DCG rule \\
\verb$``name//arity``$ & DCG indicator that does not create a
			 link \\
flag \verb$`name`$ & Create a link to the Prolog flag \arg{name}
		  if this flag exists. \\
\tt name.ext	& If <name>.<ext> is the name of an existing file
		  and <ext> is one of \fileext{pl}, \fileext{txt},
		  \fileext{md}, \fileext{png}, \fileext{gif}, \fileext{jpeg},
		  \fileext{jpg} or \fileext{svg}, create a link to the file. \\
\tt prot\verb$://$url & If <prot> is one of \const{http}, \const{https}
		  or \const{ftp}, create a link. \\
\tt\chr{<}url\chr{>} & Create a hyperlink to URL.  This construct
		  supports the expand_url_path/2 using the construct
		  <alias>:<local>.  <local> can be empty. \\
\tt\verb$[[$label\verb$][$link\verb$]]$ &
		  Create a link using the given <label>.  Label
		  can be text or a reference to an image file.
		  Additional arguments can be supplied as
		  \chr{;}<name>=\chr{"}<value>\chr{"}.  More
		  arguments are separated by commas. <link> must be a
		  filename as above or a url. \\
\tt\verb$[$label\verb$]($link\verb$)$ &
		  The markdown version of the above. \\
\tt\verb$[$@cite\verb$]$ &
		  Include \LaTeX{} citation.  Multiple citations can
		  be saparated using \chr{;}. No white space is allowed.
		  \\
\hline
\end{tabular}
\end{center}
    \caption{Wiki constructs that create links}
    \label{tab:links}
\end{table}



\subsection{Images}
\label{sec:pldoc-images}

Images can be included in the documentation by referencing an image file
using one of the extensions \fileext{gif}, \fileext{png},
\fileext{jpeg}, \fileext{jpg} or \fileext{svg}.\footnote{SVG images are
included using the \const{object} element. This is supported by many
modern browsers. When using IE, one needs at least IE9.} By default this
creates a link to the image file that must be visited to see the image.
Inline images can be created by enclosing the filename in double square
brackets. For example

\begin{code}
The [[open.png]] icon is used open an existing file.
\end{code}

The markdown alternative for images is also supported, and looks as
below.  The current implementation only deals with image \emph{files},
not external resources.

\begin{code}
    ![Caption](File)
\end{code}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Directory indices}
\label{sec:dirindex}

A directory index consists of the contents of the file \file{README}
(or \file{README.TXT}), followed by a table holding all currently loaded
source-files that appear below the given directory (i.e.\ traversal is
\emph{recursive}) and for each file a list of public predicates and their
descriptive summary.  Finally, if a file \file{TODO} or \file{TODO.TXT}
exists, its content is added at the end of the directory index.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Documentation files}
\label{sec:wikifiles}

Sometimes it is desirable to document aspects of a package outside the
source-files.  For this reason the system creates a link to files using
the extension \fileext{txt}. The referenced file is processed as Wiki
source. The two fragments below illustrate the relation between an
\fileext{pl} file and a \fileext{txt} file.

\begin{code}
%!	read_setup(+File, -Setup) is det.
%
%	Read application setup information from File.  The details
%	on setup are described in setup.txt.
\end{code}

\begin{code}
---+ Application setup data

If a file =|.myapprc|= exists in the user's home directory the
application will process this data using setup.pl. ...
\end{code}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Running the documentation system}
\label{sec:running}


\subsection{During development}
\label{sec:pldoc-development}

To support the developer with an up-to-date version of the documentation
of both the application under development and the system libraries the
developer can start an HTTP documentation server using the command
\term{doc_server}{?Port}.  A good way to deploy PlDoc for program
development is to write a file called e.g., \file{debug.pl} that
sets up the preferred development environment and loads your program.
below is an example \file{debug.pl} that starts PlDoc and prints strings
as text before loading the remainder of your program.

\begin{code}
:- doc_server(4000).	% Start PlDoc at port 4000
:- portray_text(true).	% Enable portray of strings

:- [load].		% load your program
\end{code}

\begin{description}
    \predicate{doc_collect}{1}{+Bool}
Enable/disable collecting structured comments into the Prolog database.
See doc_server/1 or doc_browser/0 for the normal way to deploy the
server in your application. All these predicates must be called
\emph{before} loading your program.

    \predicate{doc_server}{1}{?Port}
Start documentation server at \arg{Port}.  Same as
\term{doc_server}{Port, [allow(localhost), workers(1)]}.  This predicate
must be called \emph{before} loading the program for which you consult
the documentation. It calls doc_collect/1 to start collecting
documentation while (re-)loading your program.

    \predicate{doc_server}{2}{?Port, +Options}
Start documentation server at \arg{Port} using \arg{Options}.  Provided
options are:

    \begin{description}
	\termitem{root}{+Path}
Defines the root of all locations served by the HTTP server.  Default
is \const{/}.  \arg{Path} must be an absolute URL location, starting
with \chr{/} and ending in \chr{/}.  Intented for public services behind
a reverse proxy.  See documentation of the HTTP package for details on
using reverse proxies.

	\termitem{edit}{+Bool}
If \const{false}, do not allow editing, even if the connection comes
from localhost. Intended together with the \const{root} option to make
pldoc available from behind a reverse proxy. See the HTTP package for
configuring a Prolog server behind an
\href{http://httpd.apache.org/docs/1.3/mod/mod_proxy.html}{Apache
reverse proxy}.

	\termitem{allow}{+HostOrIP}
Allow connections from \arg{HostOrIP}.  If \arg{Host} is an atom
starting with a '.', suffix matching is preformed.  I.e.\
\verb$allow('.uva.nl')$ grants access to all machines in this domain.
IP addresses are specified using the library(socket) ip/4 term.  I.e.
to allow access from the 10.0.0.X domain, specify
\verb$allow(ip(10,0,0,_))$.

	\termitem{deny}{+HostOrIP}
Deny access from the given location.  Matching is equal to the
\const{allow} option.
    \end{description}

Access is granted iff

\begin{shortlist}
    \item Both \emph{deny} and \emph{allow} match
    \item \emph{allow} exists and matches
    \item \emph{allow} does not exist and \emph{deny} does not match.
\end{shortlist}

    \predicate{doc_browser}{0}{}
Open the user's default browser on the running documentation server.
Fails if no server is running.

    \predicate{doc_browser}{1}{+Spec}
Open the user's default browser on the specified page.  \arg{Spec} is
handled similar to edit/1, resolving anything that relates somehow to
the given specification and ask the user to select.%
	\bug{This flexibility is not yet implemented}.
\end{description}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{As a manual server}
\label{sec:manserver}

The library \pllib{pldoc/doc_library} defines doc_load_library/0 to
load the entire library.

\begin{description}
    \predicate{doc_load_library}{0}{}
Load all library files.  This is intended to set up a local documentation
server.  A typical scenario, making the server available at port 4000 of
the hosting machine from all locations in a domain is given below.

\begin{code}
:- doc_server(4000,
	      [ allow('.my.org')
	      ]).
:- use_module(library(pldoc/doc_library)).
:- doc_load_library.
\end{code}

Example code can be found in \file{$PLBASE/doc/packages/examples/pldoc}.
\end{description}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Using the browser interface}
\label{sec:browser}

The documentation system is normally accessed from a web-browser after
starting the server using doc_server/1.  This section briefly explains
the user-interface provided from the browser.

\subsubsection{Searching}
\label{sec:pldoc-search}

The top-right of the screen provides a search-form. The search string
typed is searched as a substring and case-insensitive. Multiple strings
separated by spaces search for the intersection. Searching for objects
that do not contain a string is written as \chr{-}<string>.  A search
for adjacent strings is specified as \chr{"}<string>\chr{"}.  Here
are some examples:

\begin{center}
\begin{tabular}{|l|p{0.6\linewidth}|}
\hline
\tt load file	& Searches for all objects with the strings
		  \texttt{load} and \texttt{file}. \\
\tt load -file  & Searches for objects with \texttt{load}, but
		  \emph{without} \texttt{file}. \\
\tt "load file" & Searches for the string \texttt{load file}.\\
\hline
\end{tabular}
\end{center}

The two radio-buttons below the search box can be used to limit the
search. \textsf{All} searches both the application and manuals.
Searching for \textsf{Summary} also implies \textsf{Name}.

\subsubsection{Views}
\label{sec:pldoc-views}

The web-browser supports several views, which we briefly summarise
here:

\begin{itemlist}
    \item [ Directory ]
In directory-view mode, the contents of a directory holding Prolog
source-files is shown file-by-file in a summary-table.   In addition,
the contents of the \file{README} and \file{TODO} files is given.

    \item [ Source File ]
When showing a Prolog source-file it displays the module documentation
from the \verb$/** <module ... */$ comment and the public predicates
with their full documentation. Using the \textsf{zoom} button the user
can select to view both public and documentated private predicates.
Using the \textsf{source} button, the system shows the source
with syntax highlighting as in PceEmacs and formatted structured
comments.%
	\footnote{This mode is still incomplete.  It would be nice to
		  add line-numbers and links to documentation and
		  definitions in the sources.}

    \item [ Predicate ]
When selecting a predicate link the system presents a page with the
documentation of the predicate.  The navigation bar allows switching
to the Source File if the documentation comes from source or the
containing section if the documentation comes from a manual.

    \item [ Section ]
Section from the manual. The navigation bars allows viewing the
enclosing section (\emph{Up}).
\end{itemlist}


\subsubsection{Editing}
\label{sec:pldoc-editing}

If the browser is accessed from \const{localhost}, each object that
is related to a known source-location has an edit icon at the right
side.  Clicking this calls edit/1 on the object, calling the user's
default editor in the file.  To use the built-in PceEmacs editor,
either set the Prolog flag \const{editor} to \const{pce_emacs} or
run \exam{?- emacs.} before clicking an edit button.

Prolog source-files have a \emph{reload} button attached. Clicking this
reloads the source file if it was modified and refreshes the page. This
supports a comfortable edit-view loop to maintain the source-code
documentation.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{docfiles.tex}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Including PlDoc in a LaTeX document
% Eating our own docfood
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\input{latex.tex}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Motivation of choices}
\label{sec:motivation}

Literate programming is an established field.  The \TeX{} source is one
of the first and best known examples of this approach, where input
files are a mixture of \TeX{} and Pascal source. External tools are
used to untangle the common source and process one branch to produce
the documentation, while the other is compiled to produce the program.

A program and its documentation consists of various different parts:

\begin{shortlist}
    \item The program text itself.  This is the minimum that must be
          handed to the compiler to create an executable (module).
    \item Meta information about the program: author, modifications,
          license, etc.
    \item Documentation about the overall structure and purpose of
	  the source.
    \item Description of the interface: public predicates, their
	  types, modes and whether or not they are deterministic as
	  wel as an informative text on each public predicate.
    \item Description of key private predicates necessary to understand
	  how the public interface is realised.
\end{shortlist}

\subsection*{Structured comments or directives}

Comments can be added through Prolog directives, a route taken by Ciao
Prolog with lpdoc \cite{DBLP:conf/cl/Hermenegildo00} and Logtalk
\cite{pmoura03}. We feel structured comments are a better alternative
for the following reasons:

\begin{itemize}
    \item Prolog programmers are used to writing comments as Prolog
          comments.
    \item Using Prolog strings requires unnatural escape sequences for
	  string quotes and long literal values tend to result in hard
	  to find quote-mismatches. Python uses comments in long
	  strings, solving this problem by using three double quotes
	  to open and close long comments.
    \item Comments should not look like code, as that makes it more
	  difficult to find the actual code.
\end{itemize}

We are aware that the above problems can be dealt with using
syntax-aware editors. Only a few editors are sufficiently powerful to
support this correctly, though, and we do not expect the required advanced
modes to be widely available.  If comments are used, we do not need to force
users into using a particular editor.

\subsection*{Wiki or HTML}

JavaDoc uses HTML as markup inside the structured comments. Although
HTML is more widely known than ---for example--- \LaTeX{} or TeXinfo, we
think the Wiki approach is sufficiently widely known today. Using text
with minimal layout conventions taken largely from plaintext newsnet and
E-mail, Wiki input is much easier to read in the source-file than HTML
without syntax support from an editor.

\subsection*{Types and modes}
\label{sec:motivation:modes}

Types and modes are not a formal part of the Prolog language.
Nevertheless, their role goes beyond pure documentation. The
test-system can use information about non-determinism to validate that
deterministic calls are indeed deterministic.  Type information can be
used to analyse coverage from the test-suite, to generate runtime type
verification or to perform static type-analysis.  We have chosen to
use a structured comment with formal syntax for the following reasons:

\begin{itemize}
    \item As comments, they stay together with the comment block
          of a predicate. We feel it is best to keep documentation
	  as close as possible to the source.
    \item As we parse them separately, we can pick up argument names
	  and create a readable syntax without introducing possibly
	  conflicting operators.
    \item As comments, they do not introduce incompatibilities with
          other Prolog systems.
\end{itemize}


\subsection*{Few requirements}

SWI-Prolog aims at platform independence. We want tools to rely as
much as possible on Prolog itself. Therefore, the entire infrastructure
is written in Prolog. Output as HTML is suitable for browsing and not
very high quality printing on virtually all platforms. Output to
\LaTeX{} requires more infrastructure for processing and allows for
producing high-quality PDF documents.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Compatibility and standards}
\label{sec:pldoc-compatibility}

Initially, the PlDoc wiki language was based on
\href{http://www.twiki.org}{Twiki}. Currently,
\href{http://en.wikipedia.org/wiki/Markdown}{markdown} is a wiki syntax
that is widely accepted and not tied to a single system.  In PlDoc~2,
we have adopted markdown, including many of the limitations and
extensions introduced by
\href{http://www.stack.nl/~dimitri/doxygen/}{Doxygen}.  Limitations are
needed to avoid ambiguities due to the common use of symbol charaters
in programming languages. Extensions are desirable to make use of
already existing conventions and to support requirements of program
documentation.

Some of the changes in PlDoc~2 are to achieve compatibility with the
\href{http://prolog-commons.org/}{Prolog Commons} project. The library
documentation conventions of this project will be based on PlDoc and the
Ciao lpdoc standards.  It is likely that there will be more changes to
the PlDoc format to synchronise with Commons.  We do not anticipate
significant impact on existing documentation.


\bibliographystyle{name}
\bibliography{pl}

\printindex

\end{document}