# Bootstrap for the protoc plugin

## Installing protobuf (on Ubuntu)

You can use the Ubuntu package `protobuf-compiler`, but it's dated
Jul 31, 2018.  Instead, you can clone from
https://github.com/protocolbuffers/protobuf and build using the
instructions in `protobuf/src/README.md`. For the `./configure`
command you may wish to use `./configure --prefix=$HOME/.local` and
`make -j4` (where "4" should be replaced by the number of cores on
your machine).

There are some additional notes on this in the `Makefile`.

TODO: These notes reflect an older stage of the bootstrap process.
Both the notes and the Makefile need to be cleaned up, to remove
stuff that's no longer needed.

## Overview of the original bootstrap process

The original bootstrap was done by using `protoc --decode` to get the
".wiredump" files (in `gen_pb/google/protobuf/**`).
These were then parsed by a simple DCG in
`parse_descriptor_proto_dump.pl` to produce the ".proto.parse"
files. The term expansion logic is in `descriptor_proto_expand.pl`,
which was then copied to `protoc-gen-swipl`.  At this point,
`gen_pb/google/protobuf/descriptor_pb.pl` and
`gen_pb/google/protobuf/compiler/plugin_pb.pl` could be
generated, and the original bootstrap code no longer needed.

The tests are fairly minimal. For proper testing, see directory `interop`.


## descriptor.proto and friends

Naming conventions:

| x.proto      | protobuf definition: used as input to protoc                         |
| x.wire       | wire-format encoding of data (e.g., produced by SerializeToString()) |
| x.proto.wire | `protoc --descriptor_set_out=x.proto.wire`                  |
| x_pb2.py     | generated by protoc from x.proto (for Python)                        |
| x.pb.{h,cc}  | generated by protoc from x.proto (for C++)                           |
| x.wiredump   | readable form of x.wire, using `protoc --decode` and the appropriate .proto file |
| x.wirerawdump | readable from of x.wire, using `protoc --decode_raw` (no .proto file) |
| x.segment    | Prolog term that contains a segmentation of the data in a .wire file (see also x.wiredump) |
| descriptor.proto.parse | Output from `parse_descriptor_proto_dump.pl`, which is hand-edited into `descriptor_proto.pl` (`descriptor_proto/1`).  |

`plugin.proto` and `descriptor.proto`  encode all `.proto` files created by
the protobuf compiler (`protoc`). Their meta-data have been bootstrapped into
`gen_pb/plugin_pb.pl` and
`gen_pb/descriptor_pb.pl`.

* `descriptor.proto.wire` contains a protobuf encoding of
  `descriptor.proto`; it was generated by
  `protoc --descriptor_set_out=descriptor.proto.wire`
* `descriptor.proto.wiredump` is generated by `protoc
  --decode=FileDescriptorSet descriptor.proto.wire`
  It is included here, to avoid a submodule dependency on
  git@github.com:protocolbuffers/protobuf.git
* `parse_descriptor_proto_dump.pl` is used to process `descriptor.proto.wiredump`
  into a Prolog term.
* `descriptor_proto.pl` is the term created by `parse_descriptor_proto_dump.pl`,
  in the clause `descriptor_proto/1`.
* `descriptor.proto.segment` is the "golden" output from rule `descriptor.segment`.
  TODO: delete this file.

TODO: use `protobuf\_segment\_message/2` to process
`descriptor.proto.wire` and any other `descriptor\_set\_out`, and then
generate Prolog code for easy handling of protobufs.

## protoc plugin

We can create a plugin by having an executable named `protoc-gen-swipl`
that is in the `PATH`, which reads a `CodeGeneratorRequest` from stdin
(defined in `src/protobuf/src/google/protobuf/compiler/plugin.proto`)
and outputs to stdout a `CodeGeneratorResponse` ... to get this
plugin, use `protoc --swipl_out=DIR`.

If you make an incompatible change to `protoc-gen-swipl`, you should
increment the version and also `protobufs:verify_version/0`.  (Also
`descriptor_proto.pl`, which is mostly obsolete.)

See also
[https://chromium.googlesource.com/external/github.com/protocolbuffers/protobuf/+/refs/heads/master/docs/implementing_proto3_presence.md](How To Implement Field Presence for Proto3).