Copyright © Gary Houston 1993, 1994.

Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies.

Permission is granted to copy and distribute modified versions of this manual under the conditions for verbatim copying, provided that the entire resulting derived work is distributed under the terms of a permission notice identical to this one.
gf User's Manual

SGML is the ``Standard Generalized Markup Language'', defined by the International Organization for Standardization (ISO 8879:1986).  SGML is a framework for the construction of generalized markup languages.  A markup language is defined by its Document Type Definition (DTD), which provides all of the information needed for a complete SGML ``application'':  a formal document type declaration which provides the syntax of the language, together with the semantics expressed in some way (such as human-readable documentation).  An SGML document is created to conform to a given DTD, rather than to a particular computer system. A DTD can be implemented by many SGML systems:  structure editors, formatters, online browsers etc., which ideally do not specify the data format but provide some useful service by operating on it.

Running gf
**********

gf is an SGML system which converts SGML documents conforming to several DTDs into a number of output formats.  It can act as a formatter by converting the SGML markup into a language supported by a typesetting system.  The LaTeX converter gives the most complete results:  others will omit facilities such as the inclusion of PostScript graphics.

The command

     gf foo

will convert a document foo.sgml into the default output format, typically LaTeX. Other output formats can be specified using the -f option, e.g., gf -frtf foo.  The formats currently supported are latex2e, latex209, rtf, texinfo and a number of plain text formats:

  * ab for the ``best'' ASCII format, similar to the one described by Markus Kuhn (see the file iso2asc.txt available by ftp from ftp.uni-erlangen.de).  This can also be selected using ``ascii''.

  * as for ``symbolic'' ASCII, in which the ISO entity name is used for symbols not in the ASCII character set.

  * l1b for ``best'' representation using the latin-1 character set.

  * l1s for ``symbolic'' representation using the latin-1 character set.

gf will automatically determine which DTD the document is supposed to conform to (see Document Types for the list of supported DTDs).  Not every combination of DTD and output format is supported.

gf uses the sgmls program to parse the SGML document, but this is transparent unless an error occurs during parsing. An error indicates that a document does not conform to the model specified by its document type declaration.

As an aid to the user, gf has an option for the creation of new documents.  For example, to create a document named ``foo.sgml'' type:

gf -n dtd foo, where dtd can be general, html, spaper, smemo or sletter. See Templates for details on how the ``templates'' can be modified.

A number of additional options are available, for example

     gf -o foo.tex foo

will write the output to the file ``foo.tex'', or even

     gf -o xxx.tex foo

which makes it easy to erase all the garbage later.  Cleaning up garbage can occasionally be important, for example when formatting with different TeX macro packages:  they often use incompatible auxiliary file formats.

See the gf man page for the complete list of options. See Style Sheets for details on how the formatting process can be controlled through the use of style sheets.

The complexity of a typical DTD makes development of reliable SGML software difficult.  There is always the possibility that some untested combination of elements and styles will cause gf to fail.  In general, if a document parses correctly but gf generates incorrect output code (e.g., illegal LaTeX) then this should be considered a bug (see Bug Reports).

Document Types
**************

This section briefly describes each of the DTDs supported by gf.

general DTD
===========

The ``general'' DTD was one of the earliest document types. It was originally published by the ISO in Annex E of the 1986 SGML standard as a demonstration of a general purpose DTD for books or articles.  The date of publication (not surprisingly) appears to have preceded the development of reliable SGML parsers, since the document type declaration contains a minor error. gf includes a corrected version, with a superfluous ``address'' element removed from the ``titlep'' element.

gf can convert documents conforming to the DTD into LaTeX. Unfortunately I don't know of any freely available documentation for the DTD. A description is rumoured to have been published in ISO TR 9573, ``Techniques for using SGML''.  Some idea of the intent of the DTD can be guessed from the HTML and Snafu documentation.

There are deficiencies (or bugs) in the conversion.  The ones that I know of are:

  * Cross references don't always work (e.g., to preface).

  * The nl list is formatted as ul (I'm not sure what this is supposed to be).

  * Many attributes are ignored:

       * all attributes of general

       * the stitle attribute of title and h0, h1, h2

       * the compact attribute of gl, dl, and other lists

       * the termhi attribute of gl

       * the headhi, termhi and tsize attributes of dl

       * the source attribute of gd

       * the depth and keep attributes or xmp

       * the width attribute of fig

       * the see and seeid attributes of ix

       * sgmls displays warnings about the endtag minimalization of a couple of elements:  this is no cause for alarm.

Others probably exist.

HTML DTD
========

The HTML DTD is used in the World Wide Web project, which makes documents available across the Internet for interactive browsing.

gf can convert documents conforming to the HTML DTD into LaTeX, RTF and the plain output formats.  Since HTML documents are often stored without an SGML declaration or DOCTYPE declaration, gf will add them on if required.  However the <HTML> and </HTML> tags must still be supplied.

The HTML document type declaration included with gf is an IETF draft version dated 1993-07-01.  It defines a modified concrete reference syntax which allows direct entry of 8-bit Latin 1 characters. The output from gf will be 7-bit LaTeX, ASCII or RTF or 8-bit Latin-1.

Some modification to the DTD was required:  the amp entity was changed from "&amp;" to "&" and the lt entity from "&lt;" to "<", since the recursion is probably an error rather than a feature.  The type of the NAME attribute in the linkattributes entity was also changed from NMTOKEN to CDATA, since it needs to match HREF (NMTOKENs would be capitalised).  I have also ignored the requirement to honour multiple spaces in the source document:  this seems outdated.

Documentation on HTML can be obtained from the sources of WWW information, e.g., by ftp from info.cern.ch or by using WWW.  The document used for this implementation (draft-ietf-iiir-html-01.txt, by Tim Berners-Lee and Daniel Connolly) was obtained by ftp from ds.internic.net.  This is somewhat out of date, since the HTML 2.0 specification is now being finalised.

Currently the creation of printed documents from HTML can be unreliable.  The HTML DTD was designed to be used by network browsing software with relatively primitive parsing support.  In addition there is the problem that it is possible for a document on the network to be readable via WWW, but not conform to the HTML DTD (and be unparsable by sgmls).

Snafu DTD
=========

I created this DTD for my own use and experimentation.  Three types types of document are supported:  letters, memos and simple technical papers.  Several versions have been created: gf should be able to process all versions from 2 onwards. Documents conforming to the memo and letter variants can be converted to LaTeX, ASCII and RTF, and technical papers can also be converted to Texinfo.

See the paper ``The Snafu SGML Document Types'' for more information.

SGML Facilities
***************

This section describes a few constructions which may be useful in SGML documents.  Note that not all SGML features are described here: consult the standard for a complete discussion.

Comments
========

SGML commenting facilities can be used to put text into a source document which will not be visible to a processing system. Any text within the construction:

     <!--  text -->

will be removed when the document is parsed.  Within a comment, pairs of double hyphens ``--'' must be balanced (Why?  I don't know.).

Likewise, the construction <!> is ignored and can be used for various purposes.

Minimalisation Techniques
=========================

The full markup of a non-empty SGML element is of the form:

     <ELEMENT>element content</ELEMENT>

It will often be possible to omit the terminating tag:  this depends on how the element is defined in the DTD.  Sometimes the initial tag can also be omitted.

When the terminating tag is required, there are shorter forms which can be used:

     <ELEMENT/element content/
     <ELEMENT>element content</>

The symbol </> closes the most recently opened element.  Beware that it may cause confusion when the element contains hidden sub-elements, since it will be the sub-element which is closed.

Likewise, there is a construction <> which reopens the last opened element.  This can be just as confusing as </>.

When a number of tags appear in sequence the closing delimiters can be omitted, e.g., <li<it> is equivalent to <li><it>.  A similar case is </ul<ul>.

A more drastic minimalisation technique is the short reference, several of which will often be defined by a DTD. They allow a single character to be used to represent an entity of some kind. Common examples are the use of " as an abbreviation for a the beginning or end of a short quotation and a blank line to begin a new paragraph.

Short references can be context sensitive, by being activated only within a subset of the document.  An example would be a vertical bar which separates horizontal cells in a table but retains its normal ASCII code in other parts of the document.

Entity Declarations
===================

There are several ways in which declaring entities within a document can be useful.  Entities must be declared in the document type declaration at the beginning of the document.

The declaration associates a name with an object of some kind.  The default ``concrete syntax'' SGML rules for an entity name are restrictive:  it may contain no more than eight characters, it must start with a letter, and remaining characters may only be letters, digits and hyphens.  Letters are case sensitive. However many SGML applications loosen the restriction on name length through the use of a modified concrete syntax.

An example of an external entity declaration is:

     <!DOCTYPE spaper PUBLIC "-//Houston//DTD snafu5//EN"[
       <!ENTITY dogfig SYSTEM "dog.ps" NDATA EPSF>
     ]>

This declares an external file with notation EPSF, to be known within the SGML document as ``dogfig''.

Replacement Text
----------------

The simplest use of an entity is to provide replacement text.  This is useful for text which is repeated several times in the document or which is subject to frequent change.  To declare such an entity, use a line like:

     <!ENTITY response "<hp1>Are you sure that you want to
     do this &rarr; </hp1>">

The entity can then be referred to within the document with the entity reference:  &response;, which will generate:

Are you sure that you want to do this ->

External Entities
-----------------

Text can also be brought into the document from external files.  When the entity is declared, the content type can also be specified.  By default, the entity contains normal marked-up SGML text and is declared using a line such as:

     <!ENTITY junk SYSTEM "junk.sgml">

It is also possible to use the ``notation'' feature of SGML to incorporate external text which is in some non-SGML format.  For example, if a notation ASCII is defined (probably by the DTD), then markup such as:

     <!ENTITY realjunk SYSTEM "junk.c" NDATA ASCII>

could be used to include a program source file into the document.

Marked Sections
===============

Text within marked sections will be treated in a non-standard way by the SGML parser.  The text is inclosed within the construction:

     <![ KEYWORD [marked text]]>

where KEYWORD can be:

CDATA
     the marked text will be treated as ``character data'', which means that potential SGML element tags or entity references will be ignored.  This is useful when including verbatim program code (or SGML markup) in a document:  only the sequence ]]> can give problems.

RCDATA
     treat as ``replaceable character data''.  Element tags will be ignored but entity references will be resolved as normal.

IGNORE
     treat the marked text as a comment

INCLUDE
     treat the marked text as ordinary text

TEMP
     treat the marked text as ordinary text (which will presumably be removed later).

Special Characters
******************

Character References
====================

A character can be generated by typing a ``character reference'' with its decimal value, e.g., with the default character set &#38; will produce & and &#60; will produce <.

Character Entity Sets
=====================

Character entity sets can be used to obtain special symbols within an SGML document.  gf supports 6 entity sets (English versions in each case:  I know nothing about any others):

 1. ISO Added Latin 1

 2. ISO Added Latin 2

 3. ISO Diacritical Marks

 4. ISO Numeric and Special Graphic

 5. ISO Publishing

 6. ISO General Technical

See Figure 1 in Character Entity Sets to Figure 6 in Character Entity Sets for tables showing the characters available.  This is not the complete set of ISO character entity sets:  others include greek letters and AMS math symbols.  Note that the conversion to LaTeX does not support all of the symbols in the supported sets. (figure 1 omitted: ISO Added Latin 1 Character Entity Set) (figure 2 omitted: ISO Added Latin 2 Character Entity Set) (figure 3 omitted: ISO Diacritical Marks Character Entity Set. The marks can be applied to any character, in this case ``a''.  Note:  they are applied to the immediately preceding character.) (figure 4 omitted: ISO Publishing Character Entity Set) (figure 5 omitted: ISO Numeric and Special Graphic Character Entity Set. ``nbsp'' is a space where a line break is forbidden.  ``shy'' is a soft hyphen, which allows a linebreak with the insertion of a hyphen.) (figure 6 omitted: ISO General Technical Character Entity Set.  Math operators are classed as relations, binary operators, etc., and include implicit spacing.)

A symbol can be used by entering the appropriate entity reference, e.g., &mdash; will generate ``---'' when the document is formatted.

It is possible to modify the generated characters or define new characters if this is permitted for a given DTD:  see Control of Character Entity Translation.

Customisation
*************

There are three possibilities for user customisation (In addition to the obvious possibility of hacking the source code):  modification of the gf style sheets, modification of the templates used when creating new documents, and modification or supplementation of the character entity translation tables.

Style Sheets
============

A gf style sheet is a specialised SGML document containing information for the control of the formatting process for one or more of the supported output formats.  It is possible to format an SGML document in many different ways, for example by changing paper size, fonts or heading styles.  DTDs intended for pure ``content-based'' markup will attempt to avoid direct specification for this type of information within the source SGML documents.

By default gf will use a system-wide style sheet, possibly stored in the directory /usr/local/lib/sgml/gf, which will be named after the document DTD, e.g., general.style (use gf -v to check the file names and use environment variables as described in the gf man page to change them at run-time). If you want to modify the style sheet, you will need to make a local copy of it.  There are two ways to do this:  if you want the modified styles to apply to all of your documents, then copy the style sheet into your .gf directory.  If you do not have such a directory then you will need to create it.

     mkdir ~/.gf
     cp /usr/local/lib/sgml/gf/general.style  ~/.gf

Alternatively, if you only want the modified style to apply to a single document, you can place it in the same directory as the document and give it the same base name, e.g.,

     cp /usr/local/lib/sgml/gf/general.style  foo.style

A gf style sheet begins with an SGML declaration redefining two default values:  NAMELEN is increased to 34 and NAMECASE GENERAL is set to NO.

The body of the style sheet is broken into sections, each containing formatting information for a class of outputs.  The classes are:

  * tex, for LaTeX output.

  * plain, for ASCII and Latin-1 output.

  * rtf, for RTF output.

  * texinfo, for Texinfo output.

Each section has a number of style options which are relevant for that class of output formats.

gf will always attempt to read the style sheet from three locations, which by default are:  the system directory, the ~/.gf directory, and the directory containing the file.  Each time a style sheet is found, its contents are merged into a single database, overwriting data already present.  Therefore if some of the style options are deleted from a local style sheet, or if a new version of gf adds extra options, then the system versions will be used.

Documentation for the various style options can be obtained by running gf --style-help with the name of the document to be processed. This will print the options available for the particular DTD and output format.  E.g.,

     gf --style-help -fab gf.sgml

will print out the style options for the ``plain'' output formats when processing Snafu documents.

Templates
=========

The gf -n option can be used to simplify the creation of new SGML documents. It works by copying a ``template'' into the current directory.  Two locations are searched for the template:

  * a personal directory, by default named ~/.gf.

  * a system directory, by default /usr/local/lib/sgml/gf.

The name of the template file is constructed by appending a standard extension (normally .template) to the DTD type, e.g., smemo.template.

Templates files can therefore be customised by copying the system templates into the local ~/.gf directory and modifying as required.

Control of Character Entity Translation
=======================================

gf uses simple mapping files for the translation of character entity references in the source SGML document into the output format (In some difficult cases the mapping is currently modified by C code, e.g., for TeX \tt mode.).  The default versions of the mapping files are usually kept in the directory /usr/local/lib/sgml/gf and are named after the character entity set and the output format (e.g., ISOlat1.2tex).

Each file contains a line for each character in the entity set, with the entity name (the SDATA replacement text), a tab character and the replacement text.

Before reading the system file, gf will check for a file of the same name in the ~/.gf directory.  If this exists, it will be read instead of the system file.  This makes it possible to redefine the translations.

In addition to the mapping files above, gf will read any mapping file attached to the current document.  This can be used either to override translations in the system files or to define mappings for characters defined in the DTD.  For example, if processing the document foo.sgml which has the following line in the document type declaration:

     <!ENTITY BleECh SDATA "[BleECh]">

then the following foo.2ab file can be used to specify the ASCII mapping:

     [BleECh]	BleECh

To check which mapping files are read when the document is processed, use the -v option.

Bug Reports
***********

If you find a bug please send a description and short example to ghouston@actrix.gen.nz.