Notes about SP version 1.3.4

Last update: [18-Mar-2000]                Valid HTML 3.2!

Introduction

Writing HTML files without a validating parser is like trying to write computer programs without a compiler: don't do it! Fortunately, help is readily available on the Internet.

James Clark <jjc@jclark.com> is developing a new implementation of a suite of SGML parser tools, called SP. These include:

nsgmls
-- an sgmls-compatible validating SGML parser.
spam
-- an SGML markup stream editor. This program is capable of supplying missing end tags, expanding abbreviated tags, and also of filling in all known attributes in every begin tag. The latter can be handy if you want to find what attributes are available, without having to read the HTML grammar.
sgmlnorm
-- a simple SGML tag normalizer.
spent
-- print SGML entity on the standard output
sx
-- translate SGML (and HTML) to XML
WARNING: The name sx conflicts with an older program distributed as part of the sx, sb, sz - XMODEM, YMODEM, ZMODEM file send package. If you have those older programs installed, you will have to rename one or the other of them. At my site, since most users now employ either PPP or SLIP for terminal connections, the need for those old modem programs has diminished, and in any event, the Columbia University kermit system provides a superior alternative to the [XYZ]MODEM package for file transfer. I therefore renamed the old sx to sx.old.

Besides being a complete redesign of the earlier successful smgls implementation, the new programs are designed for the future: they support extended character sets, such as Unicode, and various multi-byte encodings used in oriental languages.

For a textbook treatment of the use of the SP system, see the useful book

@String{pub-PH                  = "Pren{\-}tice-Hall"}

@String{pub-PH:adr              = "Englewood Cliffs, NJ 07632, USA"}

@Book{McGrath:1997:SS,
  author =       "Sean McGrath",
  title =        "ParseMe.1st: {SGML} for Software Developers",
  publisher =    pub-PH,
  address =      pub-PH:adr,
  pages =        "xxiii + 341",
  month =        jan,
  year =         "1998",
  ISBN =         "0-13-488967-3",
  LCCN =         "QA76.76.H94M388 1998",
  bibdate =      "Sat Jan 4 12:20:44 MST 1997",
  price =        "US\$33.75",
  acknowledgement = ack-nhfb,
  keywords =     "SGML (document markup language)",
  xxprice =      "US\$55.00",
}

Some details

The new code is written almost entirely in C++ (almost 78K lines at version 1.3.4, or nearly four times the size of Don Knuth's TeX or Metafont), and requires template support, a relatively new feature of C++ which is not yet widely available. [An ANSI/ISO Standard for C++, ISO/IEC 14882:1998 Programming languages -- C++, was finally adopted in 1998, and by mid-1999, a few UNIX vendors claimed conformanance to that Standard.]

WARNING: To build these programs, you will need about 60MB of disk space if you add the -g compiler debug option. Adding -g increases executable sizes about 10MB each from about 2.5MB (on a Sun SPARC Solaris 2.6 system). Without that option, about 20MB of disk space suffices.

The SP programs can be compiled and built using recent releases of GNU g++ and libg++ (2.7.1 or later). g++ itself is built as part of the GNU gcc compiler installation; although that installation takes a few hours, and requires about 120MB of disk space to be able to run the validation tests before installation, it is straightforward, and should be problem free on most current UNIX systems. The GNU compiler suite has also been built on IBM PC MS DOS and Compaq/DEC OpenVMS systems, although those versions usually lag behind.

The SP distribution site has binaries for SP version 1.3.4 for IBM PC DOS, and Windows 95 and Windows NT.

Binaries for older versions are available for Intel 386 Linux, Sun Solaris 2.5, and DEC Alpha OSF/1 3.2.

Just as with sgmls, lengthy command lines are needed to run these programs successfully. To facilitate their use, I've prepared simple UNIX shell scripts html-ncheck and html-spam to hide the complexity, so that only the HTML files need to be provided on the script command lines.

If you have installed the html-check distribution, and you want to use html-spam, you need to add to end of the HTML catalog file, /usr/local/lib/html-check/lib/catalog. these lines:

        -- Added at the suggestion of James Clark <jjc@jclark.com> --
        -- so that spam -p doesn't output the contents of html.decl --
SGMLDECL html.decl

They have already been added to the distributed catalog file, but you might already have such a file installed that needs the update.

Without this change, the contents of html.decl are copied to the output if the -p is included in the spam invocation in html-spam ; omitting -p and including html.decl doesn't help, because the <!DOCTYPE ... > line is then lost.

Installation report

I have successfully built and install sp-1.3.4 with g++ (gcc 2.95.2 [25-Oct-1999] on these systems:

using the command

make && make install prefix=/usr/local

On a few of these, minor problems cropped up and were solved; they are discussed further below.

I also made unsuccessful attempts to build SP with native C++ compilers on Compaq/DEC Alpha OSF/1 4.0, Hewlett-Packard HP-UX 10.0.1 and Silicon Graphics IRIX 5.3, with a command line like

make CXX=CC CXXFLAGS=-O DEFINES='-DANSI_CLASS_INST $(XDEFINES)'

Numerous compiler errors quickly led to my abandoning the effort.

Compilation with native Sun Solaris 2.7 CC 5.0 looked initially promising, but linking failed with errors about differing sizes of particular symbols, and with many missing functions arising from template instantiation. This linking problem is just what I found with SP 0.4 on the IBM RS/6000 AIX 3.2.5 systems too.

Compaq/DEC Alpha OSF/1 4.0F

Compilation with g++ succeeded, but linking failed because of missing Vector template expansion functions.

Compilation with native cxx -x cxx succeeded, but linking failed because of large numbers of missing template expansion functions.

I do not yet have a successful build on this system; the last version successfully built was sp-1.0.1, under OSF/1 3.x.

HP 9000/7xx HP-UX 10.01

I built sp on this system using make XDEFINES='-D_POSIX_SOURCE'; the extra definition is needed to expose the off_t type definition in <sys/types.h>.

HP 9000/7xx HP-UX 10.20

Mail from Michael Riedmann <Michael_Riedmann@hp.com> at Hewlett-Packard GmbH in Böblingen, Germany on 12 May 1998 reported a successful build of SP version 1.3 on HP-UX 10.20 with g++ version 2.7.2.3, after installing HP patch PHKL_8693 to fix a problem with a non-ANSI extern struct declaration in /usr/include/sys/time.h.

IBM RS/6000 AIX 4.2 and 4.3

Compilation with g++ 2.95.2 succeeded, but linking failed with missing functions arising from template expansion.

A build using the native C and C++ compilers finally succeeded, with this command: make CC=c89 CFLAGS='-O3 $(DEBUG)' CXX='xlC -O3' CXXFLAGS='$(DEBUG) $(WARN)' -k -i DEFINES='-DSP_ANSI_CLASS_INST -DSP_MULTI_BYTE $(XDEFINES)'

NeXT Turbostation Mach 3.0

Compilation failed in the file Location.cxx because of an undefined type, Mutex.