For Yacas developers


A crash course in Yacas maintenance for developers

This document intends to give a concise description of the way Yacas is maintained. There are a few parts to maintenance to take into account:

http://www.xs4all.nl/~apinkus/backups/


The autoconf/automake system

The important thing to remember with the autoconf/automake system is that there are files enumerating the files and directories that are part of the distribution. The Makefile.am files contain the lists of files that are part of the distribution, and are in the respective directory. If a file is added by a developer, but it is not added to the list in the file Makefile.am, it will not be added to the tar ball which is uploaded to the back up repository. It has the nice side effect that you can have local files which don't automatically get added to the distribution.

Adding a directory requires adding the directory name to various places also, in configure scripts etcerera.


Maintaining Yacas through a cvs repository

CVS provides an efficient way for developers to work together, automatically merging changes various developers make, and at the same tile is a back up system (uploading your changes to another computer from which you can easily obtain it at a later time). After a little effort setting it up it becomes very easy to use. This section describes the few commands needed for keeping your version and the version in the Yacas repository up to date.


How does cvs work?

CVS has a copy of the files in the repository somewhere in a directory on some system, possibly your computer. Then there is such a thing as a cvs server which you can talk to to synchronize your version of the source code with the version on the server.

CVS uses a diff-like scheme for merging differences: it looks at two text files, determines the different lines, and merges accordingly. It discovers the changes you made by looking at the version you checked out last and the version you have now, to discover which lines changed (it maintains an automatic version number for each file).

If the version of a file on your system and the version in the cvs repository has a line that has been changed by both you and some one else, the cvs repository will obviously not know what to do with that, and it will signal a 'collision' which you will have to solve by hand (don't worry, this rarely happens). More on that later.

In commands to be described in this document are in short:


Checking out an initial version of Yacas

There are two ways to check out a version of Yacas: as anonymous user and as maintainer. Anonymous users don't need to log in, but also have no right to commit changes. Maintainers first need to get an account (at sourceforge), and their account needs to be enabled so they are allowed by the maintainer to make changes. A maintainer needs to log in with every command. To be able to log in, you need ssh1 installed (ssh2 will not work). You can find this at http://www.ssh.org/download.html.

To check out Yacas as anonymous user, type:

cvs -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas login
cvs -z3 -d:pserver:anonymous@cvs.yacas.
  sourceforge.net:/cvsroot/yacas co yacas

To check out as a maintainer, type:

export CVS_RSH=ssh1

This will tell CVS to use ssh1 for communication. Then, in order to get yacas, type

cvs -d:ext:loginname@cvs.yacas.sourceforge.
  net:/cvsroot/yacas co yacas

where loginname is your name on the sourceforge system. This then creates a directory yacas/ with the full most recent distribution. You need to enter your password there, but other than that, that's it!

Those lines typed above are long and intimidating, but it is also the last time you need to type them. From now on, if you want to do anything with cvs, just go into the yacas/ directory you just checked out, and type the cvs command without the -d... flag. This flag just tells cvs where to find the repository. But future cvs commands will know where to find them, which is why you don't need that flag.


Use case scenario 1 : getting the latest version of Yacas

You haven't looked at Yacas for a while (shame on you!) and want to check out the latest version. Just type

cvs update -d

on the command line in the yacas directory, and that should essentially download the latest version for you in that directory (just the changes). The -d option here states that you are also interested in new directories that were added to the repository. Oddly enough, cvs will only get you changed and added files, not added directories, by default.

A command

cvs -q update -d

will print messages only about changed files.


Use case scenario 2 : you made changes to Yacas

You got the latest version, but saw this huge, glaring omission in Yacas, and start hacking away to add it yourself. After a while, after playing with the code you wrote, and if you think you are finished with it, you decide you like to add it to the cvs repository. First, you added a few files, which you need to add:

cvs add [list of file names of ascii text files]

which adds ascii text files. If you added binary files (GIF images in the documentation directory, or something like that), you can add it with

cvs add -kb [list of file names of binary files]

In case files need to be removed, there are two options:

There seems to be no easy way to rename or move files; you would have to remove them at their old location and add them at a new location.

Now, when finished with that, you might want to 'commit' all changes with

cvs commit

If the commit succeeds, an email is sent out to the maintainers, who can then scan the diff files for changes, to see if they agree with the changes, and perhaps fix mistakes made (if any).

If there is a collision, the commit fails (it will tell you so). In case of a collision, you need to invoke cvs update twice. The cvs update outputs a list of file names with a character in front of them. The important ones are the files with a 'C' before them. They have a collision. You can go into the file, and see the collision, which the cvs system conveniently marks as:

<<<<<<
old version
===========
new version
>>>>>>

You can edit the file by merging the two versions by hand. This happens very rarely, but it can happen. Use cvs commit afterwards to commit.

The commit and update commands can be performed in specific directories, and on specific files, if necessary, by stating them on the command line. Or you can go into a sub directory and do a cvs commit or cvs update there, if you are confident that is the only place that changed or whose changes you are interested in.

That is basically it, a quick crash course cvs. It is actually very convenient in that usually all that is needed is a cvs commit to fix small bugs. You type that in, and your version gets merged with the changes others made, and they get your changes, and you backed up your changes at the same time (all with that little command!).

You can find more information about cvs at http://cvsbook.red-bean.com/ .


Preparing Yacas documentation from formatted plain text files


Introduction

Yacas documentation in HTML and PS/PDF formats is generated by Yacas scripts from Yacas source files. However, it is cumbersome to write those source files in the Yacas language. The script txt2yacasdoc.pl makes it easier to create and maintain the documentation.

The "source" form of all documentation is maintained in a special plain text format. The format is such that it is clearly readable without any processing and is easy to edit. To compile the documents, the system processes the plain text docs with a script to prepare Yacas-language files and then runs the Yacas scripts to produce the final documentation.

The source text must be formatted in a certain fashion to delimit sections, code examples, and so on, but the format is easy enough to enter in a plain text editor. Text is marked up mostly by TAB characters, spaces, and asterisks "*" at the beginning of a line. The script txt2yacasdoc.pl converts this markup into Yacas code.

There is also a utility book2txt that attempts to convert existing documentation in the Yacas format into the plain text form suitable for txt2yacasdoc.pl. After conversion, the plain text docs normally require further editing.


Usage of the script txt2yacasdoc.pl

The script acts as a stream filter:

perl txt2yacasdoc.pl < file.txt \
  > file.chapt

In this example, file.txt contains some formatted plain text (source text) and the resulting file file.chapt will be produced in Yacas-language documentation format.

There is a single option for txt2yacasdoc:

perl txt2yacasdoc.pl -debug < file.txt \
  > file.chapt
This option is to be used for debugging, i.e. when the resulting file does not compile in Yacas. The effect of this option is to introduce many more breaks between text strings in the generated file, so that the Text() function is called more often. It is then easier to locate the source of the problem in the Yacas-language file.


Formatting of source text files

Formatting of source text files uses TAB symbols; if your editor does not support them and converts them to spaces, you should convert the results back to contain real TAB symbols using the standard Unix unexpand utility or a custom perl script.

Currently the following markup is implemented:

In> 1+2;

Out> 3;
Note that empty lines in a single sample code block must be also TAB indented or else the sample code block will be split into several sample code paragraphs. Text immediately following a sample code block will not be made into a separate paragraph. A sample code block may be separated from the text that follows it by an empty line.

It is necessary to separate sample code blocks from section or chapter headings when they follow them. The reason for this is that the script will assume that everything which is at least single-TAB indented (up to a separation between paragraphs) belongs to one sample code block. This makes it easier to enter multiply indented sample code: e.g. a double-TAB indentation inside a sample code block will not start a new section. For example:

While(x<0) [
	x:=x+1;
	Write(x);
];

*	Item
*	Another item

*	0. First item
*	0. Second item

Note that item text continues until the next itemized line is given or until end of paragraph.

Nesting of enumerated or itemized environments is not yet supported.

*INCLUDE ../essays/howto.chapt

Note that the included document must be a Yacas-language file. (This will become the IncludeFile() call in the Yacas document -- an alias to Load().)


Formatting text for the reference manual

The formatting explained in the previous section is enough to create most of the user guide and tutorial documentation. The script txt2yacasdoc.pl implements some additional markup features to help create reference manual sections.

The additional elements all start with an asterisk "*" in the first position on a line, followed by an uppercase keyword. A typical reference manual subsection documenting a certain function may look like this in plain text:

*CMD PrintList --- print list with padding
*STDLIB

*CALL
{PrintList}(list)
{PrintList}(list, padding);

*PARMS
{list} -- a list to be printed
{padding} -- (optional) a string

*DESC
Prints {list} and inserts the {padding} ...

*E.G.

	In> PrintList({a,b,{c, d}}, " .. ")
	Out> " a ..  b .. { c ..  d}";

*SEE Write, WriteString
Compare this with the reference manual section on the function PrintList to see how this plain text markup is rendered in the finished documentation.

Notes:

In a subsection there may be either one function documented or several at once: for example, it may make sense to document Sin, Cos and Tan together. In this case, all function names should be simply listed in the *CMD header, for example:

*CMD Sin, Cos, Tan --- Trigonometric ...

In addition, there is a tag *INTRO to denote a "reference chapter introduction" corresponding to the ChapterIntro() function, *A which creates an HTML link to a manual section corresponding to AddBody(HtmlAnchor()...), *AA which creates AddAnchor(), and *HEAD which creates a small heading (Topical()). For instance,
*PARMS
results in the same text as
*HEAD Parameters:

This markup should be sufficient for creating reference documentation in plaintext.


book2txt: Conversion of existing documentation to plain text

Currently, not all of the Yacas documentation markup functionality is implemented in the simple plaintext filter; also, documentation includes some extra HTML files. However, most of the markup needed to write documentation is present. Therefore it is possible to maintain most of the documentation in the plain text format described above. To convert existing Yacas documentation back to the plain text format, a script book2txt.ys/book2txt.sh can be used.

By using a command such as
book2txt.sh file.chapt
one can create a source text file file.chapt.txt corresponding to the Yacas documentation file file.chapt. For example:

12:51pm scriabin> book2txt.sh intro.book
[editvi.ys] [gnuplot.ys] 
True;
Out> True;
Quitting...
File 'intro.book.txt' was created.
12:51pm scriabin>

In the above example, the shell commands in book2txt.sh executed the following Yacas commands,
Use("book2txt.ys");
ToFile("file.chapt.txt")
  Load("file.chapt");
This requires that the Yacas script book2txt.ys be available in the current directory. The shell script book2txt.sh assumes that book2txt.ys is stored in the same directory as book2txt.sh.

Of course, it is possible that some features of Yacas documentation were not implemented in the script and in that case the resulting file must be edited by hand. But the purpose of the book2txt script is exactly this: to make a plain text source file to be edited and maintained.

Several files can be converted at once, for example:
book2txt.sh f1.chapt f2.chapt file3.book
Each file is processed by an independent Yacas session. Any errors of processing are printed on the screen.


book2TeX: preparing typeset documentation

The script book2TeX.sh prepares a TeX file out of a Yacas-language documentation book. Usage is similar to book2txt.sh, except that only one file is processed at a time. For example:
book2TeX.sh intro.book intro.book.tex
will create a TeX-formatted version of the introductory tutorial. It can be processed with standard tools, for example
latex intro.book.tex
latex intro.book.tex
dvips -o intro.book.ps intro.book dvi
will prepare a Postscript version, while
latex intro.book.tex
pdflatex intro.book.tex
will prepare a PDF version.

The shell commands in book2txt.sh execute the following Yacas commands:
Use("book2TeX.ys");
ToFile("file.chapt.tex")
  Load("file.chapt");
This requires that the Yacas script book2TeX.ys be available in the current directory. The shell script book2TeX.sh assumes that book2TeX.ys is stored in the same directory as book2TeX.sh.

Not all features of Yacas documentation are compatible with TeX typesetting. To prevent errors, documentation should avoid certain things. In general, it is a good idea to check the typeset appearance of documentation, since it helps detect errors.

For example, the symbols %, { }, < >, #, \, _ and & are special to TeX. They should not normally be used in plain text; it is okay to use them in "typerwriter" text or code samples -- but not in section or chapter heads, because it makes it difficult to export to TeX correctly. TeX commands may be entered but will not be correctly rendered in HTML online documentation. Sometimes fixed-font text will hang over the right edge; a workaround is to break the fixed-font text into shorter fragments or to rephrase the text.

Another concern is that code examples are typeset in a fixed-width font and may not fit into the width of the page and will be hanging over it. To avoid this, the code in the code examples should not be longer than about 44 characters.

The current implementation uses a "report" style which allows chapters, sections, subsections and includes an automatically generated table of contents. ( LaTeX is automatically run twice to produce the TOC when executing make texdocs.)

Some complicated mathematical expressions may not correctly render in TeX. This is because Yacas uses its internal function TeXForm() to transform Yacas expressions to TeX. Mathematical expressions are entered in the plain text documentation source using Yacas syntax, then transformed to a special non-evaluating call TeXMath() in the Yacas-language documentation, which formats into HTML using a Write() call or into TeX using a TeXForm() call, as necessary. Testing should be performed on documentation before releasing it.


The Yacas debugger program


Introduction

This essay describes the yacas debugger that comes with the distribution. The yacas debugger consists of:

The debug version has to be built independently from the 'normal' version of Yacas.


Why introduce a debug version?

The reason for introducing a debug version is that for a debugger it is often necessary to introduce features that make the interpreter slower. For the main kernel this is unacceptable, but for a debugging version this is defendable. It is good for testing small programs, to see where a calculation breaks. Having certain features only in the debug version keeps the release executable can be kept lean and mean, while still offering advanced debug features.


How to build yacasdebug

The yacas debugging components can be built by going into the src/ directory, and using the file makefile.debug to build it:

cd src/
make clean
make -f makefile.debug

Note the object files generated are not suited for the normal Yacas executable. To rebuild the normal version a make clean is required.

The file makefile.debug was written to compile under Linux, but should work on other platforms with little tweaks.

After the build the debug files can be found in the src/ directory.

To invoke the debugger, type

./yacasdebug --rootdir ../scripts/

The Yacas debugger assumes the current directory for scripts, so the --rootdir command line option is required.


What does yacasdebug offer?

The Yacas debugger is in development still, but already has some useful features.

Yacasdebug:

The libyacasdebug.a library is used by proteusdebugger, a graphical front-end for debugging which is in development still also.

Future versions will have the ability to step through code and to watch local and global variables while executing, modifying them on the fly.