In order to read in an input file or include file, the scanner calls a submodule called the mapper that reads a file in and creates a contiguous copy of it in memory. The scanner then performs three checks on the file, the first (file termination) of which is performed before scanning commences, and the other two of which take place during scanning before each line is scanned.
1cm 1cm File Termination: The first check the scanner makes is whether the file is terminated properly. A file is considered to be properly terminated if it either contains no lines, or if the last line in the file is terminated by an end-of-line marker. If the scanner detects that an input file is not properly terminated, it adds an end-of-line marker itself (to the copy in memory only).
1cm 1cm Unprintable Characters: The second check the scanner makes is for unprintable characters (ASCII 0–31 and 127–255 (except for EOL(10))) which it flags as errors and replaces by question marks.
1cm 1cm Line Lengths: The third check the scanner makes is input line length. When FunnelWeb starts up, a default maximum input line length of 80 is set. This can be changed dynamically during scanning using a @p maximum_input_line_length pragma. If the number of characters on a line (not including the end of line marker) exceeds this limit, FunnelWeb generates an error.