Preview / Summary for Awk Users

The following points provide a quick (albeit dense) overview of Ftwalk, particularly for users who are already familiar with awk and script programming.

1. Ftwalk is a script programming language very much like awk. Most scripts consist of test expressions paired with action blocks (along with optional BEGIN and END blocks and function definitions). Each test expression is evaluated for each input event and, if true, the corresponding action block is executed.

2. You can provide an entire ftwalk script on the command line (use single quotes to keep the shell from messing with special characters), or you can put the script into a file (use -f path to specify the file, or make the script executable and specify the ftwalk path in a first line beginning with #!).

3. Ftwalk handles the command line like awk. The first command line arguments that begin with - are options, then a script, then parameter assignments (name=value pairs that set ftwalk variables after any BEGIN blocks), then regular arguments.

4. Ftwalk evaluates test/action pairs for each input event. Ftwalk can work with a variety of input event generators, but the default is a file tree search module which starts at the command line arguments (pathnames, or . if omitted) and generates an input event for each file found. Like awk, ftwalk sets $0 to the current input (in this case a file stat object for the current file), and uses the current file as a default argument for many functions.

5. Anything you can do with the UNIX find program you can easily recode in ftwalk. In most cases, find's option names are the same as ftwalk function names, but find's + or - option parameters convert to ftwalk expressions (e.g., find -atime -5 becomes ftwalk atime < 5). A couple of differences: find's -type becomes isreg, isdir, etc.; -exec is sh, -ok is oksh; in ftwalk size is in bytes and sizeb is in blocks.

6. If you run ftwalk with the -awk option, or run ftwalk under the name hawk, the regular arguments are interpreted as text file names, like in awk. (A line #source awk in the script has the same effect.) Each argument is a file name (no arguments defaults to standard input). The file is read, split into records, and the test/action pairs are evaluated for each record.

7. The syntax for ftwalk scripts is very much like awk, but with some extensions and embellishments. You can use until as the negation of while, unless as the negation of if. The break and continue statements are multilevel (like sh). There is a switch statement (with case and default like C, but no break or dropthrough; the cases can be expressions with implicit left operands, like: case > 10 && < 20: ...). One embellishment that is almost transparent is that the newline characters which end statements in awk are context-dependent in ftwalk, so you can write code in more of a C-like style, while almost never having to backslash a newline.

8. Like awk, ftwalk allocates global variables wherever an unrecognized symbol is used, and variables may hold any type of object. (Ftwalk also has static and auto keywords to explicitly define a variable.) However, where awk has a loose type system that converts between strings and floating-point numbers, ftwalk has a rich type system and is more strongly typed. Like C++, ftwalk operators depend on their operand object types, so << will do a bit shift left on numbers or will write to an output file stream. In most cases the type of the left operand determines the behavior. The typeof function tells you the type of an object, and there are conversion functions (like int, float, string) to force conversion to particular object types.

9. Ftwalk has 300 or so built-in functions (compared to 20 for awk), plus a number of built-in variables and object-oriented functions (functions that are called like C++ member functions: object . function). Many of the functions are simple wrappers for UNIX API functions, which may be familiar and work pretty much as expected (the main differences are simplifications: common prefixes for structure members are usually dropped (e.g., pw_dir for a user's home directory becomes dir), defaults for common arguments, no need for buffer sizes and lengths, functions which in C return data indirectly usually return data directly in ftwalk). When calling a function that needs no arguments, you can omit the following ( ).

10. Some functions allow for options. A function option looks like a function argument, but is specified as an identifier followed by : and an optional parameter. Most options have single-character abbreviations as well as longer option names.

11. As in awk, you define your own functions with an initial function keyword, followed by the definition. User-defined functions can find out how many arguments they are called with, the type of each argument, and any options. User-defined functions can be called as object-oriented functions, and access their object as this.

12. Comments start with # and continue to the end of the line, unless they are recognized as directives (put a space after the # to ensure that it is ignored as a comment). Directives are inspired by C preprocessor directives, many of which are comparable (#include, #define, #if ... #endif). Directives are evaluated immediately, so they can control script compilation and set preconditions for executing the script (such as #source awk to use the awk-like input generator).

13. Ftwalk has three numeric types: int, unsigned, float. The int and unsigned types are based on the native C long (or long long, if available), and float is based on double. The whole math library is provided. If you run ftwalk with the -i option switch, you get a handy calculator, where you can type in expressions and get answers printed back. Actually, you can type in and evaluate any ftwalk expression, compound statement, or other construct. This is the interactive monitor, and it is an excellent way to explore ftwalk.

14. Ftwalk supports regular expressions as a distinct type. Regular expressions are specified as R" ... " or // ... //. Matches against regular expressions are done with the ~ and !~ operators. These have the side effect of setting RSTART and RLENGTH (like awk).

15. Ftwalk strings can be arbitrarily long, and can hold null characters (like perl). You can read a whole file into a string with getfile, and write it back out again with putfile. (The shr and shw functions do the same thing with shell command pipelines.) All of awk's string functions are provided, plus more. Conversion to/from strings can be done using C-like sprintf and sscanf, as well as perl-like pack and unpack. sprintf and related functions support a %( ... ) conversion, where whatever is in parentheses is taken as ftwalk code, evaluated, and the result converted to a string.

16. Ftwalk supports intervals, which are defined by two scalar values joined with the .. operator (e.g., 1..10). You can do comparisons against intervals, or use the in operator to test whether a value is included in an interval. Intervals are particularly useful for date/time ranges. Date/time objects convert to int in seconds or to string using a cftime format specified by the DTFMT variable, and have members like the C struct tm. Date/time literals are specified by T" ... ". If the specification string has .. in it, it produces a date/time interval.

17. Ftwalk has extensive support for lists. The list function defines a list. Specialized lists can be created by array, queue, or stack. Lists can nest or be kept flat. Lists can be kept in sorted order, and you can define your own sort functor (a data type like pointers to named or anonymous functions). The += operator adds a value to a list, and -= removes a value (del removes an indexed element). The << operator inserts a value into a list, and >> extracts a value from a list. The in operator tests whether a value is included in a list. The brackets [ ] can be used to select a list element (with an integer index starting from 1; negative indexes offset from the end of the list), a range of elements (with an interval), or a subset of elements (with a functor). The &, |, and ^ operators perform set operations on lists. split splits a string into a list; string converts a list to a string, using the LISTFMT variable to specify the default conversion. Whole files can be read into lists using the getfile $l option, and lists can be written to files using putfile (which knows a list when it sees one).

18. Like awk, ftwalk supports associative arrays, which provide a mapping from a string key to a value (we call them maps). Uninitialized variables become maps when you use the brackets [ ] on them. Use the in operator to test whether a key is defined in a map, and delete to drop a key from a map. Ftwalk also provides map-like access to the environment (ENVIRON) and to NDBM databases.

19. One aspect of awk that ftwalk didn't follow was file I/O. Awk uses shell-like redirection characters and opens/closes files/pipelines as needed. Ftwalk uses the more traditional open (or popen or socket), do something (get, put, scanf, printf, read, write, seek, tell, fcntl, ioctl, lockf, ...), close model. The << and >> operators work like C++ (except that >> cannot be type-dependent, since it reads into an untyped variable), and you can << endl. The standard files (following C++) are: cin, cout, cerr, clog. (Although there is an awk-like getline, and more compatibility may be implemented in future 1.5.X releases.)

20. Ftwalk has structures, where members are accessed C-like as object . member. Each structure has its own type name (use typeof) and member list (members). Some functions return predefined structures (e.g., getrlimit, msgctl, uname) or take structure arguments (like fcntl for setting file locks). New structure instances are created with new structure. You can define your own structures as struct { member, ... }.

21. Ftwalk has special types for other stuff you are likely to run into: user IDs, group IDs, Internet addresses, file stat objects. The user and group objects readily convert to string (name) and int (numeric ID). Internet addresses convert to dotted string form or can be used directly by functions like gethostbyaddr without having to go through pack and unpack. File stat objects hold the C struct stat for a file and its pathname (if known; file stat objects can also be obtained for open files). These objects have members (you can get a user's shell with User . shell), but you cannot change the member values.

22. Some of ftwalk's most powerful functions are: backup, which compares source and destination files and copies the source file to the destination if they differ; install, which compares a source file to an arbitrary installation file, copies the file if different, and sets ownership/permissions according to specifications; dircmp, which compares two directories and puts all of the filenames from those directores into four lists, so you can readily identify which files are the same, different, or missing on one side or the other.

23. When ftwalk runs, it creates a log file in your home directory (.ftwalk.log), which gets copies of any error messages and possibly other diagnostic messages. You can write to this log file by writing to clog or using debug. You can turn extensive run-time diagnostics on either from the command line or within your script. You can specify a different log file pathname with the -log command line option or the FTWALK_LOG environment variable, or disable the log file (with -nolog or setting FTWALK_LOG to an empty string value).

24. All of this and more is documented in the on-line help file, which you can view with the fthelp program (or almost any HTML browser, e.g., Netscape). You can access fthelp from the interactive monitor by typing ? (optionally followed by a keyword that you want to search for). When you run fthelp from the interactive monitor, type f to return to the interactive monitor.