Node:Records, Next:Fields, Previous:Reading Files, Up:Reading Files
The awk utility divides the input for your awk
program into records and fields.
awk keeps track of the number of records that have
been read
so far
from the current input file. This value is stored in a
built-in variable called FNR. It is reset to zero when a new
file is started. Another built-in variable, NR, is the total
number of input records read so far from all data files. It starts at zero,
but is never automatically reset to zero.
Records are separated by a character called the record separator.
By default, the record separator is the newline character.
This is why records are, by default, single lines.
A different character can be used for the record separator by
assigning the character to the built-in variable RS.
Like any other variable,
the value of RS can be changed in the awk program
with the assignment operator, =
(see Assignment Expressions).
The new record-separator character should be enclosed in quotation marks,
which indicate a string constant. Often the right time to do this is
at the beginning of execution, before any input is processed,
so that the very first record is read with the proper separator.
To do this, use the special BEGIN pattern
(see The BEGIN and END Special Patterns).
For example:
awk 'BEGIN { RS = "/" }
{ print $0 }' BBS-list
changes the value of RS to "/", before reading any input.
This is a string whose first character is a slash; as a result, records
are separated by slashes. Then the input file is read, and the second
rule in the awk program (the action with no pattern) prints each
record. Because each print statement adds a newline at the end of
its output, this awk program copies the input
with each slash changed to a newline. Here are the results of running
the program on BBS-list:
$ awk 'BEGIN { RS = "/" }
> { print $0 }' BBS-list
-| aardvark 555-5553 1200
-| 300 B
-| alpo-net 555-3412 2400
-| 1200
-| 300 A
-| barfly 555-7685 1200
-| 300 A
-| bites 555-1675 2400
-| 1200
-| 300 A
-| camelot 555-0542 300 C
-| core 555-2912 1200
-| 300 C
-| fooey 555-1234 2400
-| 1200
-| 300 B
-| foot 555-6699 1200
-| 300 B
-| macfoo 555-6480 1200
-| 300 A
-| sdace 555-3430 2400
-| 1200
-| 300 A
-| sabafoo 555-2127 1200
-| 300 C
-|
Note that the entry for the camelot BBS is not split.
In the original data file
(see Data Files for the Examples),
the line looks like this:
camelot 555-0542 300 C
It has one baud rate only, so there are no slashes in the record,
unlike the others which have two or more baud rates.
In fact, this record is treated as part of the record
for the core BBS; the newline separating them in the output
is the original newline in the data file, not the one added by
awk when it printed the record!
Another way to change the record separator is on the command line,
using the variable-assignment feature
(see Other Command-Line Arguments):
awk '{ print $0 }' RS="/" BBS-list
This sets RS to / before processing BBS-list.
Using an unusual character such as / for the record separator
produces correct behavior in the vast majority of cases. However,
the following (extreme) pipeline prints a surprising 1:
$ echo | awk 'BEGIN { RS = "a" } ; { print NF }'
-| 1
There is one field, consisting of a newline. The value of the built-in
variable NF is the number of fields in the current record.
Reaching the end of an input file terminates the current input record,
even if the last character in the file is not the character in RS.
(d.c.)
The empty string "" (a string without any characters)
has a special meaning
as the value of RS. It means that records are separated
by one or more blank lines and nothing else.
See Multiple-Line Records, for more details.
If you change the value of RS in the middle of an awk run,
the new value is used to delimit subsequent records, but the record
currently being processed, as well as records already processed, are not
affected.
After the end of the record has been determined, gawk
sets the variable RT to the text in the input that matched
RS.
When using gawk,
the value of RS is not limited to a one-character
string. It can be any regular expression
(see Regular Expressions).
In general, each record
ends at the next string that matches the regular expression; the next
record starts at the end of the matching string. This general rule is
actually at work in the usual case, where RS contains just a
newline: a record ends at the beginning of the next matching string (the
next newline in the input), and the following record starts just after
the end of this string (at the first character of the following line).
The newline, because it matches RS, is not part of either record.
When RS is a single character, RT
contains the same single character. However, when RS is a
regular expression, RT contains
the actual input text that matched the regular expression.
The following example illustrates both of these features.
It sets RS equal to a regular expression that
matches either a newline or a series of one or more uppercase letters
with optional leading and/or trailing whitespace:
$ echo record 1 AAAA record 2 BBBB record 3 |
> gawk 'BEGIN { RS = "\n|( *[[:upper:]]+ *)" }
> { print "Record =", $0, "and RT =", RT }'
-| Record = record 1 and RT = AAAA
-| Record = record 2 and RT = BBBB
-| Record = record 3 and RT =
-|
The final line of output has an extra blank line. This is because the
value of RT is a newline, and the print statement
supplies its own terminating newline.
See A Simple Stream Editor, for a more useful example
of RS as a regexp and RT.
The use of RS as a regular expression and the RT
variable are gawk extensions; they are not available in
compatibility mode
(see Command-Line Options).
In compatibility mode, only the first character of the value of
RS is used to determine the end of the record.
RS = "\0" Is Not PortableThere are times when you might want to treat an entire data file as a
single record. The only way to make this happen is to give RS
a value that you know doesn't occur in the input file. This is hard
to do in a general way, such that a program always works for arbitrary
input files.
You might think that for text files, the NUL character, which
consists of a character with all bits equal to zero, is a good
value to use for RS in this case:
BEGIN { RS = "\0" } # whole file becomes one record?
gawk in fact accepts this, and uses the NUL
character for the record separator.
However, this usage is not portable
to other awk implementations.
All other awk implementations1 store strings internally as C-style strings. C strings use the
NUL character as the string terminator. In effect, this means that
RS = "\0" is the same as RS = "".
(d.c.)
The best way to treat a whole file as a single record is to simply read the file in, one record at a time, concatenating each record onto the end of the previous ones.