May June 1980 - Commodore Computers

42 COMPUTE. MAY/JUNE. I 198O 1980 ISSUE 4oil
BIG FILES ON A
BIG FILES ON A
SMALL
Elizabeth Deal
337 W. First Ave.
Malvern,
COMPUTER Pa,19355
~a~~§5~
337 W First Ave.
The program described here demonstrates a way of
reducing data storage requirements by a factor of
eight. It is written in Microsoft Basic for a PET
computer.
I have seen several programs that create and
use cross-index files mes for library search, statistical
surveys and similar applications. They usually require
large computers, such as a 48K system with two
disk drives. . A very thorough file e handling system
has been described recently by Dr. Sanger in the
November, 1979, , issue of Microcomputing. In his
article each attribute is coded as two letters and six
attributes are permitted for each record. This T requires
twelve letters and, therefore, twelve bytes.
.
In the method described here, each attribute is
coded as yes or no and the user can have as many
attributes as he desires. . If the application lends itself
to such coding into a list of keys or attributes,
then this system will permit the handling of large
amounts off data in core at one onc time. It also permits
the use of logical AND, OR or NOT operators in
retrieval with any combination of attributes.
By way of illustration, , a library ry search requires
quick access to those entries that contain desired
subject matter. Two, three, or six byte coding of
each key is very core consuming, and limits the
number of records that can be in core at one time.
The solution I propose is twofold; : (1) set up a
smart coding procedure for classification of subjects
described in an article into keys that can be scored
yes or no, and (2) ""pack" the data for storing it
in corc, core, on tape or on disk, and then ""unpack"
it, , one record at a time, during the search for the
applicable attributes. This paper describes an efficient
way to ""pack" and "unpack" the data so that a
larger file e can be searched on a small computer
without the use of accessory memory devices, such as
disks. Of course, if one has a system with a disk
the method described here would permit use of an even
larger file. We are aware that the e first part of the
solution (setting up the coding procedure) is challeng·
ing. It is the real problem and the performance of
the system depends on how logical and meaningful
the selected keys are.
Each logical record consists of the text part and
the data part. The text part must be adequate for
positive identification of the articles being searched,
but the length should be kept to a minimum. . Name,
date, and page might be enough. . The data part is
what we can compress. The yes-no or 1-0 codes
are entered in groups of fi fifteen fteen ones and zeros.
These, in turn, , are packed into the e two byte bylC integer
cgcr
variable S % .
variable S%.
Fifteen attributes require two bytes, thirty attributes
require four bytes, and so on. . A user of the
system need not concern himself with what the
system need not concern himself with what the
program does with binary ry numbers. He only needs to
know that there will be as many S% values per
record as there are groups of fifteen keys. The user
record as there are groups of fifteen keys. The user
then needs to provide a decision for retrieving
the records of interest to him. . The decision is
written as a statement at the e beginning of a program
and is immediately edited for syntax-type errors.
and is immediately edited for syntax-type errors.
Logical operators AND, , OR, NOT, , as well as arithmetic
ones ( ~ , < >, < , > ) are used . The
metic ones ( =, , ) are used. The
decision can be written on one or more lines leading
to a combining variable TR. TR is set to one if true,
to a combining variable TR. TR is set to one if true,
and all records meeting TR condition are then
displayed. Complete instructions for writing TRT R lines
are listed in lines 2970 to 3420.
How is it done? For once those long tables of
powers of two, that are a part of every book on
programming, come in handy. The program is set up
in such a way that the user thinks of the list of
fifteen ftee n keys from left to right, 1 to 15. TheT program
sees them as being numbered from right to left, 0 to 14.
Like this:
-Key numbers B%(k)k = ~ 1 233 4 5676 7 8 9 101112
11 12
13 14 15
-Program sees as m ~ 1413 12 11 10 9 8765 4 3
-Program sees as m = 14 13 12 11 10 9 8 7 6 5 4 3
2 1 0
-Input key values 100011111100000
·Input key values 1 0 0 0 1 1 1 1 1 1 0 0 0 0 0
The program now takes key values and wherever it
finds a ""1" it raises 2 to the m·th m-th power. The sum of
all this is then stored in integer variable S S%(record
number, , sum number)' number)*. . The bytes are used instead
of at least 15. During the process of retrieval the
of at least 15. During the process of retrieval the
opposite procedure takes place - the sum is "un
­
packed" into working storage of 15 values. The same
packed " into working storage of 15 values. T he same
values are, of course, reused by all records. The lines
of the program that drive this system are 1470 to
1510 and 1920 to 1990 the other way. It seems like
1510 and 1920 to 1990 the other way. It seems like
a lot of hassle, but the core saving is tremendous.
The loops that do the packing and unpacking take
The loops that do the packing and unpacking take
from 0.2 second to 0.9 second, the latter representing
all fi ftee n bits on . (These times could be reduced by
all fifteen bits on. (These times could be reduced by
rewriting these two loops as machine code subrou­
tines.) Another way to save time would be to set up
tines.) Another way to save time would be to set up
the most frequently used keys next to one another as
this will leave the loop sooner. In the example pi ~ shown
above, the program will loop ten times. Had a " 1"
above, the program will loop ten times. Had a "1"
been in position 4 or 11 the loop would be executed
five times.
The program now has two sections. One packs
the data, the other unpacks it. In between, , the
values should be stored on tape. And at the beginning,
values should be stored on tape. And at the beginning,
routines for creating and updating fil files should be
provided. As listed , the program works as if it were a
provided. As listed, the program works as if it were a
file e system. . It can be used as a training ground in
writing decision lines. It should be used as a part
writing decision lines. It should be used as a part
of a larger system. .