The history of luaTEX 2006–2009 / v 0.50 - Pragma ADE

Opening up TEX started with rewriting all io related activities. Because we wanted to be
able to read from zip les, the web and more, we moved away from the traditional kpse
based le handling. Instead MkIV uses an extensible variant written in Lua. Because we
need to be downward compatible, the code is somewhat messy, but it does the job,
and pretty quickly and efciently too. Some alternative input media are implemented
and many more can be added. In the beginning I permitted several ways to specify a resource
but recently a more restrictive url syntax was imposed. Of course the le locating
mechanisms provide the same control as provided by the le readers in MkII.
An example of reading from a zip le is:
\input zip:///archive.zip?name=blabla.tex
\input zip:///archive.zip?name=/somepath/blabla.tex
In addition one can register les, like:
\usezipfile[archive.zip]
\usezipfile[tex.zip][texmf-local]
\usezipfile[tex.zip?tree=texmf-local]
The last two variants register a zip le in the tds structure where more specic lookup
rules apply. The les in a registered le are known to the le searching mechanism so
one can give specications like the following:
\input */blabla.tex
\input */somepath/blabla.tex
In a similar fashion one can use the http, ftp and other protocols. For this we use independent
fetchers that cache data in the MkIV cache. Of course, in more structured
projects, one will seldom use the \input command but use a project structure instead.
Handling of les rather quickly reached a stable state, and we seldom need to visit the
code for xes. Already after a few years of developing the rst code for LuaTEX we reached
a state of ‘Hm, when did I write this?’. When we have reached a stable state I foresee that
much of the older code will need a cleanup.
Related to reading les is the sometimes messy area of input regimes (le encoding) and
font encoding, which itself relates to dealing with languages. Since LuaTEX is utf-8 based,
we need to deal with le encoding issues in the frontend, and this is what Lua based le
handling does. In practice users of LuaTEX will swiftly switch to utf anyway but we provide
regime control for historic reasons. This time the recoding tables are Lua based and as a
result MkIV has no regime les. In a similar fashion font encoding is gone: there is still
some old code that deals with default fallback characters, but most of the les are gone.
The same will be true for math encoding. All information is now stored in a character
table which is the central point in many subsystems now.
172 The luacation of TEX and ConTEXt