The history of luaTEX 2006–2009 / v 0.50 - Pragma ADE

More documents

Recommendations

Info

When processing a not so large le but one that loads a bunch of open type fonts, we get the following values. The left set is on linux (Taco's machine) and the right set in mine. stepmul run (s) mem (MB) run (s) mem (MB) 200 1.58 69.14 5.6 84.17 1000 1.63 69.14 6.5 72.32 2000 1.64 60.66 6.8 73.53 10000 1.71 59.94 7.0 72.30 Since I use an old laptop running Windows with a probably different TEX conguration (fonts), and under some load, both columns don't compare well, but the general idea is the same. For practical usage a value of 1000 is probably best, especially because memory intensive font and script loading only happens at the rst couple of pages. 46 A fresh look at fonts
VII Token speak tokenization Most TEX users only deal with (keyed in) characters and (produced) output. Some will play with boxes, skips and kerns or maybe even leaders (repeated sequences of the former). Others will be grateful that macro package writers take care of such things. Macro writers on the other hand deal properties of characters, like catcodes and a truckload of other codes, with lists made out of boxes, skips, kerns and penalties but even they cannot look much deeper into TEX's internals. Their deeper understanding comes from reading the TEXbook or even looking at the source code. When someone enters the magic world of TEX and starts asking around on a bit, he or she will at some point get confronted with the concept of ‘tokens’. A token is what ends up in TEX after characters have entered its machinery. Sometimes it even seems that one is only considered a qualied macro writer if one can talk the right token--speak. So what are those magic tokens and how can LuaTEX shed light on this. In a moment we will show examples of how LuaTEX turns characters into tokens, but when looking at those sequences, you need to keep a few things in mind: • A sequence of characters that starts with an escape symbol (normally this is the backslash) is looked up in the hash table (which relates those names to meanings) and replaced by its reference. Such a reference is much faster than looking up the sequence each time. • Characters can have special meanings, for instance a dollar is often used to enter and exit math mode, and a percent symbol starts a comment and hides everything following it on the same line. These meanings are determined by the character's catcode. • All the characters that will end up actually typeset have catcode ‘letter’ or ‘other’ assigned. A sequence of items with catcode ‘letter’ is considered a word and can potentially become hyphenated. examples We will now provide a few examples of how TEX sees your input. Hi there! Hi there! cmd chr id name letter 72 H Token speak 47
Page 1: MKII CONTEXT CONTEXT The history of
Page 4 and 5: XXI The luacation of TEX and ConTEX
Page 6 and 7: 4 Introduction
Page 8 and 9: Keep in mind that the functionality
Page 10 and 11: tricky part of this conversion is t
Page 12 and 13: prologues := 1 ; mpprocset := 1 ; W
Page 14 and 15: code. However, the signicantly larg
Page 16 and 17: \input zip:///somezipfile.zip?name=
Page 18 and 19: ecause we could move all kind of po
Page 20 and 21: If an acrhive is not a real TEX tre
Page 22 and 23: luatools --generate This command wi
Page 24 and 25: I know that it sounds a bit messy,
Page 26 and 27: By combining Lua with TEX, we can d
Page 28 and 29: 26 An example: CalcMath
Page 30 and 31: collapser was written. In the (pre)
Page 32 and 33: 30 Going utf
Page 34 and 35: In practice we prefer a more abstra
Page 36 and 37: of making that table accessible for
Page 38 and 39: Therefore, it will be no surprise t
Page 40 and 41: {"special","pdf: 1 0 0 rg"}, {"font
Page 42 and 43: some neat feature of \TeX, once you
Page 44 and 45: OPENTYPE 144 17.571.732 ENC 268 782
Page 46 and 47: The font denition callback intercep
Page 50 and 51: letter 105 i spacer 32 letter 116 t
Page 52 and 53: egister 2 3190 dimen other_char 50
Page 54 and 55: This example uses an active charact
Page 56 and 57: function tex.printlist(data) callba
Page 58 and 59: 56 Token speak
Page 60 and 61: Interesting is that upto now I didn
Page 62 and 63: do end local function nextbyte(f) r
Page 64 and 65: \ctxlua{collectgarbage("setstepmul"
Page 66 and 67: Collecting tokens can take place in
Page 68 and 69: where TEX has to convert to and fro
Page 70 and 71: When experimenting with the new imp
Page 72 and 73: It will be clear that here we colle
Page 74 and 75: 72 Nodes and attributes t={ attr={
Page 76 and 77: Let's end with an observation. Attr
Page 78 and 79: end end tex.sprint("\\setbox0=\\hbo
Page 80 and 81: 78 Dirty tricks
Page 82 and 83: Those who read the previous chapter
Page 84 and 85: In a similar way support for xml wi
Page 86 and 87: Undoubtely this will have a huge im
Page 88 and 89: 86 Going beta
Page 90 and 91: We like testing LuaTEX's open type
Page 92 and 93: k a el--rie am, ld p n th re, l ak
Page 94 and 95: his or her font). As said, it also
Page 96 and 97: ["tag"]="calt", } }, ["name"]="ks_l
Page 98 and 99:
When testing the implementation we
Page 100 and 101:
Mr. rmn p Of course we have to test
Page 102 and 103:
ه‎ مَ‏ ه‎ لِ‏ دْ‏
Page 104 and 105:
لاۤئِهٰٖا لاۤئِه
Page 106 and 107:
َ غْتَرِفٌُم عْتَر
Page 108 and 109:
وَْر وةَ،‏ ٰ ي كَ‏
Page 110 and 111:
white black t-green a=1.000 t=0.500
Page 112 and 113:
Palets work with names: \definepale
Page 114 and 115:
• support mechanism are under con
Page 116 and 117:
So, effectively we set a box, and e
Page 118 and 119:
116 Colors redone
Page 120 and 121:
• Spaces at the end of the line (
Page 122 and 123:
兡也包因沘氓侷柵苗
Page 124 and 125:
〉 + 々 = 〉々〉々〉々
Page 126 and 127:
124 Chinese, Japanese and Korean, a
Page 128 and 129:
for k, v in pairs(_G) do print(k, v
Page 130 and 131:
} [4] = 200 I mistakingly assumed t
Page 132 and 133:
130 Optimization
Page 134 and 135:
xetex metapost We can also
Page 136 and 137:
\startluacode xml.sprint(xml.first(
Page 138 and 139:
a[position()=5] maybe a[first()] ma
Page 140 and 141:
Now we have: okay okay okay Repla
Page 142 and 143:
\startxmlsetups xml:mysetups \xmlse
Page 144 and 145:
c1d1 c2d2 d3 d4 d5 Here com
Page 146 and 147:
c2 d3 b/c[1] c1 c2 a/c[1] d3 a/c[-1
Page 148 and 149:
The code in traditional TEX that de
Page 150 and 151:
and this is where MkIV hooks in ist
Page 152 and 153:
con-text == [c][o](pre=n-,post=,rep
Page 154 and 155:
luastate_bytes min:37009927, max:87
Page 156 and 157:
dyn_used min:478158, max:554800, pa
Page 158 and 159:
str_ptr min:2131299, max:2136226, p
Page 160 and 161:
if_stack min:0, max:13, pages:326 k
Page 162 and 163:
pdf_refximage min:0, max:7, pages:3
Page 164 and 165:
162 Collecting garbage
Page 166 and 167:
\definefontfeature[coward] [kern=ye
Page 168 and 169:
We do our tests of a variant of Con
Page 170 and 171:
--make or --ini --run or --fmt= --l
Page 172 and 173:
mtxrun --script font --list --patte
Page 174 and 175:
Opening up TEX started with rewriti
Page 176 and 177:
One can question if a properly work
Page 178 and 179:
ound to fonts (instead of being com
Page 180 and 181:
• TEX primitives Access to and co
Page 182 and 183:
• Open an instance using mplib.ne
Page 184 and 185:
is dened in the les whose names sta
Page 186 and 187:
}, }, ["pen"]={ { ["left_x"]=2.4548
Page 188 and 189:
local tx, ty = p.x_coord, p.y_coord
Page 190 and 191:
\startreusableMPgraphic{test} fill
Page 192 and 193:
vardef textext(expr str) = image (
Page 194 and 195:
local factor = 65536*(7200/7227) fu
Page 196 and 197:
end return t This is a typical exam
Page 198 and 199:
The user interface is downward comp
Page 200 and 201:
key concept. There are few accessib
Page 202 and 203:
\directlua 0 { for i=1,10 do tex.pr
Page 204 and 205:
} end And still it's not okay, sinc
Page 206 and 207:
eason why such code normally is not
Page 208 and 209:
Of course we can discuss until eter
Page 210 and 211:
some text some more When using Co
Page 212 and 213:
and the document itself is processe
Page 214 and 215:
setups and functions are assigned n
Page 216:
using some kind of api and history
Page 226 and 227:
In LuaTEX with ConTEXt MkIV support
Page 228 and 229:
• Designers (or programmers) may
Page 230 and 231:
228 OpenType: too open?
Page 232 and 233:
cannot be grabbed and they make the
Page 234 and 235:
232 It works!
Page 236 and 237:
when writing the result to le, TEX
Page 238 and 239:
236 Virtual Reality
Page 240 and 241:
\start \textdir TLT one \bidilro tw
Page 242 and 243:
240 Getting lost
Page 244 and 245:
using a few arabic fonts, some chin
Page 246 and 247:
The core data structure that we nee
Page 248 and 249:
table can contain information about
Page 250 and 251:
The ConTEXt cross reference mechani
Page 252 and 253:
of them. Each lookup has a detailed
Page 254 and 255:
otf.replacements otf.sequences otf.
Page 256 and 257:
features: analyze=yes, calt=yes, cc
Page 258 and 259:
feature mark, lookup ml_arab_l_16_s
Page 260 and 261:
للِ‏ ه‎ ِ feature init, lo
Page 262 and 263:
features: analyze=yes, calt=yes, cc
Page 264 and 265:
esult: ۝ 1: [+TRT] U+6DD:۝ [-TRT]
Page 266 and 267:
2: [+TRT] U+F01DD:۝ [-TRT] ۝ The
Page 268 and 269:
Pro. Dr. Donld E. Knuth feature cal
Page 270 and 271:
268 Tracking
Page 272 and 273:
The order in which this happens now
Page 274 and 275:
normalizers characters words fonts
Page 276 and 277:
local function hyphenation(head,tai
Page 278 and 279:
276 The order of things
Page 280 and 281:
of LuaTEX Taco and I worked in para
Page 282 and 283:
Next we see (scaled) Latin Modern:
Page 284 and 285:
As you can see, it is possible to a
Page 286 and 287:
next: U+FF008 { => U+FF06E { => U+F
Page 288 and 289:
end } commands = { { "slot", 1, bas
Page 290 and 291:
(a,b) = (1.20,3.40) So we don't nee
Page 292 and 293:
} description = "SOLIDUS", directio
Page 294 and 295:
⎧ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎩
Page 296 and 297:
840 italic 60 top_accent (0,1020) f
Page 298 and 299:
296 Unicode math
Page 300 and 301:
tex.print("access") else tex.print(
Page 302 and 303:
300 User code
Page 304 and 305:
namespace with code not coming from
Page 306 and 307:
• The code denes a few global tab
Page 308 and 309:
ut this now refers to an index in a
Page 310 and 311:
On the agenda is xing some ‘hyphe
Page 312 and 313:
We also use the opportunity to slow
Page 314 and 315:
Math. Because the TEX Gyre math pro
Page 316 and 317:
Tracing on Taco's machine has shown
Page 318 and 319:
node list callback tasks - 4 unique
Page 320 and 321:
pdftex 1.28 (23) 2.07 (144) 6.96 (2
Page 322 and 323:
July 19, 2009 - The number of files
Page 324 and 325:
July 19, 2009 - The relative number
show all

The history of luaTEX 2006–2009 / v 0.50 - Pragma ADE

Create successful ePaper yourself

Delete template?

Save as template?