advanced-algorithmic-trading

Recommendations

Info

327The Reuters 21578 dataset can be found at http://kdd.ics.uci.edu/databases/reuters21578/reuters21578.tar.gz as a compressed tar GZIP file. The first task is to create a new workingdirectory and download the file into it. Please modify the directory name below as you see fit:cd ~mkdir -p quantstart/classification/datacd quantstart/classification/datawget http://kdd.ics.uci.edu/databases/reuters21578/reuters21578.tar.gzOn Windows you will need to use the respective command line syntax to create the directories,or use Windows Explorer, and use a web browser to download the data.We can then unzip and untar the file:tar -zxvf reuters21578.tar.gzOn Windows you can use 7-Zip for this procedure.If we list the contents of the directory (ls -l) we can see the following (the permissions andownership details have been omitted for brevity):... 186 Dec 4 1996 all-exchanges-strings.lc.txt... 316 Dec 4 1996 all-orgs-strings.lc.txt... 2474 Dec 4 1996 all-people-strings.lc.txt... 1721 Dec 4 1996 all-places-strings.lc.txt... 1005 Dec 4 1996 all-topics-strings.lc.txt... 28194 Dec 4 1996 cat-descriptions_120396.txt... 273802 Dec 10 1996 feldman-cia-worldfactbook-data.txt... 1485 Jan 23 1997 lewis.dtd... 36388 Sep 26 1997 README.txt... 1324350 Dec 4 1996 reut2-000.sgm... 1254440 Dec 4 1996 reut2-001.sgm... 1217495 Dec 4 1996 reut2-002.sgm... 1298721 Dec 4 1996 reut2-003.sgm... 1321623 Dec 4 1996 reut2-004.sgm... 1388644 Dec 4 1996 reut2-005.sgm... 1254765 Dec 4 1996 reut2-006.sgm... 1256772 Dec 4 1996 reut2-007.sgm... 1410117 Dec 4 1996 reut2-008.sgm... 1338903 Dec 4 1996 reut2-009.sgm... 1371071 Dec 4 1996 reut2-010.sgm... 1304117 Dec 4 1996 reut2-011.sgm... 1323584 Dec 4 1996 reut2-012.sgm... 1129687 Dec 4 1996 reut2-013.sgm... 1128671 Dec 4 1996 reut2-014.sgm... 1258665 Dec 4 1996 reut2-015.sgm... 1316417 Dec 4 1996 reut2-016.sgm... 1546911 Dec 4 1996 reut2-017.sgm... 1258819 Dec 4 1996 reut2-018.sgm... 1261780 Dec 4 1996 reut2-019.sgm
328... 1049566 Dec 4 1996 reut2-020.sgm... 621648 Dec 4 1996 reut2-021.sgm... 8150596 Mar 12 1999 reuters21578.tar.gzYou will see that all the files beginning with reut2- are .sgm, which means that they areStandard Generalized Markup Language (SGML)[1] files. Unfortunately, Python deprecatedsgmllib from Python in 2.6 and fully removed it for Python 3. However, all is not lost becausewe can create our own SGML Parser class that overrides Python’s built in HTMLParser[11].Here is a single news item from one of the files:....<REUTERS TOPICS="YES" LEWISSPLIT="TRAIN"CGISPLIT="TRAINING-SET" OLDID="5544" NEWID="1"><DATE>26-FEB-1987 15:01:01.79</DATE><TOPICS><D>cocoa</D></TOPICS><PLACES><D>el-salvador</D><D>usa</D><D>uruguay</D></PLACES><PEOPLE></PEOPLE><ORGS></ORGS><EXCHANGES></EXCHANGES><COMPANIES></COMPANIES><UNKNOWN>C Tf0704reuteu f BC-BAHIA-COCOA-REVIEW 02-26 0105</UNKNOWN><TEXT><TITLE>BAHIA COCOA REVIEW</TITLE><DATELINE> SALVADOR, Feb 26 - </DATELINE><BODY>Showers continued throughout the week inthe Bahia cocoa zone, alleviating the drought since earlyJanuary and improving prospects for the coming temporao,although normal humidity levels have not been restored,Comissaria Smith said in its weekly review.The dry period means the temporao will be late this year.Arrivals for the week ended February 22 were 155,221 bagsof 60 kilos making a cumulative total for the season of 5.93mln against 5.81 at the same stage last year. Again it seemsthat cocoa delivered earlier on consignment was included in thearrivals figures.Comissaria Smith said there is still some doubt as to howmuch old crop cocoa is still available as harvesting haspractically come to an end. With total Bahia crop estimatesaround 6.4 mln bags and sales standing at almost 6.2 mln thereare a few hundred thousand bags still in the hands of farmers,middlemen, exporters and processors.There are doubts as to how much of this cocoa would be fit
Page 2 and 3:
ContentsI Introduction 11 Introduct
Page 4 and 5:
38.2 Correlation . . . . . . . . .
Page 6 and 7:
513.2 The Kalman Filter . . . . . .
Page 8 and 9:
719 Support Vector Machines . . . .
Page 10 and 11:
928 Kalman Filter-Based Pairs Tradi
Page 12 and 13:
Limit of Liability/Disclaimer ofWar
Page 14:
Part IIntroduction1
Page 17 and 18:
4The aim of this book is to provide
Page 19 and 20:
61.3 How Is The Book Laid Out?The b
Page 21 and 22:
81.5 How Does This Book Differ From
Page 23 and 24:
101.7.1 AlternativesThere are many
Page 26 and 27:
Chapter 2Introduction to Bayesian S
Page 28 and 29:
15Table 2.1: Comparison of Frequent
Page 30 and 31:
172.2 Applying Bayes’ Rule for Ba
Page 32 and 33:
19the next chapter. At this stage,
Page 34 and 35:
21# Create and plot a Beta distribu
Page 36 and 37:
Chapter 3Bayesian Inference of a Bi
Page 38 and 39:
25• The fairness of the coin does
Page 40 and 41:
273.4.3 Multiple Flips of the CoinN
Page 42 and 43:
29x = np.linspace(0, 1, 100)params
Page 44 and 45:
31Hence, all we need to do is re-ar
Page 46 and 47:
33Figure 3.3: The prior and posteri
Page 48 and 49:
Chapter 4Markov Chain Monte CarloIn
Page 50 and 51:
374.2.1 Markov Chain Monte Carlo Al
Page 52 and 53:
39We discussed the fact that we cou
Page 54 and 55:
41Finally we specify the Metropolis
Page 56 and 57:
43we do not need to compute 100,000
Page 58 and 59:
45x, stats.beta.pdf(x, alpha, beta)
Page 60 and 61:
Chapter 5Bayesian Linear Regression
Page 62 and 63:
49non-informative priors.• Poster
Page 64 and 65:
51# Use a linear model (y ~ beta_0
Page 66 and 67:
53# Use the No-U-Turn Samplerstep =
Page 68 and 69:
55plt.show()We can see the sampled
Page 70 and 71:
57"""Calculates the Markov Chain Mo
Page 72 and 73:
Chapter 6Bayesian Stochastic Volati
Page 74 and 75:
61plt.show()Figure 6.1: Different r
Page 76 and 77:
63( )yilog ∼ t(ν, 0, exp(−2s i
Page 78 and 79:
656.3.2 Model Specification in PyMC
Page 80 and 81:
67Figure 6.5: Overlay of samples fr
Page 82 and 83:
69plt.xlabel(’Time’)plt.ylabel(
Page 84:
Part IIITime Series Analysishttps:/
Page 87 and 88:
74• Seasonal Variation - Many tim
Page 89 and 90:
767.5 How Does This Relate to Other
Page 91 and 92:
78Definition 8.1.1. Expectation. Th
Page 93 and 94:
808.2 CorrelationCorrelation is a d
Page 95 and 96:
82Definition 8.3.4. Stationary in t
Page 97 and 98:
84Figure 8.2: Correlogram plotted i
Page 99 and 100:
868.6 Next StepsNow that we’ve di
Page 101 and 102:
88• Use our knowledge of time ser
Page 103 and 104:
909.3.2 CorrelogramWe can also plot
Page 105 and 106:
929.4.2 CorrelogramThe autocorrelat
Page 107 and 108:
94created the difference series, we
Page 109 and 110:
96there are a few that are marginal
Page 111 and 112:
9810.1 How Will We Proceed?In this
Page 113 and 114:
10010.4.1 RationaleIn the previous
Page 115 and 116:
10210.4.4 Simulations and Correlogr
Page 117 and 118:
104Figure 10.2: Realisation of AR(1
Page 119 and 120:
10610.4.5 Financial DataAmazon Inc.
Page 121 and 122:
108Figure 10.6: Correlogram of Firs
Page 123 and 124:
110> plot(gspcrt)Figure 10.8: First
Page 125 and 126:
112model, as well as the conditiona
Page 127 and 128:
114Figure 10.10: Realisation of MA(
Page 129 and 130:
116Coefficients:ma1 intercept-0.729
Page 131 and 132:
118> require(quantmod)> getSymbols(
Page 133 and 134:
120> amznrt.maCall:arima(x = amznrt
Page 135 and 136:
122Figure 10.16: Residuals of MA(1)
Page 137 and 138:
124Figure 10.18: Residuals of MA(3)
Page 139 and 140:
12610.6.3 RationaleTo date we have
Page 141 and 142:
128Figure 10.20: Correlogram of an
Page 143 and 144:
130Figure 10.22: Correlogram of an
Page 145 and 146:
132Figure 10.23: Correlogram of the
Page 147 and 148:
134Notice that there are some signi
Page 149 and 150:
136I would like to thank you for be
Page 151 and 152:
138Figure 11.1: Plot of simulated A
Page 153 and 154:
140> amzn = diff(log(Cl(AMZN)))As i
Page 155 and 156:
142We fit an ARIMA model by looping
Page 157 and 158:
144being able to create strategy in
Page 159 and 160:
14611.5.2 Why Does This Model Volat
Page 161 and 162:
148ɛ t = σ t w t (11.17)σ 2 t =
Page 163 and 164:
1502.5 % 97.5 %a0 0.1786255 0.21726
Page 165 and 166:
152Figure 11.10: Residuals of an AR
Page 167 and 168:
Figure 11.13: Squared residuals of
Page 169 and 170:
156mean, but due to its stationarit
Page 171 and 172:
158x t = pz t + w x,t (12.1)y t = q
Page 173 and 174:
160Figure 12.3: Plot of x t and y t
Page 175 and 176:
162> layout(1:2)> plot(badcomb, typ
Page 177 and 178:
164We then created two subsequent s
Page 179 and 180:
166Figure 12.6: Backward-adjusted c
Page 181 and 182:
168Coefficients:(Intercept) ewaAdj3
Page 183 and 184:
170> plot(rdsaAdj, rdsbAdj,xlab="RD
Page 185 and 186:
172set.seed(123)## SIMULATED DATA##
Page 187 and 188:
174plot(comb1$residuals, type="l",x
Page 189 and 190:
176> q <- 0.6*z + rnorm(10000)> r <
Page 191 and 192:
178Figure 12.12: Plot of s t , the
Page 193 and 194:
180EWA.Adjusted.l2 EWC.Adjusted.l2
Page 195 and 196:
182library("quantmod")library("tser
Page 197 and 198:
184
Page 199 and 200:
186• Smoothing - Estimating the p
Page 201 and 202:
18813.2.1 A Bayesian ApproachRecall
Page 203 and 204:
190E[y t+1 |D t ] = E[Ft+1θ T t +
Page 205 and 206:
192of our linear regression. The ne
Page 207 and 208:
194Figure 13.1: Scatterplot of the
Page 209 and 210:
196Figure 13.2: Time-varying slope
Page 211 and 212:
198delta = 1e-5trans_cov = delta /
Page 213 and 214:
200
Page 215 and 216:
202"risk manager". They will be use
Page 217 and 218:
204Figure 14.1: Two-state Markov Ch
Page 219 and 220:
206Figure 14.2: Hidden Markov Model
Page 221 and 222:
208> bull_var <- 0.1> bear_mean <-
Page 223 and 224:
210historical financial data.14.3.3
Page 225 and 226:
212The same process will now be car
Page 227 and 228:
214install.packages(’quantmod’)
Page 229 and 230:
216
Page 232 and 233:
Chapter 15Introduction to Machine L
Page 234 and 235:
22115.3.1 Linear RegressionAn eleme
Page 236 and 237:
223While seemingly not a traditiona
Page 238 and 239:
Chapter 16Supervised LearningSuperv
Page 240 and 241:
227ŷ = ˆf(x) = argmax z∈R p(y =
Page 242 and 243:
Chapter 17Linear RegressionIn this
Page 244 and 245:
231if __name__ == "__main__":# Set
Page 246 and 247:
23317.3 Maximum Likelihood Estimati
Page 248 and 249:
235To simplify the notation the lat
Page 250 and 251:
237Once the full dataset is created
Page 252 and 253:
239if __name__ == "__main__":# Set
Page 254 and 255:
241))lr_model.coef_[0]# Create a sc
Page 256 and 257:
Chapter 18Tree-Based MethodsIn this
Page 258 and 259:
245Figure 18.2: The resulting parti
Page 260 and 261:
24718.3 Decision Trees for Classifi
Page 262 and 263:
249In quantitative finance applicat
Page 264 and 265:
251(a) Grow a tree ˆf b with k spl
Page 266 and 267:
253tslag["Lag%s" % str(i+1)] = ts["
Page 268 and 269:
255The same approach is carried out
Page 270 and 271:
257seen whether it is feasible to p
Page 272 and 273:
259amzn = create_lagged_series("AMZ
Page 274 and 275:
Chapter 19Support Vector MachinesIn
Page 276 and 277:
263• Non-Probabilistic - Since th
Page 278 and 279:
265The key point here is that it is
Page 280 and 281:
267classification. Overfitting mean
Page 282 and 283:
269Figure 19.6: Addition of a singl
Page 284 and 285:
27119.8 Support Vector MachinesThe
Page 286 and 287:
273p∑K(x i , x k ) = (1 + x ij x
Page 288 and 289:
Chapter 20Model Selection andCross-
Page 290 and 291: 277This motivates the concept of a
Page 292 and 293: 279Figure 20.1: Various estimates o
Page 294 and 295: 281error for a particular model est
Page 296 and 297: 283Figure 20.3: Amazon, Inc. - Pric
Page 298 and 299: 28520.2.5 Python ImplementationWe a
Page 300 and 301: 287At this stage we have the necess
Page 302 and 303: 289..import pylab as plt....def plo
Page 304 and 305: 291fold = 0# Loop over the k-foldsf
Page 306 and 307: 293Figure 20.5: Test MSE curves for
Page 308 and 309: 295return tsretdef validation_set_p
Page 310 and 311: 297model = Pipeline([("polynomial_f
Page 312 and 313: 299X = lags[["Lag1", "Lag2", "Lag3"
Page 314 and 315: Chapter 21Unsupervised LearningThe
Page 316 and 317: 303parameters of the model, θ.Conv
Page 318 and 319: Chapter 22Clustering MethodsIn this
Page 320 and 321: 3072. Iterate the following until c
Page 322 and 323: 309]# Generate a list of the 2D clu
Page 324 and 325: 311clustered using K-Means and then
Page 326 and 327: 313ax.set_axis_bgcolor((1,1,0.9))ax
Page 328 and 329: 315ClusterTomorrow to contain tomor
Page 330 and 331: 317Figure 22.4: 3D scatterplot of n
Page 332 and 333: 319# Apply the K-Means Algorithm fo
Page 334 and 335: 321# up days and red for down daysc
Page 336 and 337: 323sp500["ClusterMatrix"] = list(zi
Page 338 and 339: Chapter 23Natural Language Processi
Page 342 and 343: 329for export as shippers are now e
Page 344 and 345: 331tungtung-oilveg-oilwheatwoolwpiy
Page 346 and 347: 333in order to minimise memory usag
Page 348 and 349: 335’The Tokyo Grain Exchange said
Page 350 and 351: 337for t in d[0]:if t in topics:d_t
Page 352 and 353: 339This would mean that we are givi
Page 354 and 355: 341partitions into the training and
Page 356 and 357: 3430.660194174757[[21 0 0 0 2 3 0 0
Page 358 and 359: 345self.encoding = encodingdef _res
Page 360 and 361: 347"""topics = open("data/all-topic
Page 362: Part VQuantitative Trading Techniqu
Page 365 and 366: 352While vectorised backtests are s
Page 367 and 368: 354• Backtest - Encapsulates the
Page 369 and 370: 356The second strategy provides a 3
Page 371 and 372: 35825.4.1 MonthlyLiquidateRebalance
Page 373 and 374: 360"""Size the order to reflect the
Page 375 and 376: 362Figure 25.1: US Equities/Bonds 6
Page 377 and 378: 364five years relative to US equiti
Page 379 and 380: 366# Use the monthly liquidate and
Page 381 and 382: 368import datetimefrom qstrader imp
Page 383 and 384: 37026.2 Strategy ImplementationTo i
Page 385 and 386: 372final.order[1],final.order[3]),
Page 387 and 388: 374> spIntersect = merge( spArimaGa
Page 389 and 390: 376enters what looks to be more a s
Page 391 and 392:
378# Output the CSV file to "foreca
Page 393 and 394:
380
Page 395 and 396:
382natural gas.The hypothesis prese
Page 397 and 398:
384## Carry out linear regression o
Page 399 and 400:
38627.5 Python QSTrader Implementat
Page 401 and 402:
388that arrive out of order in the
Page 403 and 404:
390if self.invested is not None:if
Page 405 and 406:
392take into account commission dif
Page 407 and 408:
394adf.test(comb$residuals, k=1)# c
Page 409 and 410:
396def go_short_units(self):"""Go s
Page 411 and 412:
398from qstrader.trading_session.ba
Page 413 and 414:
400default=’’,help=’Pickle (.
Page 415 and 416:
402Hence we can "long the spread" i
Page 417 and 418:
404class KalmanPairsTradingStrategy
Page 419 and 420:
406if all(self.latest_prices > -1.0
Page 421 and 422:
408It also changes the FixedPositio
Page 423 and 424:
410’--testing/--no-testing’, de
Page 425 and 426:
412• Asset Selection - Choosing a
Page 427 and 428:
414if self.R is not None:self.R = s
Page 429 and 430:
416)self.events_queue.put(SignalEve
Page 431 and 432:
418@click.option(’--testing/--no-
Page 433 and 434:
420• Short-Term Trend - Predict m
Page 435 and 436:
422It takes the CSV file path as an
Page 437 and 438:
424produce a final truth column whe
Page 439 and 440:
426bias-variance trade-off of the m
Page 441 and 442:
428used by the machine learning mod
Page 443 and 444:
430IQFeedIntradayCsvBarPriceHandler
Page 445 and 446:
432’--filename’,default=’’,
Page 447 and 448:
Figure 29.2: Tearsheet for Random F
Page 449 and 450:
436):lookforward_minutes=5,up_down_
Page 451 and 452:
438#ts["UpDown"] = down_tot & up_to
Page 453 and 454:
440events_queue - A handle to the s
Page 455 and 456:
442from intraday_ml_strategy import
Page 457 and 458:
444if __name__ == "__main__":main()
Page 459 and 460:
446sentiment.In recent years there
Page 461 and 462:
448Ticker Name Period LinkMSFT Micr
Page 463 and 464:
450self.statistics.update(event.tim
Page 465 and 466:
452if self.end_date is not None:sen
Page 467 and 468:
454a sent_sell corresponding exit t
Page 469 and 470:
456sentiment_handler = SentdexSenti
Page 471 and 472:
458Figure 30.2:volatile, posting mo
Page 473 and 474:
460fication by adding many more sto
Page 475 and 476:
462events_queue = queue.Queue()csv_
Page 477 and 478:
464def main(config, testing, ticker
Page 479 and 480:
466In quantitative trading this pro
Page 481 and 482:
468self.tickers[ticker]["timestamp"
Page 483 and 484:
470df["Adj Close"][mask],".", lines
Page 485 and 486:
472from qstrader.event import (Sign
Page 487 and 488:
474class RegimeHMMRiskManager(Abstr
Page 489 and 490:
476# If in the undesirable regime,
Page 491 and 492:
47831.5 Strategy Results31.5.1 Tran
Page 493 and 494:
480Figure 31.3: Trend Following Reg
Page 495 and 496:
482ax.grid(True)plt.show()if __name
Page 497 and 498:
484elif short_sma < long_sma and se
Page 499 and 500:
486elif action == "SLD":if self.inv
Page 501 and 502:
488backtest = Backtest(price_handle
Page 503 and 504:
490holding an instrument representi
Page 505 and 506:
492rolling_sharpe_s = np.sqrt(self.
Page 507 and 508:
494In order to use the annualised r
Page 509 and 510:
496Figure 32.2: Aluminum Smelting S
Page 511 and 512:
498
Page 513 and 514:
500[18] Wikipedia: Viterbi algorith
Page 515 and 516:
502[56] Investor, S. Regime detecti
Page 517:
504[92] Sinclair, E. Volatility Tra
show all

advanced-algorithmic-trading

Create successful ePaper yourself

Delete template?

Save as template?