Wednesday, November 30, 2011
Euphoria
Tuesday, November 29, 2011
Share Buybacks, Part 2
Scala Performance
I just found this experiment pitting several Python-based servers versus a Scala one. After "priming the pump" to get the VM to optimize the Scala one, Scala handled 3.6x more requests per second than the Python Twisted web server. It even handled 8.5% more requests than a largely C-infused Python server Fast Asyncronous Python WSGI Server. No memory use information was given, but these results do definitely tilt towards Scala.
Interestingly, at least according to the Computer Language Shootout (which admittedly has some issues), Scala significantly outperforms the listed JVM-hosted languages such as JRuby and Clojure (in execution time). JRuby has a steady lead in terms of code size.
Monday, November 28, 2011
Share Buybacks, Part 1
I recently read an interesting article on SeekingAlpha about IBM's share buyback record. The article says that over the past 16 years, IBM has been able to half its outstanding share count, which Warren Buffett claims as one reason why he bought $10B worth. That got me thinking, there is at least one ETF focused on share buybacks, PKW PowerShares Buyback Achievers. The fund has 139 components. What is IBM's performance relative to the rest of these? The fund has only been around since 12/2006, but presumably the index (Mergent DRB) has been around a while longer. Some of the components include TGT, DTV, TWX, TJX, COH, AZO, and GPS at 4.77%, 13.4%, 4.99%, 4.82%, 9.11%, 10.62%, and 10.35% decrease in common stock shares outstanding from FY 2010 to FY 2011 respectively. From the SeekingAlpha article, IBM has bought back anywhere from 0.01% to 8.73% of shares in a single year. Stay tuned for more information on the share buyback data point.
This is the first post in a series. The next post is here.
Friday, November 25, 2011
ETF Performance
| Period | Mkt | NAV |
|---|---|---|
| 1 mo | 11.3 | 11.3 |
| 3 mo | -3.3 | -3.2 |
| 6 mo | -8.0 | -8.0 |
| 1 yr | -7.8 | -7.8 |
| Period | Mkt | NAV |
|---|---|---|
| 1 mo | 11.30 | 11.33 |
| 3 mo | -3.26 | -3.17 |
| 1 yr | 7.76 | 7.84 |
| Period | Mkt | Total |
|---|---|---|
| 1 mo | 11.31 | 11.33 |
| 3 mo | -3.15 | -3.16 |
| 6 mo | -7.96 | -7.99 |
| 1 yr | 7.83 | 7.87 |
Though the Schwab and Yahoo data are agreement, notice how the iShares reported performances diverges starting from the 3 mos point. In a low yield environment, the difference between -3.26% and -3.15% cumulative market returns is huge. I have not been able to reconcile this discrepancy exactly. It is not merely the $0.262 dividend distribution during the 3-mo period. I think the moral of the story is threefold:
- take nothing for granted
- always read the fine print on how performance is calculated
- try to obtain performance data from multiple sources
Thursday, November 24, 2011
Yahoo Finance Gotcha
Wednesday, November 23, 2011
Pre-Holiday Closing
Tuesday, November 22, 2011
Static Type System and Numerical Computing
Monday, November 21, 2011
Top 5 Open Source Quant Finance Projects
Friday, November 18, 2011
Datamining Sentiment from News Text
Options Expiration Friday
Today is options expiration Friday and Leading Indicators at 10:00am. Futures currently read as follows:
| Contract | Price |
| ES Dec | 1223.00 |
| ZN Dec | 130'155 |
| DX Dec | 77.840 |
Thursday, November 17, 2011
Correlations
| spy | sdy | tlt | zroz | uup | gld | xle | xlf | |
| spy | 1.00 | 0.93 | -0.68 | -0.64 | -0.34 | -0.57 | 0.95 | 0.94 |
| sdy | 0.93 | 1.00 | -0.41 | -0.36 | -0.15 | -0.43 | 0.83 | 0.77 |
| tlt | -0.68 | -0.41 | 1.00 | 1.00 | 0.69 | 0.51 | -0.77 | -0.87 |
| zroz | -0.64 | -0.36 | 1.00 | 1.00 | 0.72 | 0.45 | -0.75 | -0.84 |
| uup | -0.34 | -0.15 | 0.69 | 0.72 | 1.00 | -0.08 | -0.53 | -0.51 |
| gld | -0.57 | -0.43 | 0.51 | 0.45 | -0.08 | 1.00 | -0.42 | -0.61 |
| xle | 0.95 | 0.83 | -0.77 | -0.75 | -0.53 | -0.42 | 1.00 | 0.95 |
| xlf | 0.94 | 0.77 | -0.87 | -0.84 | -0.51 | -0.61 | 0.95 | 1.00 |
The correlation coefficients are computed for the past 100-days of returns calculated from adjusted closing prices for each asset. There is a little something for everyone here. Equities (SPY) and dividend-paying equities (SDY) have strong correlations. Despite appreciating crude prices, energy stocks (XLE) are still strongly tied with equities as a whole. To get negative correlation with equities, one may look to treasuries (TLT and ZROZ [Pimco Long Dated Zeros]), the dollar, and gold. TLT and ZROZ similar but in terms of correlations with other assets, it happens to be TLT that exhibits the slightly more pronounced negative correlations despite ZROZ being more of a pure play. Surprisingly, the least correlated assets in this study are gold (GLD) and dollar (UUP).
Wednesday, November 16, 2011
Programming Language Gotchas
Looks like I spoke too soon. The October rally was looking a little long in tooth, and a little warning from the credit agencies was enough to get the market to go risk off at least for this afternoon. At least yields did not decline too much (2.01% for the 10-year according to MarketWatch). The Dollar Index (DX), however, has certainly been in an uptrend since the end of October trough.
Coming from the Standard ML world of languages makes me take many things for granted. It also gives me a probably pretty narrow perspective. Hacking on my trading analysis tools in other languages helps me broaden that somewhat, though I have certainly worked in the mainstream languages long before I encountered ML and its brethren. I would like to keep a record of these gotchas as I encounter them. I do not intend this as value judgments but rather a documentation of noteworthy differences from a higher-order typed programming language perspective.
Cash Secured Puts
Cash secured put writing is a venerable strategy where an investor sells puts while simultaneously keeping enough cash for the duration of the put contracts to cover possible exercise. Note that although puts tend to be exercised upon expiration, there is a significant probability of early exercise. This strategy is typically considered a conservative strategy, in the realm of options strategies at least, because an investor will not have to fork over any additional money to cover margin calls so max loss is capped at the exercise value of the put minus premium from the put writing. Typically, this strategy is utilized when an investor expects the underlying instrument to stay largely flat (i.e., appreciate or depreciate no more than the premium generated from writing the put contract). With an options qualified brokerage account, an investor can easily execute his own cash secured put writing strategy. Some ETF managers (e.g., WisdomTree), however, have also introduced the strategy as executed on broad equity indices in the form of an ETF.
Seeing that this morning was a benign CPI day with no earth-moving news from Europe, I thought that it would be an interesting exercise to look at a volatility-selling options strategy. The strategy is a fairly conservative one, cash secured put writing. My sample is the 341 component subset of the S&P 500 with options activity today (i.e., there are bids and asks). I will be only looking at the nearest to the money strikes to the last traded price of the underlying. The puts expire in less than 3 days (Friday, 11/19/11) so there really is not much time premium left. The returns are computed as put premium (midpoint of the bid and ask) over the strike times the 100 multiplier (i.e., the amount of cash needed in case of assignment). The VIX today (the last time I looked) remains elevated over 30.
The above is a histogram of the cash secured put returns. The horizontal axis marks the returns (put premium/ (100*strike)) and the vertical the number of securities that have that return. The median, mean, and standard deviation for the returns are 0.000675, 0.00101, and 0.00111 respectively. The max is 0.01 for TSS.
Tuesday, November 15, 2011
Server Infrastructure
Recently, I have been building a market data feed and order book simulator. Since I had already built an Ncurses front-end (console GUI) for Trader Workstation, this was the opportunity to try out something new. I decided to try out Lift/web, Scala's web framework. Lift/web is a perfectly fine framework. The ORM side of things (Mapper for RDBMS and Record geared more towards NoSQL) still seem to be a work-in-progress (though a very usable work-in-progress), but Comet (Ajax-Push) works as advertised. In my experience, however, memory overhead seems to be a real issue. During development, Lift managed to gobble up all the default heap space after just a few source updates. None of the requests were large by any means. Apparently, there are a few configuration tweaks I still need to try to get this under control, but this whole episode got me thinking: with all the emphasis on massively scalable webapps (it seems everyone aspires to serve millions and billions of requests), why do most people tolerate the memory use profiles of Rails/Ruby, Python/Django, and PHP?
There are probably plenty of issues with this study on Oscigen, an OCaml-based web app framework, but it does seem to reflect the reality that there is a lot of inefficiency in modern web framework stacks. The above benchmark claims that under fairly benign circumstances, Oscigen outperforms Rails by 10x in terms of both requests/sec and reduced memory use (4.5MB versus 49MB). Oscigen's memory profile apparently ties with a simple C-based web app (using FastCGI). Interestingly, on the opposite end of the spectrum, there is also a C++-based web stack, Wt. Regardless of the merits of the benchmark, the fact is that people put up with a considerable amount of memory use. Considering that memory on Amazon EC2 is not exactly free, it does concern me. On one web app hosting organization's site, Micro instances are only good for PHP apps whereas Small instances are needed for the lightest of Rails and Java apps. Large servers with 7.5GB RAM (which currently run from $244/month) are needed for "full-size" Rails and Java apps. Is the current state of things a case of memory and servers are too cheap or caching covers over all problems? There is always the red herring of network latency, but if concurrent requests served with constrained resources is the main metric then memory use matters.
Or maybe it is because good garbage collectors are hard to come by. At least at one point, garbage collection was an issue in Ruby. This chain of thought led me to look up whether there are any good solutions for web stacks with low memory footprints, especially any that come in neat statically-typed packages. There is certainly Oscigen/OCaml. The StackOverflow people have also suggested Adam Chlipala's Ur/Web system. In the latter stack, after some searching through mail list archives, I have learned that Ur/Web uses region memory management (where the boundaries of a request delimits regions). The main website kept mentioning that Ur/Web is fast at least in part due to not having a garbage collector. That, of course, got me curious as to what did it use for memory management. Ur/Web also seems to be built on Standard ML (the Makefile has settings for both SML/NJ and MLton), which is certainly a plus according to my bias. Though explicit regions (aka memory pools [but careful, this is an overloaded term] or arenas) are pretty efficient (being part of Apache, nginx, PostgreSQL), region inference has its own issues, principally that programs written in a region-unfriendly fashion can lead to poor performance. The consensus on the matter appears to be that regions and garbage collection can be combined in a single system to address region-unfriendliness while simultaneously minimizing garbage collection.
Monday, November 14, 2011
Genetic Programming Performance
There is a whole lot of literature on the subject of genetic and evolutionary algorithms for trading strategies. The quality of the work seems to vary considerably. I have yet to discover the canonical and seminal lines of work in the field. Unlike in programming languages research and perhaps other scientific fields, the bibliographies do not all follow the same chain of citations. Or perhaps they do, but I have yet to discover it. More on this after the jump.
Friday, November 11, 2011
Genetic Programming and Trading Strategies
From an anecdotal sampling of Linkedin and web discussions, people have applied various machine learning techniques to the creation of trading strategies. The most popular techniques for trading strategies mirror that of the general applied machine learning community, meaning SVM and neural networks are frequently mentioned. One relatively lesser known technique is genetic programming (GP), a variant of genetic algorithms for tree data structures, normally abstract syntax trees. Curiously, the academic community has studied genetic programming quite a lot. Indeed, there are many papers on genetic programming for trade strategies in GECCO (a conference on genetic and evolutionary computation) and other conferences/workshops. Some people from State Street have even contributed their study. The Quant Stack Exchange has a discussion on genetic algorithms where the problem of data snooping is frequently mentioned as a shortcoming, a problem addressed by regularization in conventional learning techniques. I informally survey some of the studies of the effectiveness of GP in a follow-up post.
My cursory interest in this area is the potential impact of programming language design and especially static type systems on the performance of genetic programming for trading strategies. The nice thing about genetic programming is that the results produced can be theoretically unrestricted unlike basic machine learning techniques that make assumptions such as linearity about the underlying model. Practically, however, the results produced by genetic programming are usually small and possibly uninteresting. This is where modern programming languages theory comes in. In the past decade or so, a number of researchers have studied the problem of the generation of random test cases (of which QuickCheck is a prime example). One of the main engineering challenges there is to ensure that the test cases generated do not all fall into a trivial category. The test cases here can range from instances of simple primitive types (e.g., ints) to nontrivial data structures (e.g., syntax trees or data packet queues). Usually it falls to the developer to write functions that generate a good distribution of test cases to ensure, for example, that the probability of getting a list of length greater than 3 is not negligible. In practice, QuickCheck supplies an assortment of combinators that help produce a good distribution. The developer's domain knowledge is still necessary, but the test generation problem reduces down to a choice of combinators.
Monday, November 7, 2011
Semantics of Nested Classes
First thing is first, what are nested classes?
class A {
class B { }
}
In essence, they are scoped type declarations and definitions. Instead of defining a class B at the top-level of the program source, one declares a locally scoped one within the definition of another class, A in this case. One immediate consequence of this is a form of namespace management for user-defined aggregate types (i.e., classes). There are a lot of gotchas due the language design here. The scoping discipline is incomplete. C/C++'s restriction on shadowing prohibits the following code:
class B;
class A {
B something;
class B { };
}
In contrast, the following code compiles:
class A {
private:
class B {};
public:
B makeB();
}
Despite the fact that A::B is unnameable outside an instance of A, A exports the member function makeB which yields a value of type A::B. Though one cannot name the return value of makeB outside of A, the value can still be used as is in the case in the following:
class A {
private:
class B {
int x;
public:
B add(B _x) { x += _x; }
};
public:
B makeB();
};
int f() {
A a0, a1;
a0.makeB().add(a1.makeB());
}
The above illustrates an important point. C++ nested classes do not declare distinct types for each instance. Although one cannot annotate them as static, they are exactly that. In the example, a0.B and a1.B are of the same type. This is why a0.makeB().add(a1.makeB()) type checks. This means that writing a nested class is equivalent to lifting that class definition out to the top-level and selectively hiding the name of the nested class. Between different languages, there is a design chasm. In C++, there is only the unitary nested class and its associated semantics. Java distinguishes between static nested classes (similar to the C++ nested class) and inner classes. Java actually has a number of variants of nested classes (https://blogs.oracle.com/darcy/entry/nested_inner_member_and_top). The above C++ example translated into Java fails type checking because of A.B is private.
class A {
private class B {
public B add(B rhs) { val += rhs.val; return this; }
private int val;
}
public B makeB() { return new B(); };
}
public class Nested0 {
public static void main() {
A a0 = new A();
A a1 = new A();
a0.makeB().add(a1.makeB());
}
}
The call to add is flagged as a problem because a0.makeB()'s type (i.e., A.B) is inaccessible. If A.B were public or protected, however, the type checker is satisfied despite a0.B and a1.B supposedly denoting two distinct inner classes. Thus, at least at this point, inner classes do not provide abstract types in the sense of generative types.
On the other hand, as this discussion on Java decompilation (javap) shows, the semantics of inner classes are such that have a kind of C++
friendrelationship with their outer enclosing class. The discussion on TheServerSide shows how this is implemented: the compiler generates a static method that "breaks the encapsulation wall" to permit accessing private members of the outer class.
References
- Herb Sutter wrote an article on nested classes in C++.
