Algorithmic trading

[MediumTex]

But I do want to point out that in the old days, before central banking and before fiat currencies, there was still speculation.  It is built into the system based on weaknesses of human behavior until we all grow up and adopt something like Riegel's Valun so there are no leveraged excesses and thus, no need for speculation (although I wonder how future risks will be hedged if there are no speculators willing to take the other side of the trade...).  The most extreme "left wing" political elements would love to see futures contracts, swaps, etc. banned and eliminated because they believe speculation is damaging.  That is putting the cart before the horse of course, but they do have an unconsciousble sense that something is horribly wrong with the entire monetary/credit system.

If you haven't, I heartily recommend reading "Against The Gods: The Remarkable Story of Risk".  It will put a lot of things into perspective about all of the benefits to civilization from risk-taking and speculation.

I think what troubles some people is not speculation so much as leveraged speculation where the effect of a bad trade potentially sets off cascading failures within the larger financial system.

I don't have any problem with someone gambling with their own money.  It's when they gamble with other people's money through insane levels of leverage that I think some people get worried, in part because when this approach goes bad it is often the taxpayer that gets to pay, and not the parties that took the risk in the first place.

[akratic]

I'm also rather curious about the nuts and bolts of the tools you work with.  How complex do these systems get?  Are you sitting down and working with a much larger set of libraries in a big, complex system or do you just sort of plop down in front of a blinking cursor on a brand new Perl script every couple of weeks?

Very complex, and completely proprietary.  That's part of why the engineering challenge is so interesting.  One of my friends worked on a platform that was built completely in OCaml, which isn't even in the top 100 most popular programming languages.  I doubt there was a good reason for that, just the whim of whoever built the system first.

I also don't understand why it would be so hard to create inefficiencies to exploit them.

If you create an inefficiency, you bear the entire cost of creating that inefficiency, but then you have to split the inefficiency reduction profits with every good algo out there.  Say it cost you $x to create 100 units of inefficiency, and you subsequently profit $y from reducing the 100 units.  I can't just really imagine y being bigger than x.  Creating the inefficiency is *so costly*.  Every trade you made to create it is by definition a terrible one at the time you made it.  But then on the inefficiency reduction, most algos have only a slight edge over the next algo.  That 100 unit pie of inefficiency is probably getting shared by 100 algos that all get ~1 unit.  Sure you'd probably get a slightly bigger share, because you intimately understand what caused the inefficiency this time, but certainly you don't get anywhere near 100% of the profit, after paying the full cost yourself.  -100 units to create the inefficiency, +10 units for your share of the reduction, -90 unit loss.

The only way I could really see inefficiency creating working is if you can be the straw that broke the camel's back, making a slight shove at exactly the right time that causes the market to make a huge move despite only a small input from you.  To be honest this would be a lot easier for a human trader to do than an algo though.  The break the camel's back subroutine would be quite a challenge to code up; it's more of an intuitive/human intelligence that could sense the possibility of such a push working.

Furthermore, the more good algo trades we have, the harder it gets for any market participant to create these inefficiencies anyway!  If anything this whole example is more of an argument for automation rather than against it.

My guess is that a big part of the reason those companies have such a huge war chest is they don't do stupid stuff like throw it all away on losing inefficiency causing trades.  And why would you need to when other market participants create inefficiencies all day anyway?

Perhaps my perspective is just indicative of the types of trades that MIT kids work on though, the big math and analysis puzzles.  Perhaps if your style is more manipulation based you'd come up with a way to make what you're suggesting actually work.  The algo traders are on your side, though, stone, against these manipulators.  Every algo that can absorb volatility and instantly correct inefficiency is another layer of defense against manipulation.

At a more elemental level, I question what it is all in aid of.

I'm going to have to think more about your 2% spread non-profit market maker hypothetical.  Initial reactions:
1) You'd need to ensure that no for-profit electronic trading markets were created anywhere in the world, because if they were, everyone would switch.  This might require a military big enough to police every single country, because the reward for that country creating a more efficient market place would be enormous.
2) There's going to be lots of unintended consequences.  Take currencies for example, where the modern spread is about 0.005%.  You want to change that to 2%.  Suddenly global trade grinds to a halt.  And who are you going to redistribute the bid-ask spread to anyway?  Everyone with USD?  You could do your system for just US stocks maybe, but that's only a fraction of the algos.

I think the prevailing mentality is that trading must be either lose-win or win-lose, where if some subset of the market is profiting that profit must be coming at the expense of someone else.  Perhaps it's this mindset that is really behind most people's distaste for algo trading?  I guess the idea is that if the non-profit market maker cuts out the algos and the HFTs there would be more money for everyone?

Personally, I believe in win-win, and I believe that efficient markets with tight bid-ask spreads are a good thing.

[edsanville]

By contrast, the "decision moose" and even the "leveraged etf PP" strategies not only harvest money from such volatility, they also feed into amplifying it.

How does the leveraged ETF PP amplify volatility?  Seems like it just buys low and sells high just like the regular 1x PP.

[stone]

Akratic, thanks again for all your reassurances and explanations.
My impression (probably wrong) was that many market participants have stop loss limits. I guess those are used because they are using leverage and so have to prevent insolvancy. Don't such stop loss limits mean that all that is needed to trigger a big move is to "run the stops"? I thought that the precipitous fall in the oil price that happened last spring was just such a cascade of successive stop loss limits being triggered. Short squeezes can similarly cause upside moves I thought? You said yourself that your algos switch off when they detect craziness that they do not understand. To me that makes it sound as though a manipulator might not have to share the spoils all round - they would be the one who expected the craziness and so would be staying in there whilst all the other algos left the table and the stop loss limits were unloading their bounty. Also, wouldn't option trading provide a massive gain for a given price movement of the underlying security?

About the hypothetical clunky stock market that paid the spread as a dividend to holders of the security- I'm not sure why such a stock market would not be competative. Your fancy high speed co-located computors cost money, the legions of best of the best algo traders cost money. All of that expense would instead get recycled back to the owners of the stock. When you say that it would be less efficient- how exactly does that inefficiency manifest? My naive imagining was that all it would mean would be that the stocks would trade more like small cap stocks- the price would jerk about a bit in a step wise fashion rather than having a smooth intraday price chart. What is the harm of that? Isn’t the point of the stock market to supposedly be to allocate resources? How does a smooth intra-day chart help that? If someone is investing in a stock, surely a 2% spread is neither here nor there against a context of getting back all of the trading spreads as a dividend?

[stone]

edsanville, with a leveraged ETF PP, the person holding the ETFs is buying the ETFs low and selling them high but the manager of the leveraged ETFs has to buy high and sell low each day so as to maintain a constant ratio of debt to the securities held. Apparently when there was a 3% daily surge in the SP500 recently, at the very end of the trading day, there was a large jump up that was attributed to buying by leveraged inverse ETFs. An  analogous effect comes when leveraged bull ETFs have to sell off the underlying security after a market fall. The effect is then prone to inducing an "all aboard" bandwagon of front running or so I thought. I'm happy to be corrected though.
eg see:
http://michaeljamesmoney.blogspot.com/2 ... arket.html

http://money.cnn.com/2011/11/08/markets ... /index.htm

[6 Iron]

While there is societal value in the improved market liquidity and reduced spreads created by algorithmic trading, I do not understand why business/industry do not see the value in rewarding their most talented and productive engineers. I hate to see our best funneled into finance. That said, Akratic, thank you for the insight.

[stone]

What puzzles me is why people put so much reverence in tight spreads for stocks and bonds and yet are happy to have some of their savings as gold coins with a spread of 3% -or whatever it is-. Additionally I still don't see why in principle stock market spreads couldn't be recyled back to the stock holders from a not for profit market maker if it were set up that way.
As far as liquidity goes, it seems to me that the algo traders provide liquidity until it is actually required. Akratic has said that his algos switch off when they detect craziness that they do not understand. That sounds like a situation where a big chunk of the market vaporizes the moment the going gets tough such as in the May 2010 flash crash. Everything is wonderfully efficient and all inconsequential minor volatility gets harvested by the algo traders when the going is good and then, when consequential risk crops up, they have a microsecond route for the exit.

[stone]

What Medium Tex said about the adverse consequence of people trading with borrowed money makes a lot of sense to me. To take a simplified example, if you were “trading”? on the results of flipping a coin, then it would be possible to have a strategy that won most of the time but gave back those winnings on the rare occasions when a catastrophic loss occurred: http://en.wikipedia.org/wiki/Martingale ... ng_system)
Ofcourse if you are using your own money and have to cover the downside as well as the upside then that would be pointless. BUT if you are a finance company and you get to keep any upside but simply file for bankruptcy and get jobs in a new company whenever the once in a blue moon event occurs, then it is something that you absolutely have to do in order to remain in the game with your competitors.
From what I can see, much financial activity is in essence just such a martingale betting strategy. Trading with stop loss limits or with algos that opt out when they detect “craziness they don’t understand”? acts to substitute a continuous mix of small wins and losses for a long sequence of predominating wins punctuated by huge losses when a close succession of stop outs happen to occur.
I suppose on a macro scale what happens is that leveraging up by banks entails interest that must be got from somewhere and so eventually has to be offset by debt forgiveness in the form of bankcruptcies.
As I say, I know absolutely nothing about this. I’m just struggling to get my head around it.

[akratic]

As far as liquidity goes, it seems to me that the algo traders provide liquidity until it is actually required.

As the algos get smarter and smarter, they will have to bail less often.  There is a big incentive to be able to stay running through these events.

From what I can see, much financial activity is in essence just such a martingale betting strategy.

If you really want to game the finance system you're better off betting against fat tails than just doing a martingale.  Consider a trade that would return +50% nine years out of ten, and -500% on the one year out of ten that something completely unexpected happens.  This trade has negative expected value mathematically, but can have positive expected value to you personally.

1) You might be able to retire on just a few +50% years, and might never see the -500% event at all.

2) Even if you do see a -500% event, you might not have to pay for it; you could get bailed out for example.

(Frankly the banks being able to make high risk trades and fall back on being bailed out is a complete fucking travesty.)

Anyway, two safeguards against this happening in a non-bank situation are:
1) you would have to completely pull the wool over the eyes of your bosses and risk managers, who are on the lookout for stuff like this and are probably smarter than you are
2) you would have to live with yourself afterwards

[akratic]

I do not understand why business/industry do not see the value in rewarding their most talented and productive engineers.

In my dream world the most lucrative thing you can do is work in education.  Too bad reality doesn't work that way!

Fortunately, I can create my own reality by first saving up a bunch of money, investing it in the PP, and then working on whatever I feel like.

[stone]

akratic, I did mean that the betting was against the fat tail -500% event just as you describe. I thought that that was a martingale in essence? Abviously you are not playing a 50:50 game and doubling your stake each trade. Instead it is much more incremental but the same principle underlies it IMO. That -500% would come in the form of a succession of stop loss or "leaving the table with a loss" events such that if played out would accumulate to a -500%. Do you not think that that phenomenon is implicit in any trading strategy that ever leaves the market in response to a loss and uses 5x leverage? Doesn't that include your algos that exit when they detect craziness that they do not understand? Presumably they suffer a loss with each such exiting event. If they don't, then isn't that only due to them having priviledged trading speed and so transfering the loss to less well equiped market participants? Am I in a muddle with all of this?

In terms of algos of the future being able to avoid exiting choppy markets- in the olden days market making was by guys who had a covenant that meant that they couldn't exit choppy markets. They had to weather the storm. It seems to me to be something that needs a regulatory fix not a technological one. Again I may be in a muddle about this too.
Cheers for fielding this barrage of naysaying from me .

[stone]

akratic, the problem with education is that people have to pay for it somehow. Paying off education debts is part of what drives people to need money I guess.

[akratic]

I don't know about other people's trades, but I don't consider any of mine to be a martingale or a bet against fat tails.

We could have liquidity through regulation/force/compulsion, or we could have it through open competition by market participants.  I believe open competition will lead to better results.  You're welcome to believe the opposite.  This point in particular seems like an older debate at the heart of the clash between capitalism and communism.

[stone]

akratic, I suppose it is always a terribly hard question as to what best protects what is best about capitalism etc. Where should it be let free and where does it need boundaries to keep it serving humanity rather than having humanity in service to an ideology that perhaps in essence is not self regulating? I'm not sure myself.

[Storm]

Isn’t the point of the stock market to supposedly be to allocate resources? How does a smooth intra-day chart help that? If someone is investing in a stock, surely a 2% spread is neither here nor there against a context of getting back all of the trading spreads as a dividend?

Stone, I think akratic's point is that a 2% spread is fine, but if an algorithmic trader can cut the spread to 1.95% then everybody wins.  The company gets more volume, the investors get a better price, and the algo trader makes a little money.  This is pretty much pure capitalism at work here.  He who can create the most efficient market wins.

[Storm]

While there is societal value in the improved market liquidity and reduced spreads created by algorithmic trading, I do not understand why business/industry do not see the value in rewarding their most talented and productive engineers. I hate to see our best funneled into finance. That said, Akratic, thank you for the insight.

Well said, 6 Iron.  I don't know why either, but I suspect it is the same reason that most business is in a race to the bottom where the cheapest imported goods replace slightly more expensive but better quality "made in America" goods.  Large companies like Microsoft have told the big companies "you don't need expensive engineers, you can get cheap offshore labor."  They don't have the skills or work experience of top US engineers, but you can hire 2 or 3 for the same price.

What I see happening though is that large companies do this, and when you are making software, having a lot of poorly qualified engineers really hurts your performance.  They can introduce bugs and set your schedule back months or even years.  The ones that are really cleaning up are the startups that go into a Fortune 50 or Fortune 100 company and say "see that product that will take you 5 years to develop in house and cost $20 million?  We can do it in 6 months for a price of $2 million."  I've met smart engineers that are working for themselves on small teams of 10-20 developers, and they are making money hand over fist in this economy because the large companies can't engineer their way out of a wet paper bag.  They are strangled by their own bureaucracy and red tape.  In a large company, the average software engineer probably only spends less than 10 hours a week actually writing software!  They spend the rest of the time in "agile scrums" or tracking their time in SAP or in meetings, or chatting about the family with co-workers (not that there's anything wrong with that).  I've seen guys in a startup spend 70-80 hours a week buried in writing code.  And those 70-80 hours are extremely productive, because they spend enough time to "get in the zone" and can accomplish easily 20 times the work output of a "10 hours a week" engineer.

Anyway, I've ranted enough, but this is what I've noticed in my time spent in corporate America.  It's a race to the bottom, hire the lowest common denominator because they're cheaper kind of world.  You get what you pay for.

[Storm]

What Medium Tex said about the adverse consequence of people trading with borrowed money makes a lot of sense to me. To take a simplified example, if you were “trading”? on the results of flipping a coin, then it would be possible to have a strategy that won most of the time but gave back those winnings on the rare occasions when a catastrophic loss occurred: http://en.wikipedia.org/wiki/Martingale ... ng_system)

Actually, that is the only fool proof betting system - if you are given unlimited margin, you simply double your bet each coin flip until you win.  You are guaranteed to walk away with as much money as you want, so long as the house lets you keep playing...

[Kel]

Storm - I have automated trading systems that I have developed and trade - I suppose one could call it a form of algorithmic trading but not quite like what Akratic was mentioning  - I trade the emini futures contracts...I use esignal as a data feed....MB trading for the broker and Tickquest's Neoticker Platform to execute my automated systems. I have my home trading computer UPS backed up so if I lose power it can go for about 2 hrs. I can remote login to my trading computer to check on things. Every trading day at 6 am I have the complete login process automated so when the market opens it will execute trades etc automatically....as such I sometimes wake up to find my self in a position - i wake up late at times. I only day trade the futures and exit at the end of the day at the latest.

It has taken me a long time to develope strategies that actually work  and I only do this on a one lot basis at the moment. Neoticker can also work with other electronic brokerages including Interactive Brokers. Sometimes there is a disconnect between the broker and my pc - while it happens very seldom I do monitor how things are going and I know my strategies well so I can determine easily whether a trade is likely to occur or not on a particular day and monitor accordingly. Obviously there is no guarantee things will work into the future and as such I have a written trading plan including identifying when  a system no longer works and how much risk overall I am willing to take.

Definitely a variable portfolio type of strategy  :)

[stone]

Storm, " I think akratic's point is that a 2% spread is fine, but if an algorithmic trader can cut the spread to 1.95% then everybody wins.  The company gets more volume, the investors get a better price, and the algo trader makes a little money.  This is pretty much pure capitalism at work here.  He who can create the most efficient market wins."

I think it is important to step back and think about what you mean by "efficiency". To me efficiency means if I put £50k in stocks and those companies earn x% per year, I get as much as possible of that x% back as total returns and that, to the extent that stock price effects companies, the stocks are not grossly mispriced. Part of avoiding losses will be having a small spread when I buy and when I sell but that is only a very small part. The set up I was suggesting would have recycled all of the 2% spread back to stock holders as a dividend. How can you get more efficient than that? Trading outfits that justfy their existence on the basis of being market makers collectively cream off hundreds of billions of dollars. In what sense is that level of administrative burden efficient? Also plumets in price such as the May 2010 flash crash make a mockery of the "small spreads" theme. If you are a punter putting your monthly savings into stocks, you know very well that at least 2% is at the mercy of short term jitteryness. I'm just saying that it would be perfectly feasible to recycle such jitteryness back to stock holders rather than handing it over to algos.

No one is claiming that a large part of trading profits are earned from the spread. Closing the spread is an incidental consequence of the competition between traders.  The vast bulk of the trading profits come from volatility capture. If the traders were not there and the only market participants were  cudely choosing companies that seemed a good buy on a fundamental basis then that money available to be creamed off by volatility capture would  instead get distributed randomly amongst the market participants.

[Storm]

Storm - I have automated trading systems that I have developed and trade - I suppose one could call it a form of algorithmic trading but not quite like what Akratic was mentioning  - I trade the emini futures contracts...I use esignal as a data feed....MB trading for the broker and Tickquest's Neoticker Platform to execute my automated systems. I have my home trading computer UPS backed up so if I lose power it can go for about 2 hrs. I can remote login to my trading computer to check on things. Every trading day at 6 am I have the complete login process automated so when the market opens it will execute trades etc automatically....as such I sometimes wake up to find my self in a position - i wake up late at times. I only day trade the futures and exit at the end of the day at the latest.

It has taken me a long time to develope strategies that actually work  and I only do this on a one lot basis at the moment. Neoticker can also work with other electronic brokerages including Interactive Brokers. Sometimes there is a disconnect between the broker and my pc - while it happens very seldom I do monitor how things are going and I know my strategies well so I can determine easily whether a trade is likely to occur or not on a particular day and monitor accordingly. Obviously there is no guarantee things will work into the future and as such I have a written trading plan including identifying when  a system no longer works and how much risk overall I am willing to take.

Definitely a variable portfolio type of strategy  :)

Kel,

This is very cool, and that was what my original idea was more in line with.  I wonder though, if you have a winning strategy, would it be worthwhile to use margin to increase your profits, or would it add too much risk?  This sort of goes along with akratic's comment about how if you have a winning strategy you're better off using other people's money.

I think some viable strategies could be as simple as "the trend is your friend" in Forex (making a bet that a currency spread will continue to trend in the same direction and having a rising stop limit in place to protect your winnings).

Thanks for sharing.

[Storm]

Stone, you should probably forget about 0 spreads.  They will never exist.  Market makers need to be compensated because they take on a certain amount of risk by opening positions themselves.  Market makers commit to buying or selling a stock even if there are no buyers or sellers.  That is risk.  You get a trade that executes immediately, and the market maker pockets a few bucks.  It's a worthwhile trade.  Otherwise, what if the market is truly horrific and you want to sell?  You might be stuck with an open position for several hours because there are no buyers.

[stone]

Storm I realise  that I’m probably just naive about this but I still don’t follow why  a “mutual market maker”? run on behalf of the stockholders is not viable. If it charged a 2% spread the technology would only need to be extremely basic. The risk of buying stock when sellers outnumbered buyers would simply amount to a stock buyback such as is done all the time by companies anyway.  Clearly if the stock got sufficiently undervalued,  the 2% spread wouldn’t put off buyers even with  a short term outlook.
To my mind the current system is inane. What relevance  has trading speed and financial mathematics got to the underlying economic function of the stock market? Profits from say oil companies now get distributed on the basis of who has the fastest trading platform and has hired the best code writers and mathematicians

[Storm]

Storm I realise  that I’m probably just naive about this but I still don’t follow why  a “mutual market maker”? run on behalf of the stockholders is not viable. If it charged a 2% spread the technology would only need to be extremely basic. The risk of buying stock when sellers outnumbered buyers would simply amount to a stock buyback such as is done all the time by companies anyway.  Clearly if the stock got sufficiently undervalued,  the 2% spread wouldn’t put off buyers even with  a short term outlook.
To my mind the current system is inane. What relevance  has trading speed and financial mathematics got to the underlying economic function of the stock market? Profits from say oil companies now get distributed on the basis of who has the fastest trading platform and has hired the best code writers and mathematicians

It's all about risk and the time value of money.  Market makers do provide a valuable liquidity function.  They make sure that there will always be a buyer or seller even when there are none.  They use their own personal money to do this, and that is a risk.  Your non-profit market scenario, where companies have to become their own market makers, would never happen in the real world.  It introduces an unknown risk to corporate profits in that in a market sell off, the company could be forced to buy back shares in their own stock even if they are low on cash or suffering financial hardship.  In the event that a company stock is dropping, that is exactly the worst time to do a forced buyback.  Sure, they buy shares at a discount, but maybe their share price is dropping because they are low on cash and now you just forced them to buy more shares and take out credit to do so.

Companies list themselves on markets because investors are there, willing to buy, and take on the risk of owning shares.  Why on earth would any company list themselves on an exchange that told them "in the event that your stock has more sellers than buyers, you agree to buyback any excess shares to provide liquidity."  It would be like a bank telling you that in the event the value of your home goes down you agree to pay off your mortgage, in full, immediately.  The whole purpose of credit markets would be gone, because nobody wants credit unless the terms are well known, in advance, with no surprises.  That way they can adequately price risk.

[stone]

Storm, market makers currently are using money they get from banks. If akratic screws up and his algos buy a load of stock they can't sell, ultimately it is the bank that loses the money. I'm not saying that the hypothetical "mutual market maker" should offer an attractive price. Currently market makers sell all of the stock they have by the end of the day at whatever price they can don't they. A "mutual market maker" would do the same. I think it is worth bearing in mind just how expensive an enterprise it is that would be cut out by this. It costs billions and billions of dollars. There are hundreds of thousands of very high cost people employed.

Doesn't the "bullion vault" gold storage company do its own market making?

[stone]

Storm, I don't think the share holders of a company are analogous to the mortgage lender for a house buyer. The company does not get money from the share holders, the share holders own the company. If they sell out of the company, each owner has to find a buyer. I'm just saying that the remaining owners could in effect act as intermediary buyers. The remaining owners would take the risk of doing that and would want to do it so as to make the company attractive for new shareholders.