Posted on

My Journey to Fully Automate My Trading: Can’t Believe Two Months Have Passed

image

Last week marks the end of my second month of automated trading. It was a relatively quiet month with not much activities. Since everything is working smoothly, it is time to raise the number of contracts again. It will be interesting to see what happens when volatility picks up again at this all time high level in the indices.

My reference account is now opened at Striker. Interestingly, due to various delay, the actual trading started from May 1st. It is now trading Big-Game Hunter ES, my first trading strategy offered there. As part of my plan to make things as transparent as possible, the actual statements will be made available for download. It also opened the door for me to openly discuss the trading results without the issue of legitimacy.

I will work with my team to figure out how to offer the viewing of the statements more effectively.

Striker will also have a page in their site reporting the actual results based on the accounts trading Big-Game Hunter ES. I will post the link and other related information by the end of this month.

Next step for this project is to get more trading strategies to go live. So back to development mode and focus on getting these other strategies ready!

Posted on

My Journey to Fully Automate My Trading: Radical Transparency

image

One principle I’ve decided to follow in documenting this journey is Ray Dalio’s radical transparency. To be honest, radical transparency may not be such a good idea for many organizations or businesses. For trading and investing, however, it is a very interest model that can root out, if not all, majority of the human weaknesses, from affecting the performance. But I am not doing this just for performance sake. I am doing it for other reasons too.

What Do I Plan to Make Transparent

I have just arranged with Striker Securities to allow automated trading on one of my trading strategies on restricted number of contracts. And I am signing up with them as the first person to trade it. This will allow me to fully disclose the trading results without affecting my privacy in other areas of my private life. There will also be the third party witness, namely Striker, that can track the overall results of my trading strategies traded through them.

So there is nowhere to hide. I will report the results here regularly. I will be held accountable for what the strategies do in the future.

It will be a really interesting experience as I never need to answer to anyone else in public about my trading results.

What I see is that this process will help me discover better ways to trade even faster because of the accountability I put onto myself.

To Inspire

Many traders choose to trade discretionarily because there is really no other alternatives offered to them.

And they are being put into a path of self-destruction without knowing about it.

By putting myself out there, showing them that there is a way to make trading work for them, I hope I can inspire many aspiring traders to sit down and think about their trading seriously. Maybe there is a better way to learn to trade. Maybe it is time for them to look at trading from a different angle.

My point is simple. It is really not necessary to go through the brutal mental and physical challenges to make trading working for you.

Personal Vendetta

Ever since I “came out” to write about trading many years ago, I was and still am being trashed by trolls and enemies I made in the industry over the years. They could not argue with what I wrote because what I presented to the world has been undeniable evidence on how the markets function. So, they thought they are so clever to go for personal attacks. Little did they know that my experience in trading and every part of the industry is something beyond their wildest imagination.

This time, I am going to take this to their doorstep.

Would they dare to do what I am doing here?

Posted on

My Journey to Fully Automate My Trading: What I Learned After a Month Into Fully Automated Day Trading on TradeStation

image

I still remember the day I launched my automated trading – sitting tight in front of the screen waiting for something to happen. And then suddenly it is already a month since the launch. Time really flies!

Over this month, I actually learned quite a few interesting things that many you will find useful.

TradeStation Second Login

First trouble I ran into was that once the automated trading went live the account is logged in 7/24 from my trading server. So, if I need to develop new strategy or code on TradeStation I have to remote login to the trading server. That is not a good idea because bad coding and human mistakes can interfere with the live automated trading strategies. It turned out TradeStation can provide you with a second login id. The trick is to use this 2nd id only with the sim account so that it does not interfere with the real trading account.

As long as you do not need real-time data on this 2nd login, there is no extra data fees. I now work happily on my laptop with TradeStation this way. It is a very nice discovery.

Fills are Not as Bad as I Thought

Since I am converting my existing models developed on my proprietary platform into TradeStation, I thought latency is going to be an issue. I am pleasantly surprised that it is not the case. Fills are timely and no real slippage.

It makes sense though.

First, the day trading strategies I converted to TradeStation so far are not my HFT ones. They are slower and just need reasonable execution speed. In the past, internet stability and speed were big issues. Hence people like me did highly customized work with co-location servers near / at the exchanges. From the results so far, those concerns are no longer an issue.

Second, infrastructure has improved so much that order routing at retail brokerages are now much faster as well. So, unless you are really into HFT space, retail day trading in this era is pretty level with the institutions.

Latest Generation Computers are Overkill

TradeStation and many other retail trading platforms used to require serious computing power just 5 to 6 years ago. That is no longer true thanks to the improvement in hardware over the past few years.

You can just pick up a cheap 2nd hand computer with Windows 7 Pro for trading for as low as $300-$400 Canadian dollars. It is less than $300 US dollars. The most important thing is to get a quad core CPU so that Windows can run smoothly alongside your trading platforms.

What’s Next

I am converting next set of trading strategies now and plan to incorporate that by end of April. For the first set of trading strategies, I ended up having a complete framework ported over to EasyLanguage. This time, another framework will have to be converted too. I plan to release all these useful code libraries in near future. I just need to figure out how to do it properly.

For example, I don’t see a decent set of functions providing information like holidays and FOMC meeting schedule on any trading platform. But it is critical to know these things even for a normal trader. This is a fundamental tool for building reasonably robust trading strategies yet it is nowhere to be found.

Maybe all the pros are keeping these tools to themselves. Just kidding.

Posted on

My Journey to Fully Automate My Trading: Never Day Trade from Your Laptop

image

One extremely silly thing people do is trading from their laptops. There are so many things that can go wrong with that I seriously wonder if the motto “Hope for the best, prepare for the worst” ever come across the minds of these traders. Wireless connection can drop easily at the wrong moment – when you have a size position on and need to get out, or that you need to get into a position yet your order is not transmitted.

So it is not just an issue for mechanical traders. It is something affecting discretionary traders too. So all these people out there showing themselves trading off their laptops, is really doing the aspiring traders a disservice.

The VPN + Remote Control Workaround

The solution is really simple – have a half decent computer installed at home with a stable Internet connection. Run your trading platform on that computer. Problem solved.

But you want to be able to trade from anywhere instead of being confined to the seat in front of your computer at home. No problem – the trick is to connect your laptop to your home network through VPN and remote control your trading computer.

There are many advantages to trade through this arrangement:

– stable connection means active orders and automated trading strategies will not be affected by your flaky wireless internet connection with your laptop

– you can easily upgrade the setup to a professionally managed server that you lease near the exchange for even better stability and security

– minimize the chance of computer virus and other malwares from infecting your trading applications since they are setup on a clean computer, not your laptop for which you may visit one too many times naughty websites

– you can use any kind of computer or tablet to remote access the trading computer. For example, you may love to use Macbook Air yourself for day to day computing but your trusted trading platform could be Windows based application.

– instability of connection to your trading computer from your remote access will not affect the trading application itself from talking to your brokerage, hence your live orders, especially those target and stop orders have a much better chance to survive

There are several ready-made solutions you can try out.

If you use Windows only computers, you can use its built-in Remote Desktop application to access just your trading computer.

If you like to mix and match, TeamViewer is a functional platform allowing you to do the same.

What I Do

For my own setup, I have chosen a barebone core-i7 Windows 7 Pro computer as the trading computer. The reason why I choose Windows 7 Pro is that it is stable and we can easily stop it from getting new updates from Microsoft. It is very important that a trading computer does not automatically doing whatever it feels like and reboot randomly as if it has a mind of its own. On this front, Microsoft has totally failed itself as a mission critical OS since Windows 8.

On the trading computer, it has absolutely nothing except the trading platforms.

For VPN access, I use Softether VPN because I also like to access all the other computers in my home network. But you can keep it simple with just standard Microsoft Remote Desktop to access the trading computer only by poking a hole to your home network.

I often have to travel and this setup with the trading computer safe and sound in a secure location makes it much easier for me to monitor the automated trading from anywhere in the world.

I am very happy with this setup so far and the trading computer has been running 7/24 with no trouble at all for several weeks already. If you are serious about day trading and mechanical trading, consider doing this as soon as possible. I am sure you will be happy with the results.

Posted on

My Journey to Fully Automate My Trading: Your See a Hammer, I See a Doji

image

I conduct my research on trading strategies on multiple platforms from Excel to almost everything that I can get my hands onto. From concepts to actual strategies, it can be as simple as just several lines of code to thousand lines of code. To say the least, It is a long and tedious process that takes patience. Similar to any other scientific research, even if your method of analysis is rigorous, your analysis is as good as your data only. If the data you get has consistency issues, your analysis will be questionable. And here I am, dealing with exactly that – data inconsistency across various data sources made available to retail traders.

Forex Traders Live With Data Inconsistencies All the Time

This data inconsistency issue is very common with forex trading. In general, the quotes you are getting are just consolidated quotes from several major banks that a brokerage has clearance through. Hence if you switch to a different brokerage that use some other banks for their forex clearance, your quotes would be drastically different. You may wonder why that matters since the price would be moving in the same direction within seconds. After all, due to arbitration opportunities from this type of discrepancies, they will be taken advantage of and quotes across different sources will go back in sync quickly. However, this minor difference can alter the decision making process on hundreds if not thousands of autotrading robots when a binary condition is flipped from one way to another.

For example, an hourly closing price can be very close to the previous day closing price. Many simple trading algorithms and human traders will take the cue very seriously whether such closing price is above or below that reference price level. Drastically different actions would be taken based on that simple logic alone. Hence, many professional traders never change to a different data source for trading because they know very well that such difference can lead to disastrous effect on their trading.

Centralized Trading at Exchanges is Supposed to Avoid this Problem

For index futures like the Emini S&P and stocks, they are traded in various exchanges with centralized bid and ask queues. The quotes are unified with trades recorded in sequence. Ignoring the dark pools which are not part of the normal markets, the centralized trading of these markets is supposed to avoid this inconsistency in data issue completely. In fact, we do get very clean historical data for all these markets going all the way back for at least 20 years.

So the data inconsistency problem should not exist with these markets, shouldn’t it?

Well, that’s not what I find out from my back testing of the same trading strategies across various platforms on Emini S&P.

And after careful analysis of the different versions of the algorithms based on the same logic, I discover the root cause of inconsistency in historical trading performance has nothing to do with my code. Luckily, I have experience is dealing with such issue with my forex trading so it does not affect me much. I am just surprised that the problem of such inconsistency still exists today.

Welcome to the World of Sub-Second Timestamp

On tick data resolution, meaning that every single trade is being compared, there is really ignorable differences among the various sources of historical data. This means the centralized trading at the exchanges in fact unified the data. The problem, interestingly, comes from various implementations of data collection methods into time based records.

The issue really boils down to a simple question – what do you consider as part of a one minute interval?

What I am talking about is how a programmer choose to collect data into 1 minute records can make a big difference in the charts you see everyday.

Conceptually, a minute starts the moment right when the zero second mark hit and the it ends right before the moment the next zero second mark is hit. This is pure science. There is really nothing to dispute about this definition. However, the programmers who are tasked to write the code to record the historical data, may have interpreted the concept of a minute with their personal interpretation.

From what I see, there are programmers who keep all the trades that happen with the zero second time stamp (e.g. 1:00:00) into the current 1 minute record. i.e. 0:59:00 to 1:00:00

However, that is technically wrong because all these trades actually belongs to the next minute. i.e. 1:00:00 to 1:01:00

These programmers do not realize there is no such thing that happens precisely at zero second mark. Anything that is recorded with that time, must have happened after. If they pay attention to the fact that these trades are happening between :00 and :01 with sub-second timestamp. e.g. 0:59:00:30

I can tell you that the most expensive institutional feeds have this done correctly. So are the more expensive retail data feeds that you have to pay money for. But not true for many others platforms out there.

Nightmare of Data Inconsistencies

What does this inconsistency implies?

Well, if you use price patterns on intraday data (and on daily data occasionally too) with your trading strategies, you may get totally different results when you switch from one brokerage to another.

Your moving averages are not really that precise depending on which data source you are looking at.

For some platform, your trading strategies do not execute until 1 second later while the institutional traders have a head start of 1 second before you do.

All sorts of technical indicators including oscillators will suffer from this data inconsistency.

It is not just the close of a minute record that is being affected. If the high or low of the minute happens to be the last second and that your feed includes the extra one second of data, the high and low could be affected too.

A Lesson to Remember for All Traders Not Just the Mechanical Traders

I am not saying that the data inconsistency would render intraday price data analysis completely useless. What I am saying is that you have to be aware of this widespread issue to build robust trading algorithms. And it does not just stop there because this matters much for discretionary traders too.

Many people who have never used multiple brokerages and different data sources have a simplistic worldview of the financial data they are working with. They have no idea that when they see a bottom in the making on their trusted 5-minute bars showing a candlestick hammer is in fact just a doji for thousands of traders out there.

Or that they are getting their favourite moving averages making a turn or crossover is just a kiss between those lines for many other traders out there.

Stop seeing the charts in black and white like that is the first step towards a much more robust approach to chart reading and technical trading.

Posted on

My Journey to Fully Automate My Trading: Peaceful Launch of My Automated Trading on TradeStation

image

This past Monday, Feb 25th, I finally managed to get small part of my automated trading strategies ready for live trading on TradeStation. It was one hell of a long journey to convert my algorithms from my custom trading platforms onto a retail broker platform. I learned a lot from the process and gained incredible insights into some many areas in trading that I thought I already know.

As the Chinese proverbs say,

The journey of a thousand miles begins with one step –  Tao Te Ching, Laozi

千里之行，始於足下 – 道德經, 老子

The moment I decided to move onto this path, I know it is going to be a huge commitment on my part to get to the finish line. So many obstacles and road blocks making this project such a huge challenge. I just looked up my blog posts and found that the earliest post about my intention was back in March 2017. That was two years ago.

I was so overwhelmed by the difficulties that I kind of giving up to write about the progress. There are just so many issues to tackle in various areas of trading making it hard to decide on what to write about. I did not write about it but I do keep notes on everything. That’s my habit all these years. I even recommended the practice to all traders in my writings all the time.

This is not the end of the journey though since my goal is to convert multiple sets of algorithms into live trading strategies that work on various retail platforms so more work has to be done.

I am very glad that I have chosen to walk down this path. At this point of the journey, majority of the technical issues are now taken care of. I am at a happy place to say that the rest of the journey will focus on more strategic development of my automated trading. I will have time freed up to catch up with writing about this amazing experience.

I am setting up a page in my blog for this series finally,

https://thelawrencechan.com/my-journey-to-fully-automate-my-trading/

Enjoy the show!

Posted on

Back to Canada

Aircraft

I came back to Canada last weekend. Lucky that I do not have jet lag since I had to jump on top of several issues that night I landed. Been ultra busy since to catch up with so many things. At 2 am Saturday, this is the first moment I have some quietness to sit down and reflect on my experience from the trip.

What I Did in Asia

For majority of people travelling to Asia from North America during the year end holiday season, they are taking a vacation break. For me, it was essentially 12 hours work day pretty much everyday since the first day my flight landed in Hong Kong. Those exceptions are taking a detour to Singapore and other destinations, attended a wedding and had a special medical check-up.

Outside of trading, I invest in start-ups and do deals that interest me. Many people want to meet with me to see if there is any business opportunities. Others are looking for macro level financial advice from someone who is well connected and really know what is going on around the world. And occasionally, there are others with special reasons that have very interesting stories worth telling when I get the chance.

Going to Singapore to meet with several DaytradingBias.com members was the highlight of the trip. Very happy to see some new faces and those I met with the year before. It is nice to get to know you all better in person!

The Dilemma of Hong Kong

Hong Kong is in a very difficult position now than ever.

The unaffordability of living spaces for the middle to lower income families continue to get worse. Imagine how difficult it is to live in the city when the lower bound of around $18,000 HKD per month for a tiny 200 to 300 sq. ft. apartment in an average location. The same amount of money in Canadian Dollar is about $3,000 which can get you a decent space in a good location for a 1,800 sq. ft. condominium or an even bigger town house. Such compression of available living space has squeezed the life out of the people in Hong Kong. The so called middle class is now long gone.

Yet Hong Kong is still ranking as one of the top cities in the world for business opportunities. However these opportunities may not be accessible easily to those who are struggling at the bottom. Hong Kong is still a good playground for people who has the right connections and the necessary capital.

Everyone there seems to be more sensitive to opportunities in making money than anywhere else I have visited on Earth.

YouTube

I did a pro bono presentation titled “Investment Outlook 2019 according to STOPD” in Hong Kong at the Rotary Club of Kowloon Tong meeting on Jan 7th. It lasts about 2 hours and I did a powerpoint presentation for the event. The participants tried to capture the session in video. However, the voice recording was not as good as we hope for. Lessons learned from the experience and next time I will use a mic for direct recording separately.

The presentation was done in Cantonese so not everyone can understand it. If you want to watch the video (it is just me talking without the powerpoint slides), follow this link.

I am having the powerpoint presentation redesigned for YouTube. Once it is ready it will be uploaded. I will write a short article to explain some of the slides that may not be self explanatory.

If this approach works out well, I will try to do more videos in the future.

Planning Ahead

I have a line up of projects to be completed this first quarter. Talking about being busy in Asia, that was mainly meetings and brainstorming sessions. Here, back home, it is actual work to be done.

The 2nd edition of Trading Success Blueprint was ready …. until it is not. The trouble with the file format issue (Thanks Microsoft!) set me back couple of weeks until the pictures in the manual are restored back to the correct places within the manual. It will be done eventually.

Trading account applications that I thought was taken care of before I leave for Asia turned out to be half done. There is no mistake on my part, just that the brokerage side somehow were in party mode by the end of the year and failed to handle the matters properly. More waiting on this front.

The last pair of VIX based real-time trading signals / day trading strategies will be released for Emini S&P. This is under testing in real-time. So far so good, everything looks on track.

The manual that I expected to publish on these VIX based trading strategies is now going to be split into two manuals as suggested by one of the early readers. I am convinced because he actually did the split up for me and showed me that I was mixing up two different things into one. Namely, there is about 100 to 150 pages of manuscript written on day trading strategy design and principles that has nothing to do with the VIX trading methodology. He suggested that I turn that part into an independent volume with day trading strategies illustrating how it is done with just price data alone.

I think it is a great idea. So back to the drawing board and more work to be done. The good thing is, this additional manual will definitely resolve many issues for traders and giving them a head start in the day trading game. I will make sure it becomes an important trading manual for all Emini traders.

Good to be Home

I was asked how can I take on so many projects at the same time. Well, first, I am not doing all these things by myself. There are many others helping me out making all these things possible.

During the trip, my exercise regime is disrupted and my diet is not optimal to say the least. And I was loaded up with coffee throughout the day making me hyper (as evident in the presentation video). I just slowly re-introduce myself back into my rigorous routines since I have several goals related to health and physical fitness that I like to achieve. Will write about this front when I get the time.

I thought I would have time during the trip to post something. I ended up taking notes that filled one whole notebook with no time to spare. Interesting experience to pick up the pace and doing as much as I can with limited amount of time.

Feels great to be home.

For those of you who celebrate the season, Happy Chinese New Year!

Posted on

When Everybody is Wrong

image

My Big Picture Summary for DaytradingBias.com Market Bias Observer newsletter, Aug 2018 issue. With charts!

The large hedge funds have been underperforming the markets for several years already. They look so bad that many gurus out there came out slamming the industry as a whole. Well, even David Eihorn’s Greenlight has reported his fund is down about 19% for the year. Is the era of big hedge funds really over?

Don’t forget that these funds have all the resources to hire the best to work for them. They have access to information that retail investors and even institutional investors do not have. Yet, the results clearly pointing to something going wrong. So what goes so wrong?

My understanding is that these famous hedge fund managers are actually very good in what they are doing. The main problem is that they all depends on the framework of top-down analysis based on their specific macro view to allocate their resources. If their macro views do not lead to the correct forecast of the market environment, everything they do, will be at best crapshoot. That’s the very essence of trading and speculations. You do not win on every single trade.

image

Since the year 2013 when the S&P 500 is set up to breakout above that long time boundary at 1600 and that Nasdaq 100 making new all time high, we have entered a new era where the prior 20 years of trading environment is transformed into something new. That shift of expectations created a unique situation for which the shorter timeframes have now dominated the price movements of almost every financial market.

image

We have seen such environment before, and the pros who are not close enough to the price actions suffer. For example back in year 1999, majority of the pros working for big firms were laughed at because they cannot make money in the so-called easiest trading environment because the stock market went up pretty much every day. Pros are beaten up so bad back then many even give up their career in the industry.

But we all know what happened afterwards. When the new era eventually run its course, like in year 2000, the normal environment would return. The shift will cause chaos and the pros will be the ones who survive.

I dare to say this, when even the best hedge fund managers are not doing well in shorter timeframes, it is a sign that the bull market is going to end pretty soon.

Posted on

So Many Things Finally Done, So Many More Things to Get Done

top1_investing_amazon

Last few months has been the most productive ones over the past two years. So many projects that have been stuck were finally completed. What a relief!

For one, my book The Art of Chart Reading is now finally published at Amazon. Thanks to everyone who support the book this week, I am proud to announce that, as of this Friday, it is the Top #1 book at Amazon in Investing and Investing Basic categories and also being one of Top 20 Nonfiction books. It is a great start for the book.

The paperback version of the book is still being worked on. If all goes well, the paperback edition will be available by end of this month.

And the site The Art of Chart Reading Online is also in development. It will take a lot more to make it good for the purpose I envisioned.

On DaytradingBias.com side, the move to the new direction has finally started. It was planned at least a year ago but almost everything that could go wrong, did. Although the process has started, it is not even half way through.

It is great that the Trading Success Blueprint is finally completed. It is the grand summary of my knowledge and experience in mentoring traders over the past 20 years. From now on, I have a reference that I can point aspiring traders to if they want to build a trading career the right way.

Yet, the trading signals and trading tools that I have planned to release have been delayed for so long. I could not just roll them out without proper testing. Due to the frequencies of these trading strategies, they do not happen everyday. I need to make sure they are working correctly. It is not only necessary. It is a responsibility as well since people will put money on the line based on these tools. Thus, I have to be patient and get them ready one by one.

On money management side, the process of fully automating the trading has started too. However, it was the most eventful two months we have seen in a long time. Be that North Korea related matters, or all these explosive news about Trump, they swing the markets hard. All these happening while I am manually testing these strategies. It is a huge challenge for someone like me because I created these trading models.

The golden rule of mechanical trading is to follow the rules exactly. But being the creator of the models made me want to override them. After the fact, my interventions worked 9 out of 10 times. But the 10th time is a disaster. In short, if I had followed the rules completely, the performance will be about the same as what I have now. Being 20% better with all the crazy maneuvers is not worth it.

Again, a lot more to do to complete the transition.

oh well, challenges make life interesting.

Posted on

Story of My Book The Art of Chart Reading

Finally, after three long years of wait, my book The Art of Chart Reading will be available at Amazon in couple of days. I am very excited about the release of the book. But at the same time it feels like putting an end to a very difficult period in my past. Very difficult to explain how I feel at this moment.

aocr_3d_small

I have created a website for the book called The Art of Chart Reading Online. The purpose of the site is to showcase charts that I study and how they work out afterwards. Since I do my homework studying the charts all the time anyway. This should not put too much workload on my tight schedule.

This book would not be possible without Priscilla’s suggestion some 15 years ago. She knows very well how I feel about the financial industry and how much I hate seeing decent people being rip off in the stock market. She popped the idea that if I can do something about it what would it be. I told her it has to be a complete instruction manual on how to read the charts because it is the only tool that the retail traders have access to that is objective without spins or lies.

Priscilla became the first reader and my collaborator on the foundation sections. Only until I was able to make her understand exactly how the charts work, that those explanations are kept in the book. That’s how I know other people will understand the framework too.