My Journey to Fully Automate My Trading: What I Learned After a Month Into Fully Automated Day Trading on TradeStation

image

I still remember the day I launched my automated trading – sitting tight in front of the screen waiting for something to happen. And then suddenly it is already a month since the launch. Time really flies!

Over this month, I actually learned quite a few interesting things that many you will find useful.

TradeStation Second Login

First trouble I ran into was that once the automated trading went live the account is logged in 7/24 from my trading server. So, if I need to develop new strategy or code on TradeStation I have to remote login to the trading server. That is not a good idea because bad coding and human mistakes can interfere with the live automated trading strategies. It turned out TradeStation can provide you with a second login id. The trick is to use this 2nd id only with the sim account so that it does not interfere with the real trading account.

As long as you do not need real-time data on this 2nd login, there is no extra data fees. I now work happily on my laptop with TradeStation this way. It is a very nice discovery.

Fills are Not as Bad as I Thought

Since I am converting my existing models developed on my proprietary platform into TradeStation, I thought latency is going to be an issue. I am pleasantly surprised that it is not the case. Fills are timely and no real slippage.

It makes sense though.

First, the day trading strategies I converted to TradeStation so far are not my HFT ones. They are slower and just need reasonable execution speed. In the past, internet stability and speed were big issues. Hence people like me did highly customized work with co-location servers near / at the exchanges. From the results so far, those concerns are no longer an issue.

Second, infrastructure has improved so much that order routing at retail brokerages are now much faster as well. So, unless you are really into HFT space, retail day trading in this era is pretty level with the institutions.

Latest Generation Computers are Overkill

TradeStation and many other retail trading platforms used to require serious computing power just 5 to 6 years ago. That is no longer true thanks to the improvement in hardware over the past few years.

You can just pick up a cheap 2nd hand computer with Windows 7 Pro for trading for as low as $300-$400 Canadian dollars. It is less than $300 US dollars. The most important thing is to get a quad core CPU so that Windows can run smoothly alongside your trading platforms.

What’s Next

I am converting next set of trading strategies now and plan to incorporate that by end of April. For the first set of trading strategies, I ended up having a complete framework ported over to EasyLanguage. This time, another framework will have to be converted too. I plan to release all these useful code libraries in near future. I just need to figure out how to do it properly.

For example, I don’t see a decent set of functions providing information like holidays and FOMC meeting schedule on any trading platform. But it is critical to know these things even for a normal trader. This is a fundamental tool for building reasonably robust trading strategies yet it is nowhere to be found.

Maybe all the pros are keeping these tools to themselves. Just kidding.

My Journey to Fully Automate My Trading: Never Day Trade from Your Laptop

image

One extremely silly thing people do is trading from their laptops. There are so many things that can go wrong with that I seriously wonder if the motto “Hope for the best, prepare for the worst” ever come across the minds of these traders. Wireless connection can drop easily at the wrong moment – when you have a size position on and need to get out, or that you need to get into a position yet your order is not transmitted.

So it is not just an issue for mechanical traders. It is something affecting discretionary traders too. So all these people out there showing themselves trading off their laptops, is really doing the aspiring traders a disservice.

The VPN + Remote Control Workaround

The solution is really simple – have a half decent computer installed at home with a stable Internet connection. Run your trading platform on that computer. Problem solved.

But you want to be able to trade from anywhere instead of being confined to the seat in front of your computer at home. No problem – the trick is to connect your laptop to your home network through VPN and remote control your trading computer.

There are many advantages to trade through this arrangement:

– stable connection means active orders and automated trading strategies will not be affected by your flaky wireless internet connection with your laptop

– you can easily upgrade the setup to a professionally managed server that you lease near the exchange for even better stability and security

– minimize the chance of computer virus and other malwares from infecting your trading applications since they are setup on a clean computer, not your laptop for which you may visit one too many times naughty websites

– you can use any kind of computer or tablet to remote access the trading computer. For example, you may love to use Macbook Air yourself for day to day computing but your trusted trading platform could be Windows based application.

– instability of connection to your trading computer from your remote access will not affect the trading application itself from talking to your brokerage, hence your live orders, especially those target and stop orders have a much better chance to survive

There are several ready-made solutions you can try out.

If you use Windows only computers, you can use its built-in Remote Desktop application to access just your trading computer.

If you like to mix and match, TeamViewer is a functional platform allowing you to do the same.

What I Do

For my own setup, I have chosen a barebone core-i7 Windows 7 Pro computer as the trading computer. The reason why I choose Windows 7 Pro is that it is stable and we can easily stop it from getting new updates from Microsoft. It is very important that a trading computer does not automatically doing whatever it feels like and reboot randomly as if it has a mind of its own. On this front, Microsoft has totally failed itself as a mission critical OS since Windows 8.

On the trading computer, it has absolutely nothing except the trading platforms.

For VPN access, I use Softether VPN because I also like to access all the other computers in my home network. But you can keep it simple with just standard Microsoft Remote Desktop to access the trading computer only by poking a hole to your home network.

I often have to travel and this setup with the trading computer safe and sound in a secure location makes it much easier for me to monitor the automated trading from anywhere in the world.

I am very happy with this setup so far and the trading computer has been running 7/24 with no trouble at all for several weeks already. If you are serious about day trading and mechanical trading, consider doing this as soon as possible. I am sure you will be happy with the results.

My Journey to Fully Automate My Trading: Your See a Hammer, I See a Doji

image

I conduct my research on trading strategies on multiple platforms from Excel to almost everything that I can get my hands onto. From concepts to actual strategies, it can be as simple as just several lines of code to thousand lines of code. To say the least, It is a long and tedious process that takes patience. Similar to any other scientific research, even if your method of analysis is rigorous, your analysis is as good as your data only. If the data you get has consistency issues, your analysis will be questionable. And here I am, dealing with exactly that – data inconsistency across various data sources made available to retail traders.

Forex Traders Live With Data Inconsistencies All the Time

This data inconsistency issue is very common with forex trading. In general, the quotes you are getting are just consolidated quotes from several major banks that a brokerage has clearance through. Hence if you switch to a different brokerage that use some other banks for their forex clearance, your quotes would be drastically different. You may wonder why that matters since the price would be moving in the same direction within seconds. After all, due to arbitration opportunities from this type of discrepancies, they will be taken advantage of and quotes across different sources will go back in sync quickly. However, this minor difference can alter the decision making process on hundreds if not thousands of autotrading robots when a binary condition is flipped from one way to another.

For example, an hourly closing price can be very close to the previous day closing price. Many simple trading algorithms and human traders will take the cue very seriously whether such closing price is above or below that reference price level. Drastically different actions would be taken based on that simple logic alone. Hence, many professional traders never change to a different data source for trading because they know very well that such difference can lead to disastrous effect on their trading.

Centralized Trading at Exchanges is Supposed to Avoid this Problem

For index futures like the Emini S&P and stocks, they are traded in various exchanges with centralized bid and ask queues. The quotes are unified with trades recorded in sequence. Ignoring the dark pools which are not part of the normal markets, the centralized trading of these markets is supposed to avoid this inconsistency in data issue completely. In fact, we do get very clean historical data for all these markets going all the way back for at least 20 years.

So the data inconsistency problem should not exist with these markets, shouldn’t it?

Well, that’s not what I find out from my back testing of the same trading strategies across various platforms on Emini S&P.

And after careful analysis of the different versions of the algorithms based on the same logic, I discover the root cause of inconsistency in historical trading performance has nothing to do with my code. Luckily, I have experience is dealing with such issue with my forex trading so it does not affect me much. I am just surprised that the problem of such inconsistency still exists today.

Welcome to the World of Sub-Second Timestamp

On tick data resolution, meaning that every single trade is being compared, there is really ignorable differences among the various sources of historical data. This means the centralized trading at the exchanges in fact unified the data. The problem, interestingly, comes from various implementations of data collection methods into time based records.

The issue really boils down to a simple question – what do you consider as part of a one minute interval?

What I am talking about is how a programmer choose to collect data into 1 minute records can make a big difference in the charts you see everyday.

Conceptually, a minute starts the moment right when the zero second mark hit and the it ends right before the moment the next zero second mark is hit. This is pure science. There is really nothing to dispute about this definition. However, the programmers who are tasked to write the code to record the historical data, may have interpreted the concept of a minute with their personal interpretation.

From what I see, there are programmers who keep all the trades that happen with the zero second time stamp (e.g. 1:00:00) into the current 1 minute record. i.e. 0:59:00 to 1:00:00

However, that is technically wrong because all these trades actually belongs to the next minute. i.e. 1:00:00 to 1:01:00

These programmers do not realize there is no such thing that happens precisely at zero second mark. Anything that is recorded with that time, must have happened after. If they pay attention to the fact that these trades are happening between :00 and :01 with sub-second timestamp. e.g. 0:59:00:30

I can tell you that the most expensive institutional feeds have this done correctly. So are the more expensive retail data feeds that you have to pay money for. But not true for many others platforms out there.

Nightmare of Data Inconsistencies

What does this inconsistency implies?

Well, if you use price patterns on intraday data (and on daily data occasionally too) with your trading strategies, you may get totally different results when you switch from one brokerage to another.

Your moving averages are not really that precise depending on which data source you are looking at.

For some platform, your trading strategies do not execute until 1 second later while the institutional traders have a head start of 1 second before you do.

All sorts of technical indicators including oscillators will suffer from this data inconsistency.

It is not just the close of a minute record that is being affected. If the high or low of the minute happens to be the last second and that your feed includes the extra one second of data, the high and low could be affected too.

A Lesson to Remember for All Traders Not Just the Mechanical Traders

I am not saying that the data inconsistency would render intraday price data analysis completely useless. What I am saying is that you have to be aware of this widespread issue to build robust trading algorithms. And it does not just stop there because this matters much for discretionary traders too.

Many people who have never used multiple brokerages and different data sources have a simplistic worldview of the financial data they are working with. They have no idea that when they see a bottom in the making on their trusted 5-minute bars showing a candlestick hammer is in fact just a doji for thousands of traders out there.

Or that they are getting their favourite moving averages making a turn or crossover is just a kiss between those lines for many other traders out there.

Stop seeing the charts in black and white like that is the first step towards a much more robust approach to chart reading and technical trading.

My Journey to Fully Automate My Trading: Peaceful Launch of My Automated Trading on TradeStation

image

This past Monday, Feb 25th, I finally managed to get small part of my automated trading strategies ready for live trading on TradeStation. It was one hell of a long journey to convert my algorithms from my custom trading platforms onto a retail broker platform. I learned a lot from the process and gained incredible insights into some many areas in trading that I thought I already know.

As the Chinese proverbs say,

The journey of a thousand miles begins with one step –  Tao Te Ching, Laozi

千里之行,始於足下 – 道德經, 老子

The moment I decided to move onto this path, I know it is going to be a huge commitment on my part to get to the finish line. So many obstacles and road blocks making this project such a huge challenge. I just looked up my blog posts and found that the earliest post about my intention was back in March 2017. That was two years ago.

I was so overwhelmed by the difficulties that I kind of giving up to write about the progress. There are just so many issues to tackle in various areas of trading making it hard to decide on what to write about. I did not write about it but I do keep notes on everything. That’s my habit all these years. I even recommended the practice to all traders in my writings all the time.

This is not the end of the journey though since my goal is to convert multiple sets of algorithms into live trading strategies that work on various retail platforms so more work has to be done.

I am very glad that I have chosen to walk down this path. At this point of the journey, majority of the technical issues are now taken care of. I am at a happy place to say that the rest of the journey will focus on more strategic development of my automated trading. I will have time freed up to catch up with writing about this amazing experience.

I am setting up a page in my blog for this series finally,

http://thelawrencechan.com/my-journey-to-fully-automate-my-trading/

Enjoy the show!

Back to Canada

Aircraft

I came back to Canada last weekend. Lucky that I do not have jet lag since I had to jump on top of several issues that night I landed. Been ultra busy since to catch up with so many things. At 2 am Saturday, this is the first moment I have some quietness to sit down and reflect on my experience from the trip.

What I Did in Asia

For majority of people travelling to Asia from North America during the year end holiday season, they are taking a vacation break. For me, it was essentially 12 hours work day pretty much everyday since the first day my flight landed in Hong Kong. Those exceptions are taking a detour to Singapore and other destinations, attended a wedding and had a special medical check-up.

Outside of trading, I invest in start-ups and do deals that interest me. Many people want to meet with me to see if there is any business opportunities. Others are looking for macro level financial advice from someone who is well connected and really know what is going on around the world. And occasionally, there are others with special reasons that have very interesting stories worth telling when I get the chance.

Going to Singapore to meet with several DaytradingBias.com members was the highlight of the trip. Very happy to see some new faces and those I met with the year before. It is nice to get to know you all better in person!

The Dilemma of Hong Kong

Hong Kong is in a very difficult position now than ever.

The unaffordability of living spaces for the middle to lower income families continue to get worse. Imagine how difficult it is to live in the city when the lower bound of around $18,000 HKD per month for a tiny 200 to 300 sq. ft. apartment in an average location. The same amount of money in Canadian Dollar is about $3,000 which can get you a decent space in a good location for a 1,800 sq. ft. condominium or an even bigger town house. Such compression of available living space has squeezed the life out of the people in Hong Kong. The so called middle class is now long gone.

Yet Hong Kong is still ranking as one of the top cities in the world for business opportunities. However these opportunities may not be accessible easily to those who are struggling at the bottom. Hong Kong is still a good playground for people who has the right connections and the necessary capital.

Everyone there seems to be more sensitive to opportunities in making money than anywhere else I have visited on Earth.

YouTube

I did a pro bono presentation titled “Investment Outlook 2019 according to STOPD” in Hong Kong at the Rotary Club of Kowloon Tong meeting on Jan 7th. It lasts about 2 hours and I did a powerpoint presentation for the event. The participants tried to capture the session in video. However, the voice recording was not as good as we hope for. Lessons learned from the experience and next time I will use a mic for direct recording separately.

The presentation was done in Cantonese so not everyone can understand it. If you want to watch the video (it is just me talking without the powerpoint slides), follow this link.

I am having the powerpoint presentation redesigned for YouTube. Once it is ready it will be uploaded. I will write a short article to explain some of the slides that may not be self explanatory.

If this approach works out well, I will try to do more videos in the future.

Planning Ahead

I have a line up of projects to be completed this first quarter. Talking about being busy in Asia, that was mainly meetings and brainstorming sessions. Here, back home, it is actual work to be done.

The 2nd edition of Trading Success Blueprint was ready …. until it is not. The trouble with the file format issue (Thanks Microsoft!) set me back couple of weeks until the pictures in the manual are restored back to the correct places within the manual. It will be done eventually.

Trading account applications that I thought was taken care of before I leave for Asia turned out to be half done. There is no mistake on my part, just that the brokerage side somehow were in party mode by the end of the year and failed to handle the matters properly. More waiting on this front.

The last pair of VIX based real-time trading signals / day trading strategies will be released for Emini S&P. This is under testing in real-time. So far so good, everything looks on track.

The manual that I expected to publish on these VIX based trading strategies is now going to be split into two manuals as suggested by one of the early readers. I am convinced because he actually did the split up for me and showed me that I was mixing up two different things into one. Namely, there is about 100 to 150 pages of manuscript written on day trading strategy design and principles that has nothing to do with the VIX trading methodology. He suggested that I turn that part into an independent volume with day trading strategies illustrating how it is done with just price data alone.

I think it is a great idea. So back to the drawing board and more work to be done. The good thing is, this additional manual will definitely resolve many issues for traders and giving them a head start in the day trading game. I will make sure it becomes an important trading manual for all Emini traders.

Good to be Home

I was asked how can I take on so many projects at the same time. Well, first, I am not doing all these things by myself. There are many others helping me out making all these things possible.

During the trip, my exercise regime is disrupted and my diet is not optimal to say the least. And I was loaded up with coffee throughout the day making me hyper (as evident in the presentation video). I just slowly re-introduce myself back into my rigorous routines since I have several goals related to health and physical fitness that I like to achieve. Will write about this front when I get the time.

I thought I would have time during the trip to post something. I ended up taking notes that filled one whole notebook with no time to spare. Interesting experience to pick up the pace and doing as much as I can with limited amount of time.

Feels great to be home.

For those of you who celebrate the season, Happy Chinese New Year!

When Everybody is Wrong

image

My Big Picture Summary for DaytradingBias.com Market Bias Observer newsletter, Aug 2018 issue. With charts!

The large hedge funds have been underperforming the markets for several years already. They look so bad that many gurus out there came out slamming the industry as a whole. Well, even David Eihorn’s Greenlight has reported his fund is down about 19% for the year. Is the era of big hedge funds really over?

Don’t forget that these funds have all the resources to hire the best to work for them. They have access to information that retail investors and even institutional investors do not have. Yet, the results clearly pointing to something going wrong. So what goes so wrong?

My understanding is that these famous hedge fund managers are actually very good in what they are doing. The main problem is that they all depends on the framework of top-down analysis based on their specific macro view to allocate their resources. If their macro views do not lead to the correct forecast of the market environment, everything they do, will be at best crapshoot. That’s the very essence of trading and speculations. You do not win on every single trade.

image

Since the year 2013 when the S&P 500 is set up to breakout above that long time boundary at 1600 and that Nasdaq 100 making new all time high, we have entered a new era where the prior 20 years of trading environment is transformed into something new. That shift of expectations created a unique situation for which the shorter timeframes have now dominated the price movements of almost every financial market.

image

We have seen such environment before, and the pros who are not close enough to the price actions suffer. For example back in year 1999, majority of the pros working for big firms were laughed at because they cannot make money in the so-called easiest trading environment because the stock market went up pretty much every day. Pros are beaten up so bad back then many even give up their career in the industry.

But we all know what happened afterwards. When the new era eventually run its course, like in year 2000, the normal environment would return. The shift will cause chaos and the pros will be the ones who survive.

I dare to say this, when even the best hedge fund managers are not doing well in shorter timeframes, it is a sign that the bull market is going to end pretty soon.

So Many Things Finally Done, So Many More Things to Get Done

top1_investing_amazon

Last few months has been the most productive ones over the past two years. So many projects that have been stuck were finally completed. What a relief!

For one, my book The Art of Chart Reading is now finally published at Amazon. Thanks to everyone who support the book this week, I am proud to announce that, as of this Friday, it is the Top #1 book at Amazon in Investing and Investing Basic categories and also being one of Top 20 Nonfiction books. It is a great start for the book.

The paperback version of the book is still being worked on. If all goes well, the paperback edition will be available by end of this month.

And the site The Art of Chart Reading Online is also in development. It will take a lot more to make it good for the purpose I envisioned.

On DaytradingBias.com side, the move to the new direction has finally started. It was planned at least a year ago but almost everything that could go wrong, did. Although the process has started, it is not even half way through.

It is great that the Trading Success Blueprint is finally completed. It is the grand summary of my knowledge and experience in mentoring traders over the past 20 years. From now on, I have a reference that I can point aspiring traders to if they want to build a trading career the right way.

Yet, the trading signals and trading tools that I have planned to release have been delayed for so long. I could not just roll them out without proper testing. Due to the frequencies of these trading strategies, they do not happen everyday. I need to make sure they are working correctly. It is not only necessary. It is a responsibility as well since people will put money on the line based on these tools. Thus, I have to be patient and get them ready one by one.

On money management side, the process of fully automating the trading has started too. However, it was the most eventful two months we have seen in a long time. Be that North Korea related matters, or all these explosive news about Trump, they swing the markets hard. All these happening while I am manually testing these strategies. It is a huge challenge for someone like me because I created these trading models.

The golden rule of mechanical trading is to follow the rules exactly. But being the creator of the models made me want to override them. After the fact, my interventions worked 9 out of 10 times. But the 10th time is a disaster. In short, if I had followed the rules completely, the performance will be about the same as what I have now. Being 20% better with all the crazy maneuvers is not worth it.

Again, a lot more to do to complete the transition.

oh well, challenges make life interesting.

Story of My Book The Art of Chart Reading

Finally, after three long years of wait, my book The Art of Chart Reading will be available at Amazon in couple of days. I am very excited about the release of the book. But at the same time it feels like putting an end to a very difficult period in my past. Very difficult to explain how I feel at this moment.

aocr_3d_small

I have created a website for the book called The Art of Chart Reading Online. The purpose of the site is to showcase charts that I study and how they work out afterwards. Since I do my homework studying the charts all the time anyway. This should not put too much workload on my tight schedule.

This book would not be possible without Priscilla’s suggestion some 15 years ago. She knows very well how I feel about the financial industry and how much I hate seeing decent people being rip off in the stock market. She popped the idea that if I can do something about it what would it be. I told her it has to be a complete instruction manual on how to read the charts because it is the only tool that the retail traders have access to that is objective without spins or lies.

Priscilla became the first reader and my collaborator on the foundation sections. Only until I was able to make her understand exactly how the charts work, that those explanations are kept in the book. That’s how I know other people will understand the framework too.

When Card Counting Fails: Learn to Walk Away While You Can

roulette2

When I was in Europe last month, it was inevitable that I had to go out and socialize with my business associates. One such night we ended up going to a club with blackjack tables. One of my friends there knows very well I am an avid card counter and would like to see if I can beat the house. So there I was, sitting at one of the blackjack tables with couple hundreds of euro in chips to play the game.

The Game

I asked the dealer like how many decks are there and started playing.

I did my usual counting and noticed that the cards dealt out were collected and put back into the card dealing machine. I knew something was not right but I could not recall exactly what it was at the time, partly due to all night wine and dine activities.

Needless to say, I got cleaned out.

Whenever I tried to increase my bet size, thinking that it was the right time to do so, I encountered significant losses.

The chance for me to count wrong was very low. So low that I knew something did not add up. After I lost my last chips, I recalled exactly what I should have remembered before I started playing.

My associates asked me to keep on going so that I can win the money back. I told them I don’t want to do that because I was low on cash. In reality I had more than enough cash to keep going but I knew better not to. My friends even tried to pool together more cash for me to do it again because they probably feel bad that I lost money.

I ended up telling them the truth. I pointed out the machine on the table is something I cannot beat. Since I do not like playing blackjack to lose money, I told them we should do something else instead. They were quite upset and were afraid that I was unhappy. I had to assure and reassure them that I am perfectly alright.

I was secretly relieved that I did not need to grind at the table. I was happy because I knew this is the right decision.

The Machine

The issue I pointed out being the card dealing machine was not an excuse. This type of card dealing machines is designed to save time for the dealer, prevent cheating through manual card shuffling and, most important of all, to beat card counting. Cards are consistently redistributed back into the shoe hence the chance of these cards showing up again, say, five hands later, would be the same as all the other cards within the machine. There is no more distribution imbalance that a card counter can rely on. Thus, all card counting strategies based on the cards not being reused until next complete reshuffle would fail.

I should have walked away the moment I sit down at the table and noticed that the machine is in place for dealing cards. But I did not.

I suspect that my inability to recall the important fact about the card dealing machine is my commitment bias. I changed my Euros for chips. I sat down at the table. And most important of all, I have a crowd around me watching. My mind swing from being a careful pro into a casual player.

The Crowd

Emotions ran high when the crowd saw the dealer’s hand busted at the fourth and fifth card. Emotions ran even higher when someone on the table getting 21 on the fifth card.

Of course, emotions swing low when the dealer managed to clean the table with 20 or 21 even though many players had pretty strong hands.

Everyone has been participating in the game, with emotions invested into every card being dealt out.

I could see on the face of everyone that they were emotionally exhausted about an hour into the game.

Isn’t it interesting that we all know this is a game of chance yet we would cheer on wishing the dealer to go bust, or that some players were so anxious that it took them forever to decide whether to hit or to stand?

Maybe this is the very reason some people love gaming.

They enjoy the emotional rides and excitement that these games bring to them.

The Odds

Going a bit technical here, I will explain what the difference is between a normal Las Vegas style blackjack and the one I played with the card dealing machine.

In short, the Las Vegas style dealing will give the house a very slight edge over the players if the player is not counting cards, while exercising all normal cautions like not to hit when you cards already total at 20. With a good card counting strategy, a player can, at times, having a significant edge over the house when many high cards are left in the shoe leading to very uneven distribution.

For the card dealing machine I played, it actually offers the players a very minute edge, as long as the player is exercising the normal cautions. This means, if you have a big enough bank roll, you can win a small percentage of your capital as long as you are betting with the same size. Obviously, the goal of using these machines is not to beat the players. These machines are installed for the purpose of facilitating a service, to keep the guests entertained.

However, if a card counter is not aware of the design, the uneven betting based on the wrong projection of the existence of a bias distribution, will lead to unnecessary losses.

The Lesson

It is a good reminder to us, the professional traders, that we manage to make money in trading because we have good strategies and game plan in place. However, we have to be aware of the changes to the game itself that may completely alter the odds of our trading strategies. Keeping up with the environment change is very much part of our responsibility as a pro trader.

Things I Learned from My Hard Disk Trouble While Traveling

image

My hard disk trouble started in the early part of February yet I was not able to get it replaced for various reasons. Anyhow, I finally have it replaced by a new hard drive and everything looks fine so far. Throughout this ordeal, I learned quite a few things that’s worth sharing.

Must Have Items While Traveling

Portable Hard Drive is a must-have for quick backup and restore. Without one, trying to buy a decent one with large capacity from a remote city may be very difficult. I have a 3TB one with me all the time while I am traveling. Even if my laptop is reporting a potential failure, I can immediately backup everything so that I know all the latest work I have done with my projects and researches are safe.

USB Boot Drive is another must-have since you cannot really backup your computer efficiently without it. The one I use is the one I recommended for years called Active Boot Disk. I use it to backup my computers regularly and in this situation, I managed to take a snapshot of my laptop the night when I first see the warning message.

With the first two items above, the issue of protecting your valuable data is taken care of. Even if your laptop is toasted while you were seeking for a replacement hard disk, you can still get things working on a new laptop.

To replace the hard disk inside your laptop is a more complicated question depending heavily if you are going to let a computer shop does the replacement for you, or, your laptop can be easily taken apart. Well, I have way too many bad experience with computer stores repair services that I almost always do my own upgrades and replacements now. Even if you are not good with handling computer parts, you should be able to find disassembly videos on YouTube on almost all types of laptops.

You need the following items to complete the hard disk replacement process:

A screwdriver set that can handle all the screw types on your laptop. In my case, my trusted 30+ years old Swiss Army Knife does not work with my Lenovo laptop. I was forced to buy a screwdriver set at the computer store eventually. The story about the computer store in next section.

A replacement hard disk that is at least equal in capacity as your original hard disk.

Finding a Replacement Hard Disk is an Adventure

It took me two weeks in three cities to find a computer store that actually sell decent size internal hard disk. The first two cities are not major ones so my hope was kept really low. But then the third one, being a large city, it still took me multiple attempts before finding a computer store that really sells slightly better hard disk models.

Maybe it is because I am in Europe, but it seems that all these stores are selling both external and internal hard drives at half the capacity we find in America. They are also priced more expensively. The funny thing is that in a big retail electronics store I visited with all kinds of computer related products, their sales people do not even know where the portable external hard drives are located within the store. They have to search the store for like ten minutes before finding the items.

I seriously do not know what to say about that.

Plan Your Backup

It took about 6 hours to backup my laptop with 400G of data. The time it takes is much longer than a typical laptop with same amount of data because I have a lot of highly compressed tick data stored on my laptop. It is very time consuming to backup such data because they cannot be compressed any further by the backup application.

Knowing the amount of time it takes to backup can help you plan ahead of time to schedule a down time for your laptop so that it will not disrupt your normal work schedule. What I did was starting the backup late in the night so that early next morning I can continue with the installation of the new hard disk with restoration right after. Just in case that does not work out as planned, I would quickly swap the old drive back into the laptop to continue my day.

Restoring the hard disk image onto the laptop with the new hard disk is actually a breeze. All it took was just an hour and 20 minutes. Once it was done I just shutdown the laptop completely. When I turned it back on, it was able to boot normally and everything feels the same. Well, in my case, the replacement hard disk is a faster model than the one that came with my laptop so everything actually run much faster.

Lessons Learned

1. Have a traveller set of computer screwdrivers while I am on the road.

You never know when you are going to need to take your computer apart. You do not need to carry the whole set. Just bring the bits for your devices and the handle will do.

2. Do not use SSD on your laptop.

SSD has great feedback from all computer users due to their faster speed. They are also perceived to be safer than the regular hard drives because they do not have moving parts. In reality, however, SSD drives are more likely to fail and have bad blocks developed quickly within first 2 years of usage. Yep, you may suddenly lose some part of your important files on an SSD without knowing it has already happened.

In comparison, the classic hard drives have come a long way as manufacturers of these drives have keep advancing the technology. In my situation, I am very glad that I chose my laptop with a hard drive instead of SSD. My decision at the time was based on the capacity that I need which was not available with SSD. I do not even know that the smart warning ability of the regular hard drives until I get one in this incident.

This smart warning ability gave me the precious time window to replace the drive with ease.

When SSDs eventually fail, they just die on you and you will not get the chance to backup your data first. Hence you will lose your latest work on your laptop one day no matter how frequent you backup your data.

3. Do not use SSD external drives as your backup drive.

For the same reason above, if your data is important to you, you should keep a regular hard drive as your backup drive.

I am sure that going into the future, the SSD technology will improve a lot just like the regular hard drives with better safe guard for our data. Until then, I will avoid SSD on my laptops.

4. Know your USB 3.0 Port on the laptop

The latest external backup drives use the USB 3.0 port for much faster performance. On many laptops, however, not every USB port are USB 3.0 port. Plugging in your external drive to the wrong port, will result in much slower data transfer speed. For example, my backup time of 6 hours will likely turn into 12 hours or more if I plug my backup drive to the wrong USB port on my laptop.

I made this mistake before with my regular backup routine. You can imagine what happen the next morning when I woke up seeing my laptop is not even half way through the backup process …

End Notes

It is not my first time seeing my laptops getting into trouble while I am on the road. To name a few, I have experienced RAM failure, overheating that killed the screen on the spot and bad electricity outlet that fried the power adaptor and the laptop (yes, fire too for those looking for excitement). There are many blizzard situations that we road warriors just cannot be prepared for. My advice is that have the essentials packed with you but don’t panic when things go wrong. There is no point to get upset since computer failure is part of life.

Deal with it and you have another story to tell your friends.

Products Mentioned in this Article

Western Digital 3TB Portable External Hard Drive

List of Western Digital Portable External Hard Drive on Amazon

Active Boot Disk

Portable Screwdriver Set for Electronic Devices