“Evaluating backtesting programs: MetaStock, Traders Studio, Ami Broker, Strata Searc, Ninja Trader, Trade Ideas. Any thoughts?”, @milktrader

Source: http://twitter.com/milktrader/status/1456777201

Each of these applications and even others not on this list, like WealthLab Pro or DIY Excel, will have advantages over the other.  Some major, some insignificant. It all depends on the type of trader you are.  If I started praising how good The OddsMaker was, you’d assume I was just trying to sell you something.  Allow me to be more indirect and subtle.  I’ll just list the items you ought to consider when you choosing a backtesting tool.  I’ll let you decide if The OddsMaker is right for you.

1. Which asset classes can be backtested? Equities? Forex? Options? Etc.
2. On what basis of time is historical price information used in the backtest? To the nearest 5 minute candle? 1 minute candle? Time & Sales tick data?
3. Must the trader know the universe of stocks to be backtested a priori? Or can the backtest be applied to a pattern or event (vs. a predetemined stock or basket of stocks)?
4. What windows of time during the market can be backtested? Morning session? Lunch? Afternoon session? Trader-defined periods/Anytime?
5. What options are there for backtesting the exit condition? Price (i.e., stop loss, profit target)? An alert or event?  Time in trade (i.e., a hold for 5min or 5days)?
6. How well does the output inform your next possible trade? (i.e., does it provide a win percentage, net gain amount, average winning trade, average losing trade, confidence score, etc.)?

Ok.  Here is how The OddsMaker would score:

1. Equities, Indices, ETFs. No Forex, no option contracts.
2. Time & Sales tick data
3. No pre-knowledge of specific stocks needed. The OddsMaker is an event-based backtesting tool.
4. Trader-defined periods/Anytime
5. All items mentioned
6. All items mentioned plus a daily detail report

What kind of trader you are depends on how much significance or weight you put on certain decision criteria.  If I had 6 criteria or elements of comparison among backtesting applications and gave you a dollar to spread among them, in what order would you list them with the first item getting the largest share of the dollar and the last item getting the least? Once you do that take each application and score them along each of the elements of comparison.

Now you have a way of scoring each application.

Take the weight of each criteria and multiply it by the score you give that application for each criteria.  Add up your scores and the application with the highest number is your choice.