Tag Archives: brendan conway

ETF “BackTesting” Often = “Over-Fitting”: Is It Bait and Switch?

barrons  Below excerpt courtesy of Brendan Conway’s April 17 Focus on Funds


MarketsMuse Editor Note: Brendan’s article deserves front page focus, but in the process of publishing this piece, a bigger story has emerged and the internet has been overwhelmed by stories that suggest pro-Putin militants in East Ukraine are distributing flyers that purport to come from local government officials with formal announcement that Jews in the city will be required to reqister with the local government, upon which they will be subjected to new taxes or face deportation. MarketsMuse Editorial team says : “Sounds like a Putin-supported strategy intended to cause more chaos, and in turn, provide Putin the perfect storm in which he can defend a larger action on the part of Russia..under the auspices of having to come in and protect Russian (Jews) among others..”

Now on to Brendan Conway’s observations re: “ETF BackTesting”

Brendan Conway
Brendan Conway

“…..It’s negligence, or worse, when an investment manager’s innovative-looking strategy is the result of too much quantitative trial-and-error.

That’s the argument in a notable new study flagged by Stephen Foley of the Financial Times. “Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance” argues that what happens behind the scenes in the development of quantitative strategies is a major problem in investment management.

“Backtest” simply means reviewing historical returns to try to divine how a new strategy might perform in the future. The method has become bread-and-butter in the launch of many new ETFs.

Investors don’t know how many hypotheses managers examined before finding the perfect-looking backtest, a process which turns out to matter greatly, write David H. Bailey, Jonathan M. Borwein, Marcos Lopez de Prado and Qiji Jim Zhu. “The higher the number of configurations tried, the greater is the probability that the backtest is overfit,” they write. “Overfit” means the data has been tortured until it yielded something that looks nice.

If an investment process is driven by what looks good historically, there’s a greater chance the attractive-looking result is just a fluke.

Sure enough, a Vanguard Group study found a while back that backtested ETFs — which look great in the historical data — on average lagged the market after the real-world launch.

From Foley’s discussion: Continue reading

The “ETF bid” and Feedback Loops: Corporate Bond ETFs

Courtesy of Brendan Conway

They’re calling it the “ETF bid” — the idea that corporate bond prices get juiced when passively managed funds have to buy them. It’s known to happen in thinly traded stocks in some instances. So it shouldn’t come as a surprise that thinly traded, idiosyncratic markets like high-yield bonds are seeing a similar effect. It’s the cautionary part of an otherwise pretty encouraging story: ETFs’ power to crack open hard-to-reach asset classes for more investors.

This week’s print Barron’s ETF Focus on the subject concludes that investors should be especially wary of selling passively managed bond funds when markets turn bearish — that’s often the best time to buy. And investors who buy these ETFs when markets feel rosy pay a premium for the service. Obviously, it’s best to avoid paying extra if possible.

Yes, it’s the same old advice, to be a contrarian investor. But ETFs are only growing in importance in the bond markets. The more heavily they are traded, the more investors have to pay attention to their pricing dynamics — and that’s true even for those who don’t use ETFs. If you haven’t read our Barron’s print column, one of the key findings comes from Goldman Sachs’ (GS) Charles P. Himmelberg and Lotfi Karoui. The duo estimate that a monthly rebalanced portfolio of bonds tracked by the iBoxx $ Liquid Investment Grade Index, the benchmark driving the $24.5 billion iBoxx $ Investment Grade Corporate Bond Fund (LQD), has beaten comparable non-indexed bonds by roughly 4.7%, or about 1% a year, since the beginning of 2009.

Great, right? Well, not always. Index bonds also appear to sink harder during bad times, as they did late last year.

In this vein, we wanted to point out a meticulous look at how this works in practice, from TF Market Advisors’ Peter Tchir. Continue reading

Actively Managed ETFs Are Less Volatile, Lipper Finds

Courtesy of  Barron’s Brendan Conway:

By Brendan Conway

Actively managed exchange-traded funds attempt to pick winners much like, say, Bill Miller does in mutual funds. The number of such funds has taken off, and while they’ve tended to underperform versus passive index-tracking ETFs, they’ve also been less volatile.

Those are some of the findings of a new Lipper report by Sasha Franger, the company’s fiduciary research analyst. The group has returned an annualized 2.78% over the last four years, versus 3.20% for “pure” index peers, Franger found. The trend turned in the last year, however: Actively managed ETFs’ performance pulled ahead slightly.

Market gyrations appear to be blunted in actively managed ETFs. This makes intuitive sense: An active manager should be able to pull your assets out of plunging equities or bonds when the environment calls for it. Passive funds can’t.

Median active ETF annual performance ranged from 1.31% to 10.83% for the last four years, while median performance for pure index ETFs has experienced huge swings and has ranged from -40.09% to 55.59%, coinciding with the economic downturn and recovery.

Click here for the full article