Out of curiosity, was your vintage testing performed on performance meaning based on net return or was it based on quantity of loans (meaning you completely ignored interest rates and looked at percentage of default/charge off as a percentage of the population)?
Outputs of the back test are WAIR, and IRR, from which one can impute the annual credit loss.
My risk appetite has always been to keep IRR > 2 x credit loss. The idea behind this is that this allows a recession or credit crisis, such as the last one, to roughly triple the loss rate, and I'd still break even. Over time, I've adjusted my filters to make my back test results meet my risk criteria. As loan quality has degraded, my filters look more and more conservative.
In the past, I was able to achieve this 2x objective with IRR > 10% . And ... Actual results matched back test results.
Using recent times for the back test (ie 2015) I can achieve about 7.5% IRR in back test, but my portfolio is now doing substantially worse. (Back test numbers change a bit every month of course, as I download the new loan data files every month)
IRR for the month...
10/2016 6.3%
11/2016 5.41%
12/2016 7.65%
01/2017 3.88%
There is a lot of complexity hidden in dozens of choices one makes about how to do these back test calculations. I earlier pointed out that NSR uses the old interest rates, whereas I use the new interest rates for each grade, because I am interested in how a given filter will perform on loans I select NOW. I'm using the back test process to choose a filter. For my filters, the increase in interest rates from 2015 to present has been 0.64%, so you might expect actual loans from 2015 to do a bit worse than today's back test result. On the other hand, during 2015 my filter was a bit different, and the back test results then, using the actual rates LC applied to those loans, told me to expect > 9%, which hasn't happened by a long shot. In recent times, my experience is that loans have performed worse than the back test told me to expect.
There's a lot of noise in monthly numbers. I expect that the low January number is low mostly because of noise, and not an indication of the latest trend, but we'll have to wait and see. I hope to see the IRRs level off.
Aside: LC's Broad Based Credit fund, which is essentially filterless, ie "broad based", is now earning 0%. They have fallen more than I have.