Author Topic: What's This "Best Note" Selection Business Anyway?  (Read 38191 times)

dontvote

  • Full Member
  • ***
  • Posts: 226
    • View Profile
    • Email
Re: What's This "Best Note" Selection Business Anyway?
« Reply #30 on: January 16, 2014, 12:10:13 PM »
You're too kind! That's one explanation, but keep in mind I may be totally wrong ;-)
Of course your example may be wrong, but there are any number of other similar divergences of goals that are possible. I just didn't see that.
For the sake or argument lets say your argument is right. Does that mean some borrowers get better deals than others simply for the sake of consistency optimization?
The borrowers that get the bad deals relatively speaking are the loans we seek. Or, did I miss the point completely?

You're exactly right. If risk were priced perfectly, everyone would have a custom interest rate and there would be very little difference between holding any loans across risk categories. Yes, some borrowers are 'overpaying' given their real risk parameters making them 'safer' than their interest rates. Those are the 'better' loans everyone seeks.
dontvote

Interest Radar IRR: 46.11%
NSR ROI: 41.22%
Average Loan Age: Your Moms

Rob L

  • Hero Member
  • *****
  • Posts: 2119
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #31 on: January 19, 2014, 02:16:03 AM »
I guess it's possible to create a model better than LC's given the data we have or can get, but I don't see large obvious factors persisting over time.

Think I found an example of this while poking around on IR.
Loans are 36 month only.

For all loan grades between 2008-10 and 2010-12:
Inquires       # of Loans             Average Interest Rate               Avg IRR
      0                    6512                              11.3%                                 5.9%
      1                    3771                              11.6%                                 4.3%
      2                    1990                              11.7%                                 4.4%
      3                    1126                              12.0%                                 0.7%

For all loan grades between 2010-12 and 2014-01:
Inquires       # of Loans             Average Interest Rate               Avg IRR
      0                   79836                             12.0%                                  6.6%
      1                   42979                             13.5%                                  6.8%
      2                   19727                             14.2%                                  6.1%
      3                    8866                              14.7%                                  4.8%

In the earlier sample set LC did not increase the borrowers interest rate in response to an increased number of inquires.  As a result the lenders IRR fell sharply as inquires increased. All of these loans have been fully paid or charged off.

In the second sample set it appears LC updated its model to recognize increased risk of default when inquires are >0 and charge the borrower a higher interest rate accordingly. Note that the IRR for inquires = 1 is actually > than inquires = 0 now.  At first blush it would appear that the LC model now fully compensates the lender for multiple inquires (at least 1-2) so filtering for inquires=0 is not helpful and possibly counterproductive. Most of these loans are not yet mature so all the IRR's are higher than the older set.

From a little reading it seems number of inquires has long been recognized as a significant consumer credit default risk factor. I have no idea why LC may have left it out or under weighted it in earlier models. Makes me think I've probably got it wrong since I have so little experience with this stuff.
« Last Edit: January 19, 2014, 09:56:04 AM by Rob L »

Ran

  • Full Member
  • ***
  • Posts: 148
    • View Profile
    • Email
Re: What's This "Best Note" Selection Business Anyway?
« Reply #32 on: January 19, 2014, 10:35:13 AM »
It's not hard to beat analysts. In fact, most don't even know what the heck they are talking about. If they were smart enough to accurately predict where a stock would go, they would be fund managers.

If they can repeatedly and accurately pick where any stock or set of stocks would go, they would be billionaires.  The number of funds that have been able to produce any alpha whatsoever on a consistent basis for more than 10 years can be counted on one hand.

Think about it like this - how many funds underperform their index every year?  If you have a thousand funds and you assume it's completely random, you might expect 500 outperform the first year, of which 250 the second year, 125 the third year, 62 the fourth year, 31 the fifth year, 16 the sixth year, 8 the seventh year, 4 the eighth year, 2 the ninth year, and 1 the tenth year.  In reality though, 80% of fund managers on average underperform their index so the odds are worse.

You are missing a point there. It's true that funds generally underperform the market in the long run. However, we are talking about "market" here. In LC's case, LC is not THE market, and they are a better competitor in the best case. So LC is expected to underperform THE market and in some sectors of loans, one can expect some market participants to beat LC.

Randawl

  • Sr. Member
  • ****
  • Posts: 469
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #33 on: January 19, 2014, 11:00:38 AM »
I guess it's possible to create a model better than LC's given the data we have or can get, but I don't see large obvious factors persisting over time.

Think I found an example of this while poking around on IR.
Loans are 36 month only.

For all loan grades between 2008-10 and 2010-12:
Inquires       # of Loans             Average Interest Rate               Avg IRR
      0                    6512                              11.3%                                 5.9%
      1                    3771                              11.6%                                 4.3%
      2                    1990                              11.7%                                 4.4%
      3                    1126                              12.0%                                 0.7%

For all loan grades between 2010-12 and 2014-01:
Inquires       # of Loans             Average Interest Rate               Avg IRR
      0                   79836                             12.0%                                  6.6%
      1                   42979                             13.5%                                  6.8%
      2                   19727                             14.2%                                  6.1%
      3                    8866                              14.7%                                  4.8%

In the earlier sample set LC did not increase the borrowers interest rate in response to an increased number of inquires.  As a result the lenders IRR fell sharply as inquires increased. All of these loans have been fully paid or charged off.

In the second sample set it appears LC updated its model to recognize increased risk of default when inquires are >0 and charge the borrower a higher interest rate accordingly. Note that the IRR for inquires = 1 is actually > than inquires = 0 now.  At first blush it would appear that the LC model now fully compensates the lender for multiple inquires (at least 1-2) so filtering for inquires=0 is not helpful and possibly counterproductive. Most of these loans are not yet mature so all the IRR's are higher than the older set.

From a little reading it seems number of inquires has long been recognized as a significant consumer credit default risk factor. I have no idea why LC may have left it out or under weighted it in earlier models. Makes me think I've probably got it wrong since I have so little experience with this stuff.

Interesting data.  Just wanted to point out, FYI, that because no loss factor is included with the calculation of the second data set (and since they're so young, the riskier ones appear more inflated than others), those with increasing inquiries may or may not have had enough time to differentiate themselves and increase their default rates as they age.

Rob L

  • Hero Member
  • *****
  • Posts: 2119
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #34 on: January 19, 2014, 12:21:00 PM »
Interesting data.  Just wanted to point out, FYI, that because no loss factor is included with the calculation of the second data set (and since they're so young, the riskier ones appear more inflated than others), those with increasing inquiries may or may not have had enough time to differentiate themselves and increase their default rates as they age.

Quite true, we certainly don't know how the newer loans will eventually turn out. What we do know is that LC now appears to increase interest rates at loan initiation based on increasing number of inquires whereas in the past it appears they did not. However, the updated LC model (if that's what this is) may now still be under compensating or over compensating for inquires at loan initiation. We'll know that answer in a couple of years.

Rob L

  • Hero Member
  • *****
  • Posts: 2119
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #35 on: January 19, 2014, 02:34:42 PM »
Borrowers with FICO 700 have been assigned grades A to G by LC.

Here's an extreme example. FICO at origination 770-774 yet LC assigned loan grade F2 (24.08%). Also the IR01 score is Poor.
The LC and IR models knew this one was a much riskier bet than the high FICO score indicated.

https://www.lendingclub.com/account/loanPerf.action?loan_id=7044744&order_id=10971049&note_id=29848844

And they were right. Two months into the loan, the borrower has stopped paying and the loan has been sent off to external collections.

neals384

  • Full Member
  • ***
  • Posts: 113
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #36 on: January 20, 2014, 09:55:30 AM »
LC has no incentive to price every loan at the "perfect" interest rate.  Almost all of their listings fill anyway.

Lenders has a financial incentive to select notes with the most attractive interest rate given the known risks.

People and organizations with financial incentives, talent and the willingness to work hard (Lend Academy folks) will almost always outperform those with no incentive (LC).


Emmanuel

  • Full Member
  • ***
  • Posts: 157
    • View Profile
    • LendingRobot
Re: What's This "Best Note" Selection Business Anyway?
« Reply #37 on: January 20, 2014, 01:46:28 PM »
LC has no incentive to price every loan at the "perfect" interest rate.  Almost all of their listings fill anyway.

Lenders has a financial incentive to select notes with the most attractive interest rate given the known risks.

People and organizations with financial incentives, talent and the willingness to work hard (Lend Academy folks) will almost always outperform those with no incentive (LC).

I respectfully disagree... It is in LC or Prosper's interest to price their loans as well as possible. First, for the sake of market efficiency and avoiding supply/demand unbalance. Second, because of their financial incentives. They need to be borrower-friendly because of the originating fees they get when issuing new loans (too expensive = less demand), and lender-friendly because a) that's where the money is coming from and b) they get 1% of service fees on the payments (too cheap = less supply, and less service fees).

The hardest thing when investing in the Stock Market is sorting out signal from noise, because the amount of information is almost infinite. Therefore the value of investment advisers / analyst / gurus come for picking which data to focus on. In Peer Lending, there are relatively few data available (about 100 data fields in LC loans' history), so finding solutions is relatively easy and attainable with several well-know optimization algorithms. Even a brute-force simulation would probably work.

Something LC could easily do, even more so since they have more data than us (for instance, they could use the identify of the borrower to establish his 'social' credentials, grade his job situation based on his LinkedIn profile, assess its worth or lifestyle based on his FB posts). And I still have to meet a LC employee who seems stupid or lazy. So, even smart and dedicated Lend Academy members are unlikely to 'beat' them. Unless 'perfect' means different things for LC and for us. Thinking about it, it should be possible to 'find' what they're optimizing for based on their historical data.
« Last Edit: January 20, 2014, 05:53:03 PM by Emmanuel »

quantalcontent

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #38 on: January 22, 2014, 11:05:23 PM »
In Peer Lending, there are relatively few data available (about 100 data fields in LC loans' history), so finding solutions is relatively easy and attainable with several well-know optimization algorithms. Even a brute-force simulation would probably work.

So here's an observation (and a question) from someone contemplating getting started with investing. Over time, LC changes their underwriting criteria and rate-setting algorithm, perhaps frequently. Not only do they not tell us what these criteria and algorithms are, they also don't tell us when they change them. So, if you use historical data to classify notes, you run the risk of making serious errors. Yet, this is exactly what many third-party note-picking services seem to do.

The example provided by Rob L shows just what I mean. I had hit on the same observation. After playing around with LC's data for 36 month loans completed by 2013, I thought I was a genius: all you have to do is get rid of borrowers who made more than 0 inquiries, and you'll easily reduce your defaults even on C, D and higher loans! But it seems that LC might have caught onto this and changed the influence of inquiries on interest rate. All those note-picking services use algorithms more complicated than my "avoid inquiries" algorithm, but the problem is exactly the same: LC could be pulling the rug out from under them and investors wouldn't know it for years.

I'm curious about what the denizens of this forum think about this concern. Am I missing something? How likely is it that "picking the best notes" today based on yesterday's data will prove to be counterproductive?

brycemason

  • Hero Member
  • *****
  • Posts: 801
    • View Profile
    • P2P-Picks.com
    • Email
Re: What's This "Best Note" Selection Business Anyway?
« Reply #39 on: January 23, 2014, 09:52:58 AM »
You are missing something. Incorporate the interest rate into your selection of notes. Then if LC changes it, so too will your selection of notes. If LC today made a stray "A" note with 40% interest, it would automatically be at the top of my profit max list.

The past is the absolute best thing we have to predict the future. I imagine LC set their initial risk model with personal loan data from some other market, and then has updated it as their own loans matured. How else are you imagining they set interest rates besides historically consistent relationships between borrower characteristics and the probability of default?

quantalcontent

  • Newbie
  • *
  • Posts: 2
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #40 on: January 23, 2014, 12:33:49 PM »
I'm not sure what you mean by "incorporate interest rate". I did that by making a simple logistic regression model to predict default rates, using all the data for completed 36 month loans through 2013. I included a few parameters that seemed to contribute substantially to the variance; one of the best predictors was "inquiries in the last 6 months". I also included "interest rate" and a few others. So then I had a regression equation that let me determine the probability that any given loan would default. Of course it works well for the historical data (because that's what the model is based on). If LC recently bumped up the interest rate for borrowers with a high number of inquiries, the model might still work roughly OK because it includes interest rate as one of the predictors. (Although using "category", i.e., A1-G5, as a predictor actually works better than interest rate, presumably because it's not subject to the variability in interest rates that LC introduces by responding to market pressure.)

HOWEVER, this does not take into account the possibility that LC actually changed their underwriting for loans with higher inquiries. Suppose they rejected more of them unless the borrower met other requirements, such as having a higher income, lower DTI, or some other as yet non-transparent factor. Then the influence of inquiries on default rate would be fundamentally different today than it was in the last few years (because the population sampled by LC is now different: they are accepting and rejecting a different group of borrowers with different characteristics, among which "inquiries" is no longer strongly related to default rate), and the model would overpredict default rates. I'd be hurting myself by using "inquiries" as a criterion for rejecting notes.

So, while I completely agree with you that historical data is the best way to predict the future, those predictions assume some consistency between the past and the present. Because LC periodically changes its underwriting and rate-setting alrgorithms, there is less consistency - perhaps a lot less. So predictions are less valid, meaning that the risk of using them is greater.

dontvote

  • Full Member
  • ***
  • Posts: 226
    • View Profile
    • Email
Re: What's This "Best Note" Selection Business Anyway?
« Reply #41 on: January 23, 2014, 01:37:08 PM »
It's a tautology - the only data is the best data. The past is the only information we have to predict the future but while statistical methods can help shed light on some possible fundamental relationships, they are not able to show them explicitly.

I don't think anyone would disagree that the limited fields we have avail are mere proxies for what is actually going on in people's lives.

dontlietomedamnrobot
dontvote

Interest Radar IRR: 46.11%
NSR ROI: 41.22%
Average Loan Age: Your Moms

Emmanuel

  • Full Member
  • ***
  • Posts: 157
    • View Profile
    • LendingRobot
Re: What's This "Best Note" Selection Business Anyway?
« Reply #42 on: January 23, 2014, 03:37:46 PM »
So here's an observation (and a question) from someone contemplating getting started with investing. Over time, LC changes their underwriting criteria and rate-setting algorithm, perhaps frequently. Not only do they not tell us what these criteria and algorithms are, they also don't tell us when they change them. So, if you use historical data to classify notes, you run the risk of making serious errors. Yet, this is exactly what many third-party note-picking services seem to do.

A way around that is to estimate the default probability based on all criteria BUT the grade/interest rate. That gives you an interest rate to be at equilibrium, and you deduct it from the grade/interest determined by LC. The higher the value, the better the bargain.

edward

  • Full Member
  • ***
  • Posts: 195
    • View Profile
Re: What's This "Best Note" Selection Business Anyway?
« Reply #43 on: January 23, 2014, 05:46:02 PM »
For those of you with a credit granting background, does LC see the FICO history when they run an inquiry or just the snapshot at that moment in time? In your experience, would it make any difference if they did--that is, how predictive is past FICO of future repayment likelihood?

brycemason

  • Hero Member
  • *****
  • Posts: 801
    • View Profile
    • P2P-Picks.com
    • Email
Re: What's This "Best Note" Selection Business Anyway?
« Reply #44 on: January 23, 2014, 07:46:07 PM »
So here's an observation (and a question) from someone contemplating getting started with investing. Over time, LC changes their underwriting criteria and rate-setting algorithm, perhaps frequently. Not only do they not tell us what these criteria and algorithms are, they also don't tell us when they change them. So, if you use historical data to classify notes, you run the risk of making serious errors. Yet, this is exactly what many third-party note-picking services seem to do.

A way around that is to estimate the default probability based on all criteria BUT the grade/interest rate. That gives you an interest rate to be at equilibrium, and you deduct it from the grade/interest determined by LC. The higher the value, the better the bargain.

Winner winner, chicken dinner.