Lend Academy Network Forum

Lending Club Discussion => Investors - LC => Topic started by: sociallender on November 28, 2014, 06:48:28 PM

Title: Funded Amount?
Post by: sociallender on November 28, 2014, 06:48:28 PM
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed?  I poll the API every second  (abiding by the 1 second rule) until new notes are listed.  However, many of the notes are significantly funded before doing any note selection/processing.  Is there some sort of "pre" funding that happens before being listed?  Or is it that the 1 second between API calls allows the notes to be funded by others before I have a chance? 

For example, today at 14:00:15 PST, the following notes (I added funding percent column) were downloaded after a 1 second interval polling of the API:

loanId   loanAmount   fundedAmount   Funded Percent
35743454   20825   10700   51.38%
35813760   4000   850   21.25%
35853954   9000   3950   43.89%
35949631   10800   4775   44.21%
35959506   12000   5275   43.96%
35959805   35000   25225   72.07%
35989872   5000   950   19.00%
36069520   21000   25   0.12%
36149280   8000   3500   43.75%
36200923   12000   2275   18.96%
36200974   9000   1700   18.89%
36210774   19725   13350   67.68%
36210926   11200   4200   37.50%
36210946   20000   8800   44.00%
36230840   10525   9875   93.82%
36230843   24125   10600   43.94%
36230868   13000   6150   47.31%
36230875   20000   3800   19.00%
36240881   6000   2625   43.75%
36250824   6000   4450   74.17%
36250832   7000   3075   43.93%
36250844   15000   6525   43.50%
36250846   6000   4400   73.33%
36250877   16000   7025   43.91%
36260287   12000   5275   43.96%
36260875   8000   3925   49.06%
36260887   3000   550   18.33%
36260888   22200   10450   47.07%
36270336   35000   15400   44.00%
36270903   8000   1500   18.75%
36270915   12000   5450   45.42%
36270925   15250   10225   67.05%
36270939   24000   4600   19.17%
36270951   3600   775   21.53%
36270953   15000   3325   22.17%
36280857   16750   7350   43.88%
36280863   35000   15400   44.00%
36280931   10000   5200   52.00%
36290825   12000   9900   82.50%
36290857   28000   5300   18.93%
36300679   10750   2975   27.67%
36300902   5600   1075   19.20%
36310256   14975   6300   42.07%
36320975   8000   3850   48.13%
36320984   25000   16725   66.90%
36320991   15000   6600   44.00%
36321013   8000   3650   45.63%
36321019   4300   925   21.51%
36330875   35000   15575   44.50%
36330879   25000   11025   44.10%
36340818   3000   650   21.67%
36340845   10000   5200   52.00%
36340846   13000   6075   46.73%
36350774   8200   3600   43.90%
36350798   35000   6650   19.00%
36350828   18000   7900   43.89%
36360868   35000   14750   42.14%
36370806   10000   8075   80.75%
36370854   10000   7500   75.00%
36380435   3500   1525   43.57%
36390884   17000   8975   52.79%
36400855   10000   4400   44.00%
36400882   35000   15400   44.00%
36410908   20600   9050   43.93%
36410950   28000   26500   94.64%
36410952   10000   4825   48.25%
36410956   15000   7475   49.83%
36410970   20000   3825   19.13%
36420844   5325   4000   75.12%
36420861   6000   1125   18.75%
36420882   3000   1300   43.33%
36420883   7800   1800   23.08%
36440697   15000   6625   44.17%
36440852   6775   4750   70.11%
36440868   20000   8850   44.25%
36450498   16800   4750   28.27%
36450502   7500   3300   44.00%
36490418   5000   950   19.00%
36490419   28000   21500   76.79%
36490430   21000   10350   49.29%
36490438   13000   3050   23.46%
36490441   3500   3325   95.00%
Title: Re: Funded Amount?
Post by: MarinBB on November 28, 2014, 08:07:22 PM
I suspect that new loans appear first on the website and several seconds later show up on the API. This would allow lenders using the website to see the new listings slightly before lenders using the API.
Title: Re: Funded Amount?
Post by: Lovinglifestyle on November 28, 2014, 08:48:38 PM
I'm amazed at how many have jumped straight to closing in the time it takes me to make 4 clicks to get to the View Order page.
Glad you asked the question about 1 sec. or pre-funding.  Seems to me some awesome automatic purchasing is going on, or maybe some whole loan remainders join the group at feeding time? 
Title: Re: Funded Amount?
Post by: GS on November 29, 2014, 02:34:19 PM
Could it be that LC's "Automatic Investing" gets to fund at the exact instance the loans go live?  Or even "prefund" the loans?
Title: Re: Funded Amount?
Post by: Fred93 on November 29, 2014, 03:54:08 PM
or maybe some whole loan remainders join the group at feeding time?

The whole-loan market is called the whole-loan market because people there are allowed to buy the whole loan, not pieces.  Therefore, I believe that when loans roll from the whole-loan market to the fractional-loan market, it is the whole loan that arrives at the fractional-loan market.
Title: Re: Funded Amount?
Post by: sociallender on November 29, 2014, 09:31:45 PM
Could it be that LC's "Automatic Investing" gets to fund at the exact instance the loans go live?  Or even "prefund" the loans?

I am having a hard time believing that web site users are funding these notes this quickly.  That would be some fast clicking or even the locking with the shopping cart.  If it's not the whole loan market (as Fred93 mentioned) and not manual web site users, the only thing I can think of is auto investing or super fast API transactions by other users.  I am more concerned about the former as the latter may be just a better system implementation.  If it's the former, it raises my eyebrow of equality.

I have been monitoring for some time now, and its damn near impossible to find any "prime" loans that are not already majority funded when listed on API.  Truth be told, I still find plenty of loans that match my criteria and keep my cash drag low.  However, I am frustrated by the fact that if I was only investing in the 'popular' loans, I would be hard pressed to find adequate surplus of notes. 

As an example, I built a personal system that is very fast (at least I think it is) in processing the API data.  The only dependent functions are the API list download and API order submission.  Here is an example of timing for the last run:

Note List Time: 2014-11-29 18:00:15 PST  --> When new listed notes downloaded via API
Processing Time: 0.38 seconds -->  Filtering/Modeling time
API Order Time: 0.68 seconds --> API order completion time
Total Time: 1.06 seconds  --> From download to order complete

This resulted in this order:
loanId    requestedAmount    investedAmount    executionStatus
36241131    25.000    0.000    NOT_AN_IN_FUNDING_LOAN
36351000        25.000    0.000    NOT_AN_IN_FUNDING_LOAN

Many times, all these types of loans are not in funding within less than a second.  I would be OK with this if the funded percent amount was closer to 0, but by the time the notes appear on the API, they are majority funded. 

Has anyone else had this problem or had better luck with obtaining the notes without being majority funded?  I do host my system on the east coast, so perhaps its a speed connection issue?  I have FIOS so it shouldn't be a bandwidth issue. 

For reference, here are the funding percentages when new note listing was detected:
id   loanAmount   fundedAmount   
36351000   5000   4375   87.50%
36371040   3000   2300   76.67%
36241131   3000   2175   72.50%
36450735   2650   1450   54.72%
36059716   4000   2650   66.25%
35938741   3000   1550   51.67%
36149679   3600   1950   54.17%
36251091   8000   6225   77.81%
36421089   30000   28150   93.83%
36271164   4000   2125   53.13%
36221210   4000   2025   50.63%
36271001   9000   6900   76.67%
36201069   3000   775   25.83%
36301219   5000   2700   54.00%
36371069   5000   2550   51.00%
36341091   13500   10850   80.37%
36211199   7500   4675   62.33%
36241134   19200   15850   82.55%
36231135   5675   2275   40.09%
36281131   8000   4500   56.25%
36321231   8000   4075   50.94%
36500217   8800   4825   54.83%
36251063   10000   5950   59.50%
36401135   9500   5350   56.32%
36139407   8500   4325   50.88%
36078912   8500   4325   50.88%
35783259   10000   5475   54.75%
36251042   7800   3175   40.71%
36490649   12000   7200   60.00%
36381129   10500   5375   51.19%
36341063   12000   6100   50.83%
36261156   15000   8050   53.67%
36169560   15000   7675   51.17%
36321235   15000   7675   51.17%
36301211   15000   7650   51.00%
34492754   10000   2625   26.25%
36271181   35000   27200   77.71%
36230860   18000   9175   50.97%
36450740   12000   3100   25.83%
36490668   20000   10225   51.13%
36321227   20000   10200   51.00%
36331117   35000   24900   71.14%
36490666   21000   10825   51.55%
36421088   22525   11500   51.05%
36221195   15000   3900   26.00%
36390902   18000   4675   25.97%
36411178   30000   15350   51.17%
36381159   32000   16300   50.94%
36211190   14250   10650   74.74%
36201063   11200   3425   30.58%
36350808   11200   2900   25.89%
36500236   12800   3875   30.27%
36370877   12500   3275   26.20%
36250932   14400   4250   29.51%
35653450   17475   5075   29.04%
36221070   18000   4675   25.97%
36301077   18000   4675   25.97%
36360959   18000   4675   25.97%
36200607   35000   19550   55.86%
36190130   29500   7650   25.93%
36261158   34025   11025   32.40%
36370953   35000   10225   29.21%
36450571   35000   9150   26.14%
36210559   35000   9100   26.00%



Title: Re: Funded Amount?
Post by: MarinBB on November 30, 2014, 02:04:12 AM
SocialLender, I'm seeing the exact same thing. New listings are getting filled extremely quickly after they hit the website - a good chunk of them are filled before they ever appear in the API. There seems to be a 4-6 second gap between when new listings show up on the website and when they first appear in the API. It's not much, but might be enough for a very large investor to add them to an order manually and take them down.


Title: Re: Funded Amount?
Post by: thezfunk on November 30, 2014, 02:13:15 AM
SocialLender, I'm seeing the exact same thing. New listings are getting filled extremely quickly after they hit the website - a good chunk of them are filled before they ever appear in the API. There seems to be a 4-6 second gap between when new listings show up on the website and when they first appear in the API. It's not much, but might be enough for a very large investor to add them to an order manually and take them down.

Come on.  I can't believe you guys are even entertaining the idea someone is sitting at their computer, at feeding time, and in a couple seconds investing in even a single loan, manually.  This is automation without question.  From the evidence there are only two possibilities.  Either people are screen scraping or LC is back door dealing with someone or someones with faster API access.

This isn't even a 'Core' level conspiracy here.  I can't imagine what he could come up with as a theory with this evidence.  In fact, I hope he has one, for entertainment sake.
Title: Re: Funded Amount?
Post by: Fred on November 30, 2014, 05:05:50 AM
As you might already know, LC investors can invest in loans in one of the following channels:
1. whole-loans
2. notes
3. certificates

Since we can eliminate the first 2, the only remaining suspect would be that the notes were purchased by LC Advisors (LCA) as trust certificates. The Trust certificates are settled with cash flows from underlying loans in a manner similar to the notes.
Title: Re: Funded Amount?
Post by: Half Right on November 30, 2014, 01:07:39 PM
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all
Title: Re: Funded Amount?
Post by: sociallender on November 30, 2014, 01:27:39 PM
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all

I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system?  If so, is LCA more successful obtaining these loans then you were using the API?  Was your system performance competitive (less than a second filter/order time)? 

If LCA is able to capture these types of notes, it leads me to believe they must have first right of refusal.  Or using technology that sits local to their systems and is therefore faster than everyone else. 

I am starting to feel like the API has lost its appeal....  Ugghhhh
Title: Re: Funded Amount?
Post by: thezfunk on November 30, 2014, 02:08:37 PM
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all

I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system?  If so, is LCA more successful obtaining these loans then you were using the API?  Was your system performance competitive (less than a second filter/order time)? 

If LCA is able to capture these types of notes, it leads me to believe they must have first right of refusal.  Or using technology that sits local to their systems and is therefore faster than everyone else. 

I am starting to feel like the API has lost its appeal....  Ugghhhh

I have wondered how long it would take for LC to give their system, priority.  It makes sense and if it is happening, I am not surprised.
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 02:57:07 PM
Since we can eliminate the first 2, the only remaining suspect would be that the notes were purchased by LC Advisors (LCA) as trust certificates. The Trust certificates are settled with cash flows from underlying loans in a manner similar to the notes.

It is my understanding that the LC Advisors funds purchase trust certificates from their SPV called "LC Trust", and that these certificates represent whole loans.  If this understanding is correct, then they would not be the folks causing the problem discussed here.  (ie this thread is discussing the mystery of how loans seem to have big chunks already gone when folks first see them)

It is my understanding that LCA has three funds now...
Conservative Consumer Credit (buys A & B loans)
Broad Based Consumer Credit (buys A...G loans)
High Yield Credit (this one is new, and I don't know anything about it)
Title: Re: Funded Amount?
Post by: Fred on November 30, 2014, 03:24:08 PM
Per LC 10-Q:

Quote
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.

It seems certificates and whole loans are mutually exclusive.

It is clear that the notes were only 8% of the total, which is very depressing.
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 03:27:17 PM
I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system?  If so, is LCA more successful obtaining these loans then you were using the API?

You don't specify filters.  You deposit money into one of three (until recently two) funds, and they take it from there. 

(Each of these funds exists in more than one legal form, but that's a legal/regulatory thing.)  Monthly reports show only returns for the month, and the number and mix of loans by grade, and a few other high level stats.  Investors don't see the individual loans or any other details.

You can read LC Advisors form ADV on the SEC web site.  The contains significant information.  For example, the ADV tells us about the 2 funds (doesn't yet talk about the high yield one), tells us the amounts involved...

Conservative   $586M from 792 investors.
Broad Based  $122M ftrom 137 investors.

If you want more, you can call up LC Advisors, ask them questions, and ask for a copy of their offering memorandum.  They don't bite
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 03:39:12 PM
Per LC 10-Q:

Quote
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.

It seems certificates and whole loans are mutually exclusive.

And its funny that 0.2+0.3+0.5 does not add up to 1.2  :o

I don't claim to understand the terminology that their accountants are using in the 10Q paragraph you quoted 100%. 

I don't think I'm supposed to quote the trust document (and I'm not energetic enough to read everything again to find the sentence I vaguely remember to figure out for sure), so I'd suggest you call up LCA and get your own copy of the documents, where you can read about what they do.


Quote
It is clear that the notes were only 8% of the total, which is very depressing.

Keep in mind that as LC branches out ... policy 2 loans, small business loans, god-knows-what loans ... it becomes more and more difficult to use published numbers to compute ratios such as this.   The fractional market (notes) only accesses one kind of loans.  LC publishes totals in their 10Q and doesn't break the numbers out by type of loan.

There's lots we don't know.  For example, are the whole loans sold thru the certificate channel shown in the data files we can download?   Maybe not, eh?

There's a chart on nickelsteamroller which shows the breakdown of LC loans between "whole", "fractional, and "other".  It claims that 23% of the loans are going to the fractional marketplace.  I don't understand the source of their data.  The history files don't appear (to me) to show anything other than "w" and "f", so the NSR guys must be milking this information out in some mysterious way.
Title: Re: Funded Amount?
Post by: Fred on November 30, 2014, 03:52:21 PM
Per LC 10-Q:

Quote
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.

It seems certificates and whole loans are mutually exclusive.

And its funny that 0.2+0.3+0.5 does not add up to 1.2  :o

Ha ha ... you're right.  Between my eyes failing me, and my wife calling me to leave for shopping, I was a bit lazy to check and simply cut-and-paste:

http://www.sec.gov/Archives/edgar/data/1409970/000119312514414075/d766811ds1a.htm

I'll see if there are better numbers in their filings to show this.
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 03:57:42 PM
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed?  I poll the API every second  (abiding by the 1 second rule) until new notes are listed.

***  Which API?  CSV?  Old SOAP API?  New REST API?
***  And are you asking to see all loans, or just the "new" loans?

You are polling 1/second, but how long does it take for you to finish the data transfer?  Back when I was testing the SOAP API I found that it could take a damn long time for the data transfer to complete, and I found that this was highly sensitive to one's internet connection speed.  (This was so even tho only a tiny fraction of available speed was used.)  I believe this was due to a poorly configured server.  (Deep subject, which I won't elaborate on here, 'cause I'd be typing all afternoon.)  I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.

Now they improved things after my comments, and I sorta wore myself out with all that effort, so I never got back to this to test after their changes, and I have never tried the new API, so things might be different now, I dunno.

I am wondering whether you are operating at a disadvantage because your data transfer takes too long.


Title: Re: Funded Amount?
Post by: avid investor on November 30, 2014, 04:43:46 PM
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed?  I poll the API every second  (abiding by the 1 second rule) until new notes are listed.
I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.

Fred, do you know how long a note is considered a "new loan"?  As we have discussed ad nauseum, loans ARE dropped at times other than the scheduled golden moments (which currently seem to give us only a hand full).  If a loan is considered a "new loan" for only a few minutes of time, then pulling only the "new loans" down at the designated times will mean that you won't include these loans.  Any thoughts on that?
Title: Re: Funded Amount?
Post by: sociallender on November 30, 2014, 04:50:27 PM
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed?  I poll the API every second  (abiding by the 1 second rule) until new notes are listed.

***  Which API?  CSV?  Old SOAP API?  New REST API?
***  And are you asking to see all loans, or just the "new" loans?

You are polling 1/second, but how long does it take for you to finish the data transfer?  Back when I was testing the SOAP API I found that it could take a damn long time for the data transfer to complete, and I found that this was highly sensitive to one's internet connection speed.  (This was so even tho only a tiny fraction of available speed was used.)  I believe this was due to a poorly configured server.  (Deep subject, which I won't elaborate on here, 'cause I'd be typing all afternoon.)  I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.

Now they improved things after my comments, and I sorta wore myself out with all that effort, so I never got back to this to test after their changes, and I have never tried the new API, so things might be different now, I dunno.

I am wondering whether you are operating at a disadvantage because your data transfer takes too long.


Using the new restful API on new loans only (not all loans as it would take considerably longer).  Typically takes between .7 to 1.5 seconds between sequential API calls.  On average around 1 second.  Coincidentally, when API calls are less than a second, I have not noticed any denial from LC servers.  Here is the elapsed time in seconds between successive calls taken recently which all responded successfully even though some were less than a second apart:

   user  system elapsed
   0.04    0.00    0.77
   user  system elapsed
   0.01    0.00    0.72
   user  system elapsed
   0.03    0.02    0.76
   user  system elapsed
   0.07    0.00    1.22
   user  system elapsed
   0.04    0.00    0.79
   user  system elapsed
   0.02    0.00    0.89
   user  system elapsed
   0.04    0.00    0.91
   user  system elapsed
   0.04    0.00    0.87
   user  system elapsed
   0.01    0.00    0.69
   user  system elapsed
   0.05    0.00    0.79
   user  system elapsed
   0.01    0.00    0.74

It is a little bit longer during feeding times but not by much.  Again, average of around a second.  So, at least every second I poll the RESTful API for new notes only.  If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point).  However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes. 

I thought it could be a speed issue as well, but odds are that by now I would have by chance polled the API at the optimal time, but yet no significant change in initial funding.  I still am investigating by my gut tells me that the notes are being funded prior to list.  Or perhaps captured by local/on-premise technology?.  MarinBB seems to be experiencing the same problem but I would like confirmation from others for a sanity check.
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 05:04:03 PM
Fred, do you know how long a note is considered a "new loan"?

When I talked with LC folks about this, their story was that "new" means "the current batch".  They tend to think from their end, not from our end, so in the past there have been many ill-defined concepts, so I don't know what this means except for what they said.  Their view is that loans are deposited by them in batches at the feeding times AND NEVER AT ANY OTHER TIME.  I haven't done more recent testing, so I'm not giving an opinion, just restating incoming info.

Quote
As we have discussed ad nauseum, loans ARE dropped at times other than the scheduled golden moments (which currently seem to give us only a hand full).

The LC guys view is that they drop at the specified times.  They didn't seem to be counting things like the loans that roll over from the whole-loan marketplace.  Those loans look "new" to me, as I've never seen them before, but LC guys think from their end, not from my end, so we use words differently.

Quote
If a loan is considered a "new loan" for only a few minutes of time, then pulling only the "new loans" down at the designated times will mean that you won't include these loans. 

I can only tell you what they told me.  I haven't experimentally tested the "new loan" stuff, nor have I tested the new API yet, so I'd be guessing.  When I asked questions like this they seemed to say things like ... well they're new until the next batch at the next feeding time.  What that means exactly to a person writing code at our end, I have no idea. 

I tried when I was talking with them to suggest a different API scheme in which there was no ambiguity about what "new" meant.  I was suggesting things like "gimme the 2PM batch" to which they could respond "not here yet dude" or else give me a batch of loans.  Looking from their end, they didn't seem to have the appreciation for the ambiguities related to time that I saw, and perhaps I didn't explain myself well enough.

I will be experimenting with the new API soon, so later I may have opinions about all these questions.  The absurd observations seen by sociallender are interesting as heck and are drawing me in.
Title: Re: Funded Amount?
Post by: GS on November 30, 2014, 05:06:40 PM
It kinda sucks that from I'm seeing, it looks like individual investors are only getting about 25% of the loans, and the ones we do get are half funded before we even see them ...  Come on LC ...
Title: Re: Funded Amount?
Post by: Fred93 on November 30, 2014, 05:13:11 PM
So, at least every second I poll the RESTful API for new notes only.  If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point).  However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes. 

Thank you.  That's very clear.  So your "less than 1 second" includes the download of the new loans in its entirety, so your data is very fresh, and yet loans are already heavily invested.

    Astounding.

Does this happen every feeding time, or just some?

[begin speculation]
Now I'm wondering about the architecture of LC's servers.  Likely they have more than one server, and perhaps different servers get the data at slightly different times.

Suggest you begin recording IP address of server you're talking with on each hit.  Once you start using one particular server you probably stick with it for a long time due to DNS caching.  You might experiment by flushing your DNS cache between hits, so you can learn more about how many servers they have, and whether some behave differently. 
[end speculation]
Title: Re: Funded Amount?
Post by: avid investor on November 30, 2014, 05:22:19 PM
Coincidentally, when API calls are less than a second, I have not noticed any denial from LC servers.  Here is the elapsed time in seconds between successive calls taken recently which all responded successfully even though some were less than a second apart:

I can address that one.  When I first went up on the new restful API, I wasn't putting any time between requests (yes, I ignored the rule).  The rule existed for the XML interface, and never triggered a failure, so I figured it was worth a try.  It must have just been slower, ergo no failures.

With the new API, however, I would get 6, 8, 10 loans and then the process would just hang and not return.  I put in timeouts in the interface, to avoid the hanging.  I never did get the http 500 error that the documentation said I would get, but then I found numerous issues with the documentation so I just shrugged it off.  After forcing the the 1-second delay, however, it never hung again.  So, it doesn't surprise me that you would be processing at a rate faster than one-per-second and not getting "caught" at it, as I was able to do the same, at least when doing "gets".  The "posts", as I said, would work for awhile before they through the yellow flag.  I suspect that they are managing it over time.  As in 4 POST transactions in 3 seconds might be fine, but 12 transactions in 10 seconds - maybe not.
Title: Re: Funded Amount?
Post by: Lovinglifestyle on November 30, 2014, 05:40:00 PM
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed.  Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.
Title: Re: Funded Amount?
Post by: avid investor on November 30, 2014, 05:44:05 PM
Ok, so I just ran a test.  Had to open up my filters to get higher-grade, lower-interest B's than I usually want, but I changed my linux usleep to 250000 (1/4 of a second) between POST transactions.  Funded 4 notes in less than 2 seconds.  There's definitely either a little more tolerance (as in maybe 2 transactions/sec?) or a measuring of throughput of a higher number of transactions over a period of time, as I previously surmised.
Title: Re: Funded Amount?
Post by: sociallender on November 30, 2014, 06:00:18 PM
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed.  Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.

Hmmm... Interesting...

Attached are the notes I picked up at 2014-11-30 14:00:11 PST.  Can you have a look to see if the notes you were able to fund are included and the corresponding percent funded (I added this column)?  There will still be unanswered questions but maybe it will give us a bit more insight.

Well just realized I can't attach a CSV file.  Here is a link:

https://dl.dropboxusercontent.com/u/415842/csv/new_notes.csv
Title: Re: Funded Amount?
Post by: sociallender on November 30, 2014, 06:47:49 PM
So, at least every second I poll the RESTful API for new notes only.  If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point).  However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes. 

Thank you.  That's very clear.  So your "less than 1 second" includes the download of the new loans in its entirety, so your data is very fresh, and yet loans are already heavily invested.

    Astounding.

Does this happen every feeding time, or just some?

[begin speculation]
Now I'm wondering about the architecture of LC's servers.  Likely they have more than one server, and perhaps different servers get the data at slightly different times.

Suggest you begin recording IP address of server you're talking with on each hit.  Once you start using one particular server you probably stick with it for a long time due to DNS caching.  You might experiment by flushing your DNS cache between hits, so you can learn more about how many servers they have, and whether some behave differently. 
[end speculation]

Yes, it happens every feeding time.  I will give your suggestion a try and let you know.  However, I would hope that their server farm would be configured similarly.
Title: Re: Funded Amount?
Post by: core on November 30, 2014, 07:03:33 PM
Look for "ECS" and an alphanumeric ID or anything else strange in the Server: header or X- headers.  I'd check myself but I refuse to sign that API agreement.  The IP won't tell you anything in a reverse proxy situation as it will always look the same from your end.  But if you want to look at IPs for whatever reason, it would be easier to dump all the relevant 'A' records first from their nameserver and hit them directly in an orderly manner by IP rather than monkeying with DNS cache and trial&error. 
Title: Re: Funded Amount?
Post by: Lovinglifestyle on November 30, 2014, 08:27:51 PM
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed.  Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.

Hmmm... Interesting...

Attached are the notes I picked up at 2014-11-30 14:00:11 PST.  Can you have a look to see if the notes you were able to fund are included and the corresponding percent funded (I added this column)?  There will still be unanswered questions but maybe it will give us a bit more insight.

Well just realized I can't attach a CSV file.  Here is a link:

https://dl.dropboxusercontent.com/u/415842/csv/new_notes.csv

I'm sorry, if they were there I just couldn't find them.  They weren't what I wanted after they were in my cart so I let them go.  However, I do have some loan numbers if that's of any interest:

36169416
19466180
36371234
36441184
36381241
36381241
36221299
36360733 (did not show up on Int. Radar nor PeerCube)
36490477

Two of those must have snuck in from the 10/11AM or 6/7AM feed, but I don't know which ones because I tossed all of them back.
That's why there are 8 numbers here but only 6 are from 2/3:00PM.  Also, I have no way of knowing what the % funded status was.  I could write that down another time.  Again--sorry for your trouble.  I'm pretty useless at this sort of thing!
Title: Re: Funded Amount?
Post by: core on December 01, 2014, 04:21:02 PM
Now I'm wondering about the architecture of LC's servers.  Likely they have more than one server, and perhaps different servers get the data at slightly different times.

Since my last post on this I have done some poking around regarding this, without being a registered API user.  I found no interesting headers from api.lendingclub.com while they were in the process of delivering my 401 unauthorized responses.

api.lendingclub.com resolved _every_ time for me to 216.115.73.155.  That belongs to this subnet:

Lending Club LENDINGCLUB-73 (NET-216-115-73-144-1) 216.115.73.144 - 216.115.73.159
SWITCH Communications Group LLC SWITCH-COMMUNICATIONS (NET-216-115-64-0-1) 216.115.64.0 - 216.115.95.255


*.144 to *.159 is quite a tight block.  And this is obviously not Edgecast and obviously not where their site is being served from.  It could be that complex in Nevada or wherever.  (Zach once mentioned the URL to their employee test site, might be interesting to see that falls in the same subnet.)  No matter.  The point is, you're not going to get anywhere here looking at IPs.  The best you could hope for is try all 16 (14? whatever) IPs in that range and see if anyone else answers to the virtual host api.lendingclub.com.  You could find a spare server which is more "lively", or you might even find the origin which is being proxied.  Likely not.

If they were using Edgecast for the API requests I might have some suggestions.  Because the Edgecast crap definitely _did_ have a timewarp, no doubt about it.  But here, they still could be handing off the requests to separate machines and you'd never know it because the HTTP headers are lean & mean and give no clues.

You guys are kinda behind, thinking about all this stuff just now.  I'm sure the institutional guys here on this forum have been at this for years.  It's too bad I never had reason to get into it; it might have been fun.
Title: Re: Funded Amount?
Post by: sociallender on December 05, 2014, 10:46:10 AM
It appears something has changed in a good way at LC.  The past 3 loan lists, I have been able to download the notes via the API and the percent funded has been 0%.  I am not sure if this is a reporting error or if I hit the API at the optimal time for each.  I will keep an eye on it but so far so good.  For example, here are the notes from the last list for one of my accounts.  Note the download time of the notes via API took 2.43 seconds.  This leads me to believe the I hit the API at the right time but took a while to process during heavy load.  Also, notice that the API list was at 2 seconds after the hour which is typically sooner than normal.  I am used to around 6 to 10 seconds after the hour.

Total New Notes: 116
Total Filtered Notes: 7
Ordered Notes: 5
Maximum Notes Per Order: 5
Requested Amount: $125.00
Investment Amount: $125.00
   
API Note Download Time: 2.43 seconds
Note List Time: 2014-12-04 14:00:02 PST
Processing Time: 0.13 seconds
API Order Time: 0.82 seconds
Total Time: 0.95 seconds


Title: Re: Funded Amount?
Post by: Fred93 on December 05, 2014, 01:06:49 PM
I can verify sociallender's recent good result.  (I can't verify his earlier difficulties, because by the time I got my test program running, he had declared success.)

So here are the first few and last few of this morning's 10AM batch, fresh off the press.  You can see they were all 0% funded as they first appeared.

 99 loans asOfDate 12/5/2014 10:00:02 AM
 id              loan amnt  funded     funded% posting time
 37077254  01200.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37137335  01500.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37137331  02000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36201362  02375.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37217208  02500.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 31206959  02900.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37317213  03500.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37137352  03500.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36330661  03625.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37217221  04000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37287253  04000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37107271  05000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37207219  05000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
...
 37147227  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 35874002  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36049533  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37147265  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36950381  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37167229  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36641316  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37117240  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36661316  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37147239  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 36341700  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37267150  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
 37187266  35000.00  00000.00  00.00%  12/5/2014 10:00:00 AM
end

Pardon the unusual leading zeros.  Was my trick for making the table line up nicely.
Title: Re: Funded Amount?
Post by: rawraw on December 05, 2014, 01:39:51 PM
Good news indeed
Title: Re: Funded Amount?
Post by: Kombinator on December 05, 2014, 01:46:39 PM
This is really excellent news and a very positive development. Makes me able to forgive them a little bit for taking away a lot of useful screening criteria (not forgive completely of course,just a little bit :)