Lend Academy Network Forum
Lending Club Discussion => Investors - LC => Topic started by: sociallender on November 28, 2014, 06:48:28 PM
-
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed? I poll the API every second (abiding by the 1 second rule) until new notes are listed. However, many of the notes are significantly funded before doing any note selection/processing. Is there some sort of "pre" funding that happens before being listed? Or is it that the 1 second between API calls allows the notes to be funded by others before I have a chance?
For example, today at 14:00:15 PST, the following notes (I added funding percent column) were downloaded after a 1 second interval polling of the API:
loanId loanAmount fundedAmount Funded Percent
35743454 20825 10700 51.38%
35813760 4000 850 21.25%
35853954 9000 3950 43.89%
35949631 10800 4775 44.21%
35959506 12000 5275 43.96%
35959805 35000 25225 72.07%
35989872 5000 950 19.00%
36069520 21000 25 0.12%
36149280 8000 3500 43.75%
36200923 12000 2275 18.96%
36200974 9000 1700 18.89%
36210774 19725 13350 67.68%
36210926 11200 4200 37.50%
36210946 20000 8800 44.00%
36230840 10525 9875 93.82%
36230843 24125 10600 43.94%
36230868 13000 6150 47.31%
36230875 20000 3800 19.00%
36240881 6000 2625 43.75%
36250824 6000 4450 74.17%
36250832 7000 3075 43.93%
36250844 15000 6525 43.50%
36250846 6000 4400 73.33%
36250877 16000 7025 43.91%
36260287 12000 5275 43.96%
36260875 8000 3925 49.06%
36260887 3000 550 18.33%
36260888 22200 10450 47.07%
36270336 35000 15400 44.00%
36270903 8000 1500 18.75%
36270915 12000 5450 45.42%
36270925 15250 10225 67.05%
36270939 24000 4600 19.17%
36270951 3600 775 21.53%
36270953 15000 3325 22.17%
36280857 16750 7350 43.88%
36280863 35000 15400 44.00%
36280931 10000 5200 52.00%
36290825 12000 9900 82.50%
36290857 28000 5300 18.93%
36300679 10750 2975 27.67%
36300902 5600 1075 19.20%
36310256 14975 6300 42.07%
36320975 8000 3850 48.13%
36320984 25000 16725 66.90%
36320991 15000 6600 44.00%
36321013 8000 3650 45.63%
36321019 4300 925 21.51%
36330875 35000 15575 44.50%
36330879 25000 11025 44.10%
36340818 3000 650 21.67%
36340845 10000 5200 52.00%
36340846 13000 6075 46.73%
36350774 8200 3600 43.90%
36350798 35000 6650 19.00%
36350828 18000 7900 43.89%
36360868 35000 14750 42.14%
36370806 10000 8075 80.75%
36370854 10000 7500 75.00%
36380435 3500 1525 43.57%
36390884 17000 8975 52.79%
36400855 10000 4400 44.00%
36400882 35000 15400 44.00%
36410908 20600 9050 43.93%
36410950 28000 26500 94.64%
36410952 10000 4825 48.25%
36410956 15000 7475 49.83%
36410970 20000 3825 19.13%
36420844 5325 4000 75.12%
36420861 6000 1125 18.75%
36420882 3000 1300 43.33%
36420883 7800 1800 23.08%
36440697 15000 6625 44.17%
36440852 6775 4750 70.11%
36440868 20000 8850 44.25%
36450498 16800 4750 28.27%
36450502 7500 3300 44.00%
36490418 5000 950 19.00%
36490419 28000 21500 76.79%
36490430 21000 10350 49.29%
36490438 13000 3050 23.46%
36490441 3500 3325 95.00%
-
I suspect that new loans appear first on the website and several seconds later show up on the API. This would allow lenders using the website to see the new listings slightly before lenders using the API.
-
I'm amazed at how many have jumped straight to closing in the time it takes me to make 4 clicks to get to the View Order page.
Glad you asked the question about 1 sec. or pre-funding. Seems to me some awesome automatic purchasing is going on, or maybe some whole loan remainders join the group at feeding time?
-
Could it be that LC's "Automatic Investing" gets to fund at the exact instance the loans go live? Or even "prefund" the loans?
-
or maybe some whole loan remainders join the group at feeding time?
The whole-loan market is called the whole-loan market because people there are allowed to buy the whole loan, not pieces. Therefore, I believe that when loans roll from the whole-loan market to the fractional-loan market, it is the whole loan that arrives at the fractional-loan market.
-
Could it be that LC's "Automatic Investing" gets to fund at the exact instance the loans go live? Or even "prefund" the loans?
I am having a hard time believing that web site users are funding these notes this quickly. That would be some fast clicking or even the locking with the shopping cart. If it's not the whole loan market (as Fred93 mentioned) and not manual web site users, the only thing I can think of is auto investing or super fast API transactions by other users. I am more concerned about the former as the latter may be just a better system implementation. If it's the former, it raises my eyebrow of equality.
I have been monitoring for some time now, and its damn near impossible to find any "prime" loans that are not already majority funded when listed on API. Truth be told, I still find plenty of loans that match my criteria and keep my cash drag low. However, I am frustrated by the fact that if I was only investing in the 'popular' loans, I would be hard pressed to find adequate surplus of notes.
As an example, I built a personal system that is very fast (at least I think it is) in processing the API data. The only dependent functions are the API list download and API order submission. Here is an example of timing for the last run:
Note List Time: 2014-11-29 18:00:15 PST --> When new listed notes downloaded via API
Processing Time: 0.38 seconds --> Filtering/Modeling time
API Order Time: 0.68 seconds --> API order completion time
Total Time: 1.06 seconds --> From download to order complete
This resulted in this order:
loanId requestedAmount investedAmount executionStatus
36241131 25.000 0.000 NOT_AN_IN_FUNDING_LOAN
36351000 25.000 0.000 NOT_AN_IN_FUNDING_LOAN
Many times, all these types of loans are not in funding within less than a second. I would be OK with this if the funded percent amount was closer to 0, but by the time the notes appear on the API, they are majority funded.
Has anyone else had this problem or had better luck with obtaining the notes without being majority funded? I do host my system on the east coast, so perhaps its a speed connection issue? I have FIOS so it shouldn't be a bandwidth issue.
For reference, here are the funding percentages when new note listing was detected:
id loanAmount fundedAmount
36351000 5000 4375 87.50%
36371040 3000 2300 76.67%
36241131 3000 2175 72.50%
36450735 2650 1450 54.72%
36059716 4000 2650 66.25%
35938741 3000 1550 51.67%
36149679 3600 1950 54.17%
36251091 8000 6225 77.81%
36421089 30000 28150 93.83%
36271164 4000 2125 53.13%
36221210 4000 2025 50.63%
36271001 9000 6900 76.67%
36201069 3000 775 25.83%
36301219 5000 2700 54.00%
36371069 5000 2550 51.00%
36341091 13500 10850 80.37%
36211199 7500 4675 62.33%
36241134 19200 15850 82.55%
36231135 5675 2275 40.09%
36281131 8000 4500 56.25%
36321231 8000 4075 50.94%
36500217 8800 4825 54.83%
36251063 10000 5950 59.50%
36401135 9500 5350 56.32%
36139407 8500 4325 50.88%
36078912 8500 4325 50.88%
35783259 10000 5475 54.75%
36251042 7800 3175 40.71%
36490649 12000 7200 60.00%
36381129 10500 5375 51.19%
36341063 12000 6100 50.83%
36261156 15000 8050 53.67%
36169560 15000 7675 51.17%
36321235 15000 7675 51.17%
36301211 15000 7650 51.00%
34492754 10000 2625 26.25%
36271181 35000 27200 77.71%
36230860 18000 9175 50.97%
36450740 12000 3100 25.83%
36490668 20000 10225 51.13%
36321227 20000 10200 51.00%
36331117 35000 24900 71.14%
36490666 21000 10825 51.55%
36421088 22525 11500 51.05%
36221195 15000 3900 26.00%
36390902 18000 4675 25.97%
36411178 30000 15350 51.17%
36381159 32000 16300 50.94%
36211190 14250 10650 74.74%
36201063 11200 3425 30.58%
36350808 11200 2900 25.89%
36500236 12800 3875 30.27%
36370877 12500 3275 26.20%
36250932 14400 4250 29.51%
35653450 17475 5075 29.04%
36221070 18000 4675 25.97%
36301077 18000 4675 25.97%
36360959 18000 4675 25.97%
36200607 35000 19550 55.86%
36190130 29500 7650 25.93%
36261158 34025 11025 32.40%
36370953 35000 10225 29.21%
36450571 35000 9150 26.14%
36210559 35000 9100 26.00%
-
SocialLender, I'm seeing the exact same thing. New listings are getting filled extremely quickly after they hit the website - a good chunk of them are filled before they ever appear in the API. There seems to be a 4-6 second gap between when new listings show up on the website and when they first appear in the API. It's not much, but might be enough for a very large investor to add them to an order manually and take them down.
-
SocialLender, I'm seeing the exact same thing. New listings are getting filled extremely quickly after they hit the website - a good chunk of them are filled before they ever appear in the API. There seems to be a 4-6 second gap between when new listings show up on the website and when they first appear in the API. It's not much, but might be enough for a very large investor to add them to an order manually and take them down.
Come on. I can't believe you guys are even entertaining the idea someone is sitting at their computer, at feeding time, and in a couple seconds investing in even a single loan, manually. This is automation without question. From the evidence there are only two possibilities. Either people are screen scraping or LC is back door dealing with someone or someones with faster API access.
This isn't even a 'Core' level conspiracy here. I can't imagine what he could come up with as a theory with this evidence. In fact, I hope he has one, for entertainment sake.
-
As you might already know, LC investors can invest in loans in one of the following channels:
1. whole-loans
2. notes
3. certificates
Since we can eliminate the first 2, the only remaining suspect would be that the notes were purchased by LC Advisors (LCA) as trust certificates. The Trust certificates are settled with cash flows from underlying loans in a manner similar to the notes.
-
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all
-
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all
I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system? If so, is LCA more successful obtaining these loans then you were using the API? Was your system performance competitive (less than a second filter/order time)?
If LCA is able to capture these types of notes, it leads me to believe they must have first right of refusal. Or using technology that sits local to their systems and is therefore faster than everyone else.
I am starting to feel like the API has lost its appeal.... Ugghhhh
-
as an investor through both LC Advisors and as an individual investing directly I have recently decided that it is much easier to continue through LC Advisors. I had attempted to beat the returns provided through LC Advisors by using my own filtering systems but lately it has become impossible to overcome the cash drag due to the inability to find enough notes meeting my filter criteria to keep me fully invested.
I have begun to request withdrawals from my individual account which i am sending into LC Advisors.
It appears that High Frequency Investing has overtaken P2P lending just as High Frequency Trading has overtaken trading of every other capital asset. it was fun while it lasted but the time has come to start looking for the next niche. Good luck to all
I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system? If so, is LCA more successful obtaining these loans then you were using the API? Was your system performance competitive (less than a second filter/order time)?
If LCA is able to capture these types of notes, it leads me to believe they must have first right of refusal. Or using technology that sits local to their systems and is therefore faster than everyone else.
I am starting to feel like the API has lost its appeal.... Ugghhhh
I have wondered how long it would take for LC to give their system, priority. It makes sense and if it is happening, I am not surprised.
-
Since we can eliminate the first 2, the only remaining suspect would be that the notes were purchased by LC Advisors (LCA) as trust certificates. The Trust certificates are settled with cash flows from underlying loans in a manner similar to the notes.
It is my understanding that the LC Advisors funds purchase trust certificates from their SPV called "LC Trust", and that these certificates represent whole loans. If this understanding is correct, then they would not be the folks causing the problem discussed here. (ie this thread is discussing the mystery of how loans seem to have big chunks already gone when folks first see them)
It is my understanding that LCA has three funds now...
Conservative Consumer Credit (buys A & B loans)
Broad Based Consumer Credit (buys A...G loans)
High Yield Credit (this one is new, and I don't know anything about it)
-
Per LC 10-Q:
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.
It seems certificates and whole loans are mutually exclusive.
It is clear that the notes were only 8% of the total, which is very depressing.
-
I am not sure how LCA works, but are you using the same filter criteria through LCA that you were using in your own system? If so, is LCA more successful obtaining these loans then you were using the API?
You don't specify filters. You deposit money into one of three (until recently two) funds, and they take it from there.
(Each of these funds exists in more than one legal form, but that's a legal/regulatory thing.) Monthly reports show only returns for the month, and the number and mix of loans by grade, and a few other high level stats. Investors don't see the individual loans or any other details.
You can read LC Advisors form ADV on the SEC web site. The contains significant information. For example, the ADV tells us about the 2 funds (doesn't yet talk about the high yield one), tells us the amounts involved...
Conservative $586M from 792 investors.
Broad Based $122M ftrom 137 investors.
If you want more, you can call up LC Advisors, ask them questions, and ask for a copy of their offering memorandum. They don't bite
-
Per LC 10-Q:
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.
It seems certificates and whole loans are mutually exclusive.
And its funny that 0.2+0.3+0.5 does not add up to 1.2 :o
I don't claim to understand the terminology that their accountants are using in the 10Q paragraph you quoted 100%.
I don't think I'm supposed to quote the trust document (and I'm not energetic enough to read everything again to find the sentence I vaguely remember to figure out for sure), so I'd suggest you call up LCA and get your own copy of the documents, where you can read about what they do.
It is clear that the notes were only 8% of the total, which is very depressing.
Keep in mind that as LC branches out ... policy 2 loans, small business loans, god-knows-what loans ... it becomes more and more difficult to use published numbers to compute ratios such as this. The fractional market (notes) only accesses one kind of loans. LC publishes totals in their 10Q and doesn't break the numbers out by type of loan.
There's lots we don't know. For example, are the whole loans sold thru the certificate channel shown in the data files we can download? Maybe not, eh?
There's a chart on nickelsteamroller which shows the breakdown of LC loans between "whole", "fractional, and "other". It claims that 23% of the loans are going to the fractional marketplace. I don't understand the source of their data. The history files don't appear (to me) to show anything other than "w" and "f", so the NSR guys must be milking this information out in some mysterious way.
-
Per LC 10-Q:
In the third quarter of 2014, our marketplace facilitated nearly $1.2 billion in loan originations, of which approximately:
1. $0.2 billion were invested in through notes issued pursuant to the Note Registration Statement,
2. $0.3 billion were invested in through certificates issued by the Trust and
3. $0.5 billion were invested in through whole loan sales.
It seems certificates and whole loans are mutually exclusive.
And its funny that 0.2+0.3+0.5 does not add up to 1.2 :o
Ha ha ... you're right. Between my eyes failing me, and my wife calling me to leave for shopping, I was a bit lazy to check and simply cut-and-paste:
http://www.sec.gov/Archives/edgar/data/1409970/000119312514414075/d766811ds1a.htm
I'll see if there are better numbers in their filings to show this.
-
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed? I poll the API every second (abiding by the 1 second rule) until new notes are listed.
*** Which API? CSV? Old SOAP API? New REST API?
*** And are you asking to see all loans, or just the "new" loans?
You are polling 1/second, but how long does it take for you to finish the data transfer? Back when I was testing the SOAP API I found that it could take a damn long time for the data transfer to complete, and I found that this was highly sensitive to one's internet connection speed. (This was so even tho only a tiny fraction of available speed was used.) I believe this was due to a poorly configured server. (Deep subject, which I won't elaborate on here, 'cause I'd be typing all afternoon.) I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.
Now they improved things after my comments, and I sorta wore myself out with all that effort, so I never got back to this to test after their changes, and I have never tried the new API, so things might be different now, I dunno.
I am wondering whether you are operating at a disadvantage because your data transfer takes too long.
-
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed? I poll the API every second (abiding by the 1 second rule) until new notes are listed.
I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.
Fred, do you know how long a note is considered a "new loan"? As we have discussed ad nauseum, loans ARE dropped at times other than the scheduled golden moments (which currently seem to give us only a hand full). If a loan is considered a "new loan" for only a few minutes of time, then pulling only the "new loans" down at the designated times will mean that you won't include these loans. Any thoughts on that?
-
I am trying to understand how newly listed notes could have the majority of the loan fully funded immediately after being listed? I poll the API every second (abiding by the 1 second rule) until new notes are listed.
*** Which API? CSV? Old SOAP API? New REST API?
*** And are you asking to see all loans, or just the "new" loans?
You are polling 1/second, but how long does it take for you to finish the data transfer? Back when I was testing the SOAP API I found that it could take a damn long time for the data transfer to complete, and I found that this was highly sensitive to one's internet connection speed. (This was so even tho only a tiny fraction of available speed was used.) I believe this was due to a poorly configured server. (Deep subject, which I won't elaborate on here, 'cause I'd be typing all afternoon.) I learned that to go fast you need a high speed internet connection and to use the "new loans only" version of the API.
Now they improved things after my comments, and I sorta wore myself out with all that effort, so I never got back to this to test after their changes, and I have never tried the new API, so things might be different now, I dunno.
I am wondering whether you are operating at a disadvantage because your data transfer takes too long.
Using the new restful API on new loans only (not all loans as it would take considerably longer). Typically takes between .7 to 1.5 seconds between sequential API calls. On average around 1 second. Coincidentally, when API calls are less than a second, I have not noticed any denial from LC servers. Here is the elapsed time in seconds between successive calls taken recently which all responded successfully even though some were less than a second apart:
user system elapsed
0.04 0.00 0.77
user system elapsed
0.01 0.00 0.72
user system elapsed
0.03 0.02 0.76
user system elapsed
0.07 0.00 1.22
user system elapsed
0.04 0.00 0.79
user system elapsed
0.02 0.00 0.89
user system elapsed
0.04 0.00 0.91
user system elapsed
0.04 0.00 0.87
user system elapsed
0.01 0.00 0.69
user system elapsed
0.05 0.00 0.79
user system elapsed
0.01 0.00 0.74
It is a little bit longer during feeding times but not by much. Again, average of around a second. So, at least every second I poll the RESTful API for new notes only. If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point). However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes.
I thought it could be a speed issue as well, but odds are that by now I would have by chance polled the API at the optimal time, but yet no significant change in initial funding. I still am investigating by my gut tells me that the notes are being funded prior to list. Or perhaps captured by local/on-premise technology?. MarinBB seems to be experiencing the same problem but I would like confirmation from others for a sanity check.
-
Fred, do you know how long a note is considered a "new loan"?
When I talked with LC folks about this, their story was that "new" means "the current batch". They tend to think from their end, not from our end, so in the past there have been many ill-defined concepts, so I don't know what this means except for what they said. Their view is that loans are deposited by them in batches at the feeding times AND NEVER AT ANY OTHER TIME. I haven't done more recent testing, so I'm not giving an opinion, just restating incoming info.
As we have discussed ad nauseum, loans ARE dropped at times other than the scheduled golden moments (which currently seem to give us only a hand full).
The LC guys view is that they drop at the specified times. They didn't seem to be counting things like the loans that roll over from the whole-loan marketplace. Those loans look "new" to me, as I've never seen them before, but LC guys think from their end, not from my end, so we use words differently.
If a loan is considered a "new loan" for only a few minutes of time, then pulling only the "new loans" down at the designated times will mean that you won't include these loans.
I can only tell you what they told me. I haven't experimentally tested the "new loan" stuff, nor have I tested the new API yet, so I'd be guessing. When I asked questions like this they seemed to say things like ... well they're new until the next batch at the next feeding time. What that means exactly to a person writing code at our end, I have no idea.
I tried when I was talking with them to suggest a different API scheme in which there was no ambiguity about what "new" meant. I was suggesting things like "gimme the 2PM batch" to which they could respond "not here yet dude" or else give me a batch of loans. Looking from their end, they didn't seem to have the appreciation for the ambiguities related to time that I saw, and perhaps I didn't explain myself well enough.
I will be experimenting with the new API soon, so later I may have opinions about all these questions. The absurd observations seen by sociallender are interesting as heck and are drawing me in.
-
It kinda sucks that from I'm seeing, it looks like individual investors are only getting about 25% of the loans, and the ones we do get are half funded before we even see them ... Come on LC ...
-
So, at least every second I poll the RESTful API for new notes only. If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point). However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes.
Thank you. That's very clear. So your "less than 1 second" includes the download of the new loans in its entirety, so your data is very fresh, and yet loans are already heavily invested.
Astounding.
Does this happen every feeding time, or just some?
[begin speculation]
Now I'm wondering about the architecture of LC's servers. Likely they have more than one server, and perhaps different servers get the data at slightly different times.
Suggest you begin recording IP address of server you're talking with on each hit. Once you start using one particular server you probably stick with it for a long time due to DNS caching. You might experiment by flushing your DNS cache between hits, so you can learn more about how many servers they have, and whether some behave differently.
[end speculation]
-
Coincidentally, when API calls are less than a second, I have not noticed any denial from LC servers. Here is the elapsed time in seconds between successive calls taken recently which all responded successfully even though some were less than a second apart:
I can address that one. When I first went up on the new restful API, I wasn't putting any time between requests (yes, I ignored the rule). The rule existed for the XML interface, and never triggered a failure, so I figured it was worth a try. It must have just been slower, ergo no failures.
With the new API, however, I would get 6, 8, 10 loans and then the process would just hang and not return. I put in timeouts in the interface, to avoid the hanging. I never did get the http 500 error that the documentation said I would get, but then I found numerous issues with the documentation so I just shrugged it off. After forcing the the 1-second delay, however, it never hung again. So, it doesn't surprise me that you would be processing at a rate faster than one-per-second and not getting "caught" at it, as I was able to do the same, at least when doing "gets". The "posts", as I said, would work for awhile before they through the yellow flag. I suspect that they are managing it over time. As in 4 POST transactions in 3 seconds might be fine, but 12 transactions in 10 seconds - maybe not.
-
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed. Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.
-
Ok, so I just ran a test. Had to open up my filters to get higher-grade, lower-interest B's than I usually want, but I changed my linux usleep to 250000 (1/4 of a second) between POST transactions. Funded 4 notes in less than 2 seconds. There's definitely either a little more tolerance (as in maybe 2 transactions/sec?) or a measuring of throughput of a higher number of transactions over a period of time, as I previously surmised.
-
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed. Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.
Hmmm... Interesting...
Attached are the notes I picked up at 2014-11-30 14:00:11 PST. Can you have a look to see if the notes you were able to fund are included and the corresponding percent funded (I added this column)? There will still be unanswered questions but maybe it will give us a bit more insight.
Well just realized I can't attach a CSV file. Here is a link:
https://dl.dropboxusercontent.com/u/415842/csv/new_notes.csv
-
So, at least every second I poll the RESTful API for new notes only. If they do not contain any previous notes, then listing has been completed (although listing can take longer for it to complete for all notes and some could be added at a later point). However, the notes that I do capture (even if listing has not completed) are majority funded for "popular" notes.
Thank you. That's very clear. So your "less than 1 second" includes the download of the new loans in its entirety, so your data is very fresh, and yet loans are already heavily invested.
Astounding.
Does this happen every feeding time, or just some?
[begin speculation]
Now I'm wondering about the architecture of LC's servers. Likely they have more than one server, and perhaps different servers get the data at slightly different times.
Suggest you begin recording IP address of server you're talking with on each hit. Once you start using one particular server you probably stick with it for a long time due to DNS caching. You might experiment by flushing your DNS cache between hits, so you can learn more about how many servers they have, and whether some behave differently.
[end speculation]
Yes, it happens every feeding time. I will give your suggestion a try and let you know. However, I would hope that their server farm would be configured similarly.
-
Look for "ECS" and an alphanumeric ID or anything else strange in the Server: header or X- headers. I'd check myself but I refuse to sign that API agreement. The IP won't tell you anything in a reverse proxy situation as it will always look the same from your end. But if you want to look at IPs for whatever reason, it would be easier to dump all the relevant 'A' records first from their nameserver and hit them directly in an orderly manner by IP rather than monkeying with DNS cache and trial&error.
-
This conversation is way beyond me, but I just want to say I picked up 6 new notes at 0% funding at the 2/3:00 feed. Viewing the cart 3 sec. later had 2 closing and the others 40-80% along.
Hmmm... Interesting...
Attached are the notes I picked up at 2014-11-30 14:00:11 PST. Can you have a look to see if the notes you were able to fund are included and the corresponding percent funded (I added this column)? There will still be unanswered questions but maybe it will give us a bit more insight.
Well just realized I can't attach a CSV file. Here is a link:
https://dl.dropboxusercontent.com/u/415842/csv/new_notes.csv
I'm sorry, if they were there I just couldn't find them. They weren't what I wanted after they were in my cart so I let them go. However, I do have some loan numbers if that's of any interest:
36169416
19466180
36371234
36441184
36381241
36381241
36221299
36360733 (did not show up on Int. Radar nor PeerCube)
36490477
Two of those must have snuck in from the 10/11AM or 6/7AM feed, but I don't know which ones because I tossed all of them back.
That's why there are 8 numbers here but only 6 are from 2/3:00PM. Also, I have no way of knowing what the % funded status was. I could write that down another time. Again--sorry for your trouble. I'm pretty useless at this sort of thing!
-
Now I'm wondering about the architecture of LC's servers. Likely they have more than one server, and perhaps different servers get the data at slightly different times.
Since my last post on this I have done some poking around regarding this, without being a registered API user. I found no interesting headers from api.lendingclub.com while they were in the process of delivering my 401 unauthorized responses.
api.lendingclub.com resolved _every_ time for me to 216.115.73.155. That belongs to this subnet:
Lending Club LENDINGCLUB-73 (NET-216-115-73-144-1) 216.115.73.144 - 216.115.73.159
SWITCH Communications Group LLC SWITCH-COMMUNICATIONS (NET-216-115-64-0-1) 216.115.64.0 - 216.115.95.255
*.144 to *.159 is quite a tight block. And this is obviously not Edgecast and obviously not where their site is being served from. It could be that complex in Nevada or wherever. (Zach once mentioned the URL to their employee test site, might be interesting to see that falls in the same subnet.) No matter. The point is, you're not going to get anywhere here looking at IPs. The best you could hope for is try all 16 (14? whatever) IPs in that range and see if anyone else answers to the virtual host api.lendingclub.com. You could find a spare server which is more "lively", or you might even find the origin which is being proxied. Likely not.
If they were using Edgecast for the API requests I might have some suggestions. Because the Edgecast crap definitely _did_ have a timewarp, no doubt about it. But here, they still could be handing off the requests to separate machines and you'd never know it because the HTTP headers are lean & mean and give no clues.
You guys are kinda behind, thinking about all this stuff just now. I'm sure the institutional guys here on this forum have been at this for years. It's too bad I never had reason to get into it; it might have been fun.
-
It appears something has changed in a good way at LC. The past 3 loan lists, I have been able to download the notes via the API and the percent funded has been 0%. I am not sure if this is a reporting error or if I hit the API at the optimal time for each. I will keep an eye on it but so far so good. For example, here are the notes from the last list for one of my accounts. Note the download time of the notes via API took 2.43 seconds. This leads me to believe the I hit the API at the right time but took a while to process during heavy load. Also, notice that the API list was at 2 seconds after the hour which is typically sooner than normal. I am used to around 6 to 10 seconds after the hour.
Total New Notes: 116
Total Filtered Notes: 7
Ordered Notes: 5
Maximum Notes Per Order: 5
Requested Amount: $125.00
Investment Amount: $125.00
API Note Download Time: 2.43 seconds
Note List Time: 2014-12-04 14:00:02 PST
Processing Time: 0.13 seconds
API Order Time: 0.82 seconds
Total Time: 0.95 seconds
-
I can verify sociallender's recent good result. (I can't verify his earlier difficulties, because by the time I got my test program running, he had declared success.)
So here are the first few and last few of this morning's 10AM batch, fresh off the press. You can see they were all 0% funded as they first appeared.
99 loans asOfDate 12/5/2014 10:00:02 AM
id loan amnt funded funded% posting time
37077254 01200.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37137335 01500.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37137331 02000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36201362 02375.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37217208 02500.00 00000.00 00.00% 12/5/2014 10:00:00 AM
31206959 02900.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37317213 03500.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37137352 03500.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36330661 03625.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37217221 04000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37287253 04000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37107271 05000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37207219 05000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
...
37147227 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
35874002 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36049533 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37147265 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36950381 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37167229 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36641316 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37117240 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36661316 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37147239 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
36341700 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37267150 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
37187266 35000.00 00000.00 00.00% 12/5/2014 10:00:00 AM
end
Pardon the unusual leading zeros. Was my trick for making the table line up nicely.
-
Good news indeed
-
This is really excellent news and a very positive development. Makes me able to forgive them a little bit for taking away a lot of useful screening criteria (not forgive completely of course,just a little bit :)