Author Topic: Understanding screen scraping  (Read 2263 times)

hoggy1

  • Sr. Member
  • ****
  • Posts: 401
    • View Profile
    • Email
Understanding screen scraping
« on: August 02, 2014, 06:19:58 PM »
When I open a loan performance file like
https://www.lendingclub.com/account/loanPerf.action?loan_id=13077966&order_id=19503016&note_id=43589840

all the fico history and payment data are in the HTML in plain txt.

If I parse this data, is that screen-scraping? There is no way to get this with the API that I can see?
Steve

Rob L

  • Hero Member
  • *****
  • Posts: 2102
    • View Profile
Re: Understanding screen scraping
« Reply #1 on: August 02, 2014, 06:50:30 PM »
Yes, if you are parsing HTML I think that's the basic definition of "screen scraping".
The API is only about loans before they are issued.
When you do a loanBrowseloans API transaction you pretty much get the same information as downloading the browsenotes csv  file.
There are a few minor differences in information content.
Once a loan is fully funded, much less issued, it's all over for the API and browsenotes.csv.

hoggy1

  • Sr. Member
  • ****
  • Posts: 401
    • View Profile
    • Email
Re: Understanding screen scraping
« Reply #2 on: August 04, 2014, 09:04:33 AM »
Well then, I have asked this question before in the forums and everyone goes quiet and says nothing. Most or all of the third party tools have access to either payment history and or FICO history and or Status changes. They have to be accessing these by screen scraping using the protfolio's orderid and noteid? The data is not even available for them to archive for themselves anymore since LC only updates notes file at the end of a quarter. If there is another way I would like to use it? Please tell me how else to get this data?
Steve

turing

  • Full Member
  • ***
  • Posts: 222
    • View Profile
    • PeerToPeerQuant.com
    • Email
Re: Understanding screen scraping
« Reply #3 on: August 05, 2014, 01:39:15 PM »
What you describe is screen scraping.

As far as I know there is no way to get the payment history on all notes via screen scraping because you need to be an investor in that note to get the payment history.  You can screen scrape your own notes, but not all notes.

You can visit the note url page at any time and get updated status - Current, Late, Default, etc.
https://www.lendingclub.com/account/loanDetail.action?loan_id=17402889

If you checked that every month you would have a record of the payment history for every note from the point you started doing this.  Other services that have been around longer might be doing this already so they can infer payment history from public info.