Author Topic: Need Help Writing Automation Script  (Read 2002 times)

jz451

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Need Help Writing Automation Script
« on: July 14, 2019, 02:59:59 PM »
Hi All,

Been a while since last posting as I haven't been as active investing, but have lurked from time to time.

For those of you have have written scripts for automating loan selection using the API I need some of your assistance. My script will select loans using a model I have backtested to classify which new loans will "Pay in Full". So far I am able to get the cash balance and have written the code for transforming the data, but need help with how to retrieve the data and for placing an order. Here's some questions I have:

1. What platform did you use to run the script? I'm planning to use AWS but wanted to see what others have used.

2. I'm new to web scraping using Python, so running Get/Post calls using Requests isn't intuitive to me when there's no examples in the LC API documentation. For those willing may I see your script sans selection filters/models to get a better idea of the structure for writing a script that functions properly?

3. For retrieving the data, is there a way to get a subset of the variables or am I only able to retrieve all variables? Also, I noticed when trying to convert the JSON to a dataframe, it's formatted as an overall dictionary with a list of loans and each loan is a dictionary of the data as you can see in the attached image. What's the best way to properly turn it into a dataframe so each loan is a separate row with the variables as columns?

Thanks for the help.



rawraw

  • Hero Member
  • *****
  • Posts: 2761
    • View Profile

jz451

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: Need Help Writing Automation Script
« Reply #2 on: July 15, 2019, 12:35:27 PM »
Yes I did, tried the different flavors of converting to dataframe and got the same output as using pd.Dataframe()

Edit: After only selecting the "loans" column it correctly put it into a dataframe.

Also, is there a way to retrieve specific variables instead of all 119 variables when retrieving data for new loans?
« Last Edit: July 15, 2019, 12:58:28 PM by jz451 »

MarinBB

  • Jr. Member
  • **
  • Posts: 83
    • View Profile
Re: Need Help Writing Automation Script
« Reply #3 on: July 16, 2019, 11:33:37 AM »
Also, is there a way to retrieve specific variables instead of all 119 variables when retrieving data for new loans?

I'm not aware of a way to request only a subset of the fields in a listing via the API.

jz451

  • Full Member
  • ***
  • Posts: 118
    • View Profile
Re: Need Help Writing Automation Script
« Reply #4 on: July 17, 2019, 10:32:36 AM »
That was my assumption. Not a big deal but makes the list of datafame column names a little long to write out in a specific order. Seems like I answered my own questions for the most part after some trial and error. Last step is to figure out how to pickle the final model into the script file and setup the AWS modules.