Find Jobs
Hire Freelancers

Help me to finish-tune up a java scraping project TODAY -- 3

$30-250 USD

In Progress
Posted almost 9 years ago

$30-250 USD

Paid on delivery
Hello i'm looking for a senior developer to finish a project today. I need to add a gui to my project, some functionalities and improve speed between multithreading queries. I can pay a maximum of 70 usd. I'll hire the freelancer today. Thank you ______________________________________ What the project currently does? ______________________________________ Main goal is to obtain books data with contactable author. So are important both: book details and author contact details. To do this i loop in several nodes in a xml - called [login to view URL] - collecting the url of the bookstores. All the bookstores has the same layout, so the only change is the url itself. Every url i collect start a thread, which try to download the information of n books, starting from a number specified in [login to view URL] and finalizing in other number specified in [login to view URL] Because the way the bookstores is programmed, i first go to a form, search for the book' sku and then i go the search result and finaly i get the url of the book. Then i feed the url to an scraping api ([login to view URL] api) and i get the information of the book in a html table. Then i look for if the book has biography of the author, if so, i feed an api wich do a semantic analizis -called opencalais- and after get the answer in xml, and i get the important data like city where the author lives, email, and every contact detail. To get its phone and other relevant details, i feed another api -called intelius- which let me find all people named like the author living in specific city and i get phone numbers. So i put all the book information and the possilbe contact details in a row in a table in a html file Every succesful 50 records i start a new exportable file. Each export is a html file with a table which has in each row the information of a book and the associated author details and posible phones. The files exported includes a js file which let the user export the file to excel. What is required? _________________________________ Well, because i need to improve speed, I thought to start new threads (as set in [login to view URL]) to increase the number of threads querying a url, to download more than one book at once, save the data in a sqlite db and when it goes to ask if has contactable data in the author biography, then start a new thread in opencalais and if it gets data to feed intelius (city where the autor lives indeed), then start a new thread in intelius too, and add the information collected to the book record in the database in a new table in author table with the possible contact details. So the approach is increase the number of threads, and create a db (sqlite is the approach) to storage partial information, with book, and authors tables. I need help in design and program the new approach. All this scrapping information has log just in console and modifies the xml file so, its the way the user see the progress, but it needs a gui. Finally the gui must be able to export to excel the information too, so it needs some queries to the database and the exportability managment. ___ Other part is to get more information of new bookstores, increasing book information and narrowing the posibilites about the author contact details with the new bio information obtained ... well, may be it could be a second part of the project. ___ Project is attached.
Project ID: 7920145

About the project

4 proposals
Remote project
Active 9 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
4 freelancers are bidding on average $73 USD for this job
User Avatar
Hi sir, I am scraping expert, I have did too many similar projects, please check my feedback then you will know. Can you tell me more details? then I will provide demo data for you. Thanks, Kimi
$71 USD in 2 days
5.0 (353 reviews)
7.9
7.9
User Avatar
Hi, I can complete the requirements but i will need 4 days to complete and wont be able to complete today itself. Thanks
$77 USD in 4 days
4.9 (30 reviews)
5.4
5.4
User Avatar
I have 5 years of programming experience. I can help you solve this problem. When you work with me you will feel very comfortable.
$72 USD in 3 days
4.8 (53 reviews)
4.9
4.9
User Avatar
Hi, I am a java programmer and quite an expert in creating bots/scrapers. Please check the kind of jobs that I have done and the feedback that I have got. If you can do something on web manually, chances are high that I might create a bot that will do that automatically for you. Ping me to discuss things in detail. Thanks
$70 USD in 1 day
5.0 (12 reviews)
3.8
3.8

About the client

Flag of MEXICO
QUERETARO, Mexico
5.0
12
Member since Nov 13, 2013

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.