Find Jobs
Hire Freelancers

Ruby (on Rails) expert for Web Scraper / Web Scraping Job needed

$250-750 USD

Closed
Posted almost 5 years ago

$250-750 USD

Paid on delivery
Intro. I am looking for a multi-talented Ruby (on Rails) expert in web scraping. You are going to build an entire webapplication with Ruby (on Rails) - using the newest Rails 6.0.0 version and the latest compatible Ruby version. You will find all the links to Frameworks, Templates, Gems etc. below. Target website. The first website will be AirBnB's room / housing listings. Please take a very careful look at a few of those listings. AirBnB's is randomly generating id and name HTML-attributes to protect their site from scraping. The structure seems to stay the same though. You will always be presented with 300 search results - even in cities like New York City. To see all listings you have to adjust the price (increase, decrease). In addition to that, use the map to get all the results (zoom in and out, move the map). Backend. • You will use the Kimura framework for scraping jobs (link below) • You will create a "command script" that triggers an AWS EC2 server to be setup, configured and destroyed for every single scraping job (see tutorial / links below). This server will function as a proxy. You are going to use Terraform for that. • Authorization through CanCanCan (Guest, User, Manager, Admin) • Authentication through Devise (Basic User table with login is enough) • Historization of all listings through Papertrail Scraping job. The scraping job needs to run when triggered (e.g. if I want to have a new city to be scraped). Also, the job needs to re-scrape an existing listing - the old version can be saved through Papertrail, or you just create a new listing and reference the old with the new one. Frontend. I will provide you with a Bootstrap admin template that needs to be used for the frontend. This Bootstrap admin template is built on top of the Bootstrap framework, SASS, jQuery, HTML5, and CSS. So you need to know how to work with those! Links. [login to view URL] (Ruby Framework) [login to view URL] (Create AWS servers tutorial) [login to view URL] (EC2 proxies) [login to view URL] [login to view URL] [login to view URL]
Project ID: 20015968

About the project

11 proposals
Remote project
Active 5 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
11 freelancers are bidding on average $523 USD for this job
User Avatar
Nice to meet you. I've read the task description and can develop definitely. I'll provide you with the highly skill to complete this task in time. I'll also do my best for you. Please contact with me. Thank you very much.
$555 USD in 10 days
4.8 (12 reviews)
5.0
5.0
User Avatar
Dear Customer! I have built many scrapers. This is my favorite type of job. I am absolutely confident I can complete your project easily and on time. Ready to begin immediately and finish as soon as possible. Please contact to discuss in more details. Thank you.
$500 USD in 14 days
3.9 (17 reviews)
5.5
5.5
User Avatar
Dear Sir. Glad to meet you. I'm Web developer specializing in web scraping crawling and indexing web pages. Skills: python, scrapy, selenium, requests, beautifulsoup, mechanize, lxml, urllib2, automation, bots, spiders I worked with 50+ differenct individuals/startups/companies and helped them archive their goals. Here's what you will get by working with me: * Working only on projects that I can deliver 100% * Ridiculous quality of work * Smart suggestions on increasing results/revenue and decreasing costs * Responsibility for tasks/project I work on * Reliability * Short response time * Good sense of humor ;) Hope I could awake you interest in me. Thank you for your attention. Shuaixing.(call me Shuaixing L.)
$500 USD in 7 days
4.9 (7 reviews)
4.6
4.6

About the client

Flag of UNITED KINGDOM
Wem, United Kingdom
0.0
0
Member since Jun 16, 2019

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.