Find Jobs
Hire Freelancers

Build database to automatically import my ecommerce web scraped data.

$8-15 USD / hour

Closed
Posted over 5 years ago

$8-15 USD / hour

I currently have a web scraping service that scrapes a competitors website. I only scrape 3 brands. The web scraper service can then put the file into Google Drive, FTP or email it. This CSV file has really good data but it does need cleaned up. Currently I clean up the data manually each time. I also change prices manually, and sometimes change the description slightly. What I would like is to have a database (maybe access?) that automatically imports the csv from google drive, then looks for changes. I would like to set parameters in place that can do things like: If the product price is between $.02 to $50, then set the price to $.01 less. I also would like to get reporting of new products, removed products, products with price changes, etc. Then I would like this cleaned up date to automatically upload to my live ecommerce site, which is currently hosted by bigcommerce.
Project ID: 17542748

About the project

22 proposals
Remote project
Active 6 yrs ago

Looking to make some money?

Benefits of bidding on Freelancer

Set your budget and timeframe
Get paid for your work
Outline your proposal
It's free to sign up and bid on jobs
22 freelancers are bidding on average $12 USD/hour for this job
User Avatar
Hi, I can create such tool that will track changes. However I suggest that you do it as a web-based page utilizing your server's database instead of Access. I can also make changes in your ecommerce to automatically update price. Please give me all details on which products you scrape, how you scrape them and what you currently have on your websites so I can estimate this project.
$12 USD in 40 days
5.0 (332 reviews)
9.0
9.0
User Avatar
Dear Employer, Greetings of the Day! Thanks for sharing your requirements with us. For your project scope, I would be really glad to let you know that you can hire from our team of skilled developers and database experts who will perform the following tasks which are reflection of my understanding till now based on your posting (Please verify ). 1) Build database to automatically import my e-commerce web scraped data. 2)Set the parameters in accordance to your preference . 3)Reports generation for new products, removed products, products with price changes, etc. 4) Set the cleaned and scraped data to get automatically uploaded to the e-commerce site. It would be really nice if we can have a brief project discussion so kindly ping me on freelancer chat and share with us complete project specifications so that we can proceed further. Looking Forward to work with you on long term basis Warm Regards, Sovi
$12 USD in 48 days
4.9 (186 reviews)
8.6
8.6
User Avatar
Hello, how are you? I'm a senior web developer and I have rich experience in database management. Database seed, CSV import and export, statistical report, cron job, background service on server etc are no problem for me. Which framework do you use for scrapping? Maybe it is base on Python? Your job is not easy, but no problem for me. I will give you perfect result as you want within your time line. Let us discuss more via chat. Best regards. Thanks.
$12 USD in 40 days
5.0 (55 reviews)
8.0
8.0
User Avatar
Hi there, It can be done and we can make it an automated process where through a cron , on a particular time gap,we can update the data base of your big commerce site and it will get updated accordingly. we have done this earlier for several betting sites and it worked well for 1-2 months ,then it started crashing day by day only for one limitations on this process:-- Such Scraping are being done on two major parameters:- 1. url of the site 2. HTML structure of the pages. if any of them started getting changed ,then the cron will not effect as the scrapper will not work properly. Are you aware of this fact? It basically happens when the source website owner get to knows their site being scrapped,they started changing above changes very frequently to get rid of the scrapping. We can actually get rid of this issues as well in our solution if we can keep updating and changing whenever required depending on the source. I wanted to be honest with you from the beginning for that reason shared a true lacking in scrapping automation solution. Please let me know the target websites and the data types and also share your website reference too. -- Regards, Sumantra
$13 USD in 30 days
4.8 (66 reviews)
7.8
7.8
User Avatar
Hello, Myself having has 10 + years experience of web site development. Expert in Custom programming in php / Mysql / Jquery / Wordpress/ Joomla / OSC / Magento /OpenCart / Android / Mobile development / Responsive. Ready to start the work immediately. I am confident to handle your projects with dedication and good quality. Thanks and regards, Harpreet Singh
$11 USD in 40 days
4.9 (266 reviews)
7.6
7.6
User Avatar
Hello, I can help with you in your project Build database to automatically import ecommerce web scraped data. I have more than 5 years of experience in Database Programming, MySQL, PHP, Software Architecture, Web Scraping. We have worked on several similar projects before! We have worked on 380+ Projects. Please check the profile reviews. I can deliver your job with in your deadline. Please ping me for more discussion. I can assure the 100% job satisfaction. Thanks,
$12 USD in 40 days
4.9 (87 reviews)
6.8
6.8
User Avatar
We are highly expertise and experienced to developed custom eCommerce website and CMS website using WordPress, Woo commerce, Big commerce and Shopify. Please talk to us, we will start work on it straight away.
$11 USD in 40 days
5.0 (24 reviews)
5.1
5.1
User Avatar
I believe what you are looking for in this project is a spider with the ability to process scraped data efficiently. With this in mind, I suggest to use Scrapy - a Python spider. I'm currently working in a data mining project which utilize several cutting-edge technologies such as Scrapy for web scraping, Elasticsearch for indexing, AWS S3 for storage and Kurbernetes for packaging and deployment. The spider will patrol a list of urls every 2 hours to pull all newly posted articles. I believe I can apply my skills from this project and from 4 years experience working with various data mining technologies to handle your case study. I am willing to have a further discussion in your arranged time. Thank you.
$8 USD in 40 days
4.9 (16 reviews)
5.3
5.3
User Avatar
Hello There, I have gone through your requirement, I have understood your work. I have extreme experience in Web Scraping. Have done a lot of scraping using python. I scrapped data from many e-commerce websites and pushed that in CSV file & in a database. I use tools like scrappy, selenium, scoop for scraping. Waiting for a quick discussion on chat..!!! Akhilesh
$15 USD in 40 days
4.5 (11 reviews)
4.8
4.8
User Avatar
Hi Do you need quality work? I am expert on WordPress Enfold Theme, Divi Theme. Using Divi Theme I can webpages if you required. Lets discuss about the project. Sincerely Touhida
$12 USD in 40 days
2.9 (2 reviews)
1.5
1.5
User Avatar
Hello, From what I understand, You want to pull the products/data of your competitor and then save them as a 'base'. Then you want to keep watching and if they change,add or remove or move products/server then you want to know about it. (either remove value from your database or add new value). I have built bots and automation, web scraping programmes many times. I'm confident I can complete your goal. Admittedly i'm not familiar with ecommerce but i'm not worried, I can learn this for your purpose. I understand its some form of online shop? Presumably they will have an API, or if not I can work around this. Please let me know if you would like to discuss this further. Regards Anthony
$22 USD in 40 days
0.0 (0 reviews)
1.2
1.2
User Avatar
Hi, To introduce myself, I am working as a data analyst for NTT Data, i have experience in data pump, my proposal will be to import your csv files to any relational db which comes as free license line postgre.. We can discuss more on this. Thanks, Pramod
$11 USD in 40 days
0.0 (0 reviews)
0.0
0.0
User Avatar
Hello, 2 years experienced data analytics company and I can make your requests using MSSQL, Excel, CSV, XML I am expert how to manage data and I think I can do this works I wish we will work. Thank you. Furkan
$15 USD in 20 days
0.0 (0 reviews)
0.0
0.0

About the client

Flag of UNITED STATES
Macedonia, United States
5.0
7
Payment method verified
Member since Dec 8, 2014

Client Verification

Thanks! We’ve emailed you a link to claim your free credit.
Something went wrong while sending your email. Please try again.
Registered Users Total Jobs Posted
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Loading preview
Permission granted for Geolocation.
Your login session has expired and you have been logged out. Please log in again.