Scality S3 project with MongoDB as a metadata database

Scality S3 project with MongoDB as a metadata database

Scality S3 is an open source Amazon S3 server written in Nodejs. It uses LevelDB for storing objects metadata, which would be useful for small projects however it is not reliable for large-scale projects and data might become inconsistent after a system crash or power failure. Also creating backup is not an easy task and is very tricky. The goal of this project is to replace LevelDB with MongoDB.

Scality S3 provided different drivers for handling metadata like bucketfile, in_memory, etc and uses a wrapper to use proper driver according to user configuration. We have to add a MongoDB driver and then set backend metadata to MongoDB in application configuration in order to keep Scality S3 files untouched and add our features.

There is another thing to keep in mind that In S3 API there is a function that lists all objects of a bucket with specified limit. Current implementation loads all objects and filters them with application. Although it uses stream, this would definitely makes problem as data grows and creates performance issues. We need to filter data in database layer by passing required filters and using MongoDB aggregate framework to avoid performance problems.

Another important thing is that Scality S3 has implemented all amazon features like versioning and ACL, and using MongoDB should not lead to ignoring these features or making them work improperly. To do so, we need to use VersioningRequestProcessor which uses WriteCache and WGM classes and we have to re-implement these classes as well.

Finally, the main project modification should be as least as possible in order to keep project updateable and receive latest features and bug fixes.

Skills: node.js, NoSQL Couch & Mongo

See more: project manager database open office, project hotel reception database, prepare project proposal mysql database client, project join mathtype database, computer project show web database msfrontpage, project management access database, quicktime iphone open stream, php copy failed open stream, project management tool database design, project using jquery database, php warning copy failed open stream permission denied, project create music database, php functioncopy failed open stream permission denied

About the Employer:
( 5 reviews ) Frankfurt, Germany

Project ID: #15343311

18 freelancers are bidding on average $2635 for this job


Dear. I am a programmer in china - nodejs expert. I have enough experiences to complete your job. I could help you as well. Relevant Skills and Experience nodejs, mongo db Proposed Milestones $2268 USD - complete I More

$2268 USD in 30 days
(83 Reviews)

Hi, I can write a mongodb driver for Scality S3 project, will create optimised queries for filtering data. Will do the versioning and ACL, whatever Scality has created will be ported to use MongoDB Relevant Skills More

$3000 USD in 20 days
(32 Reviews)

Key Projects ⓐ Project: Ed-Admin+ [url removed, login to view] Ed-admin+ is a browser-based software application which streamlines and simplifies education management, integrating administrative, teaching and learning t More

$3000 USD in 30 days
(9 Reviews)

Hi, I have gone through the project description and we can surely help you out here. We have been developing/maintaining various web applications in NodeJS (both Express and Sails) Relevant Skills and Experience These More

$2000 USD in 45 days
(5 Reviews)

We are a London (Shoreditch) based web development studio. Please get in touch to discuss further. Following are some of our recent projects; [url removed, login to view] A social media post scheduler and manager More

$3060 USD in 30 days
(1 Review)

Hello, Hope you are doing Great. We are a team of young and enthusiastic Web developer having 5+ years experience,good knowledge and expertise in Java Script,AngularJS,Node Js and MongoDB. Relevant Skills and Exper More

$2533 USD in 30 days
(22 Reviews)

Hi there, Thank you for sharing your requirement. We are a team of technocrats having 8+ years of experience in Web and Mobile design plus development. Talking about our expertise, we possess very good expertise More

$2400 USD in 30 days
(7 Reviews)

We have full experience in PHP, MySql, HTML5 and Many php frameworks. We have developed school management website, shopping cart and e-commerce project. Relevant Skills and Experience php,mysql,html5,css,javascript ht More

$2368 USD in 30 days
(3 Reviews)

yes i have worked on S3 and other amazon web services can we discuss more in details to understand your requirements in more details? Relevant Skills and Experience i have relevant skills and experience as per your pr More

$2938 USD in 30 days
(1 Review)

Hi, sir. Thanks for your job posting! I am a senior web developer with +6 years experience and your project is very interesting for me. I will do my best for your job. Relevant Skills and Experience PHP/Laravel/Cod More

$2500 USD in 30 days
(2 Reviews)

Hi, Good day! We have read your post & would like to discuss as I have expertise in developing applications with complex databases using NodeJs, MongoDb etc. & have delivered over 500+ projects in last few years. More

$2500 USD in 30 days
(2 Reviews)

Hi there, I have good experiences on Node.JS, MongoDB and AWS for 3 years up, so I can complete your project as well. Please inform me. Relevant Skills and Experience Node.JS, MongoDB, AWS Proposed Milestones $2500 U More

$2500 USD in 30 days
(2 Reviews)

We are a Team of Data Scientists having healthy experience into Big Data technologies like Hadoop,Scala and Spark etc. Relevant Skills and Experience The Team has qualified engineers having expertise in solving comple More

$2388 USD in 30 days
(0 Reviews)

Hi, we are open for a new project for right now. Need to clarify as many details as possible in order to provide you with an accurate quote and our vision. Let's discuss. Relevant Skills and Experience Our agency has More

$2500 USD in 30 days
(0 Reviews)
$2500 USD in 30 days
(0 Reviews)

Hey There, Thanks for posting this project. I just checked the details of the project and this is something which my company can help you with. Relevant Skills and Experience Square Pixel is a company with a team o More

$3000 USD in 45 days
(0 Reviews)

Hi We are AMAZON specialist , we work on : Amazon FLEX Blocks Amason Data Scrapping We are aware of Scality S3 and its features & how it is integrated to Amazon bucket files being implemented. We will repla More

$3055 USD in 30 days
(1 Review)

We are happy to bid on this project. We provides dedicate developers and Development and design services as well. ***** We are having a great team of Mobile developers with 7-8 years of experience Relevant Skills a More

$2916 USD in 30 days
(0 Reviews)