I have an existing MongoDB app that is used by a Spring Boot App to provide REST services to Front End (Angular) clients. Need to port the MongoDB schema over to Apache Cassandra and Port the ETL logic from Spring Boot over to Apache Storm
Need to pair program with someone in moving data around and doing data wrangling in Amazon Web Services. Time zone is in UTC +8 area or if you can do these times. Maybe an hour a day weekly or more time depending on workload. Tools used: Python, SQL, ETL, lambdas, Redshift, Mysql When bidding on this project let me know what number is a baker's dozen
Hello, I am a data scientist currently working on a data engineering task and I would need some assistance. I have the data on S3 in 7z JSON, which I would like to convert to Parquet run ETL jobs and finally create a dashboard. I am trying the following: Glue -> Parquet -> Athena or Batch -> docker image -> R jobs I am running into issues ...
good morning, we have an SQL server 2008 R2 on windows 2008 R2, it has an ETL process that builds a Cube. The ETL has worked fineso far, but now it does not complete (continues forever). would you be interested in taking a look, launching it in debug mode and see what is the problem?
Process and clean a list of large amount of raw data (export and import data) obtained from trade facilitation companies. Build a framework and process for data cleaning. Some level of text cleaning and API call is needed to successfully map each row of data into a correct category (eBay or Google API). Develop a program to process more similar
Transition 15 fields and 1 million records from source 1 to source 2. More information and attachments will follow after your bid.
We have a need for someone to help in development and operations work. Initially some ETL development using Python/pandas. Knowledge of cassandra/nosql databases, Ubuntu, virtual environments (docker), and parallel/distributed programming is required.
I want to customize an open source ETL in order to made some changes and integrate some new features/Solve bugs and trace/debug errors.
I am working as Data Warehouse Engineer. And I need some support(proxy) to help in my work. Its related to Data Migration and requires creating ETL mapping documents, maintaining them, ER diagram development, writing SQL queries, creating SSIS packages.
Need a female proxy with good knowledge of hadoop ecosystem such as hive, mapreduce, hdfs, hbase. should know spark [login to view URL] that would be good to know - java, scala, python. knowledge of ETL processes are a plus
I started building a visual ETL tool but now lacking the time to finish it up. I need a strong WPF and C# developer to help me cross the finish line. Skillset includes the following: - C# WPF view/layer - Data mapping technical and best practice with SQL Server in the backend
I'm looking for a python expert who deep knowledge of AWS S3, data warehouse including obtaining data with external APIs. You should be python expert with knowledge of ETL and Mongo. This is long-term task, so you need to commit everyday and update me via call. Hope to hiring python guru. Thanks.
...using MSSQL 2014 and need an SSIS package for data importing and exporting that includes logging errors and following an alternate path when errors encountered. Data is from our database to Salesforce using Kingswaysoft to access Salesforce objects. We would prefer the method follow standard etl patterns. Object is about a dozen fields; our main conc...
I'd like to create a dashboard at a Team level, then also rep level too from various KPIs to be defined. The data at the moment is one (perhaps 2) table linked to a local free sql . We can control/update the table/view so calculations can be done at source if required by etl process or in excel The tables contain all the information we shoul...
We are looking for an experienced python developer who can help us with various automations. One of the tasks will be writing plugins for the ETL scheduler Apache Airflow. If you have experience with airflow, that's a bonus. If you don't that's fine too as we can catch you up on it, but you need to have a strong motivation to learn ...
Need a developer with skills in Python development. The goal would be create queries and scripts for deployment of features in the sandbox and production database. The data extraction would need to be done from AWS S3, data warehouse, including obtaining data from multiple external API’s. Please answer following 3 questions. These questions are important
Hi, I need a ETL talend or Pentaho Kettle where it connects to oracle and replicate tables to postgress this would be scheduled to be done automatically and only replicate new records
Looking for someone to be available for a full-time ETL-BI Specialist. The work hours will be Monday to Friday (35 hours a week) during EST hours (Usually 8:30 – 4:30). The client hosts the DB/Applications with student data in a SaaS model as well as in the customers’ data centres. The position length is 1 year with a pos...
Position: ETL Developer / Sr. ETL Developer Technology: SSIS & SQL Server (Python Optional) Experience: 4 to 5 years No. of Positions: 1 immediate joiner. Location: Nagpur Type of Opening: Freelancing – for 3 months & based on inputs might extend to a year’s contract.
As a Data Scientist on our team, you will be responsible for solving complex big-data problems for various clients (on-site and off-site) using data mining, statistical analysis, machine learning, deep learning. One of the primary responsibilities will be to understand the business need and translate it into an actionable analytical plan in consultation
...prepare all processes according to business requirements Develop, enhance and maintain SQL Stored Procedures and Shell scripts Design, build and launch efficient & reliable data pipelines to move and transform data (both large and small amounts) Develop and perform tests Provide support and maintenance on ETL/DWH processes and documentation
...This project is create a scen for the attached story board. You will have access to my library of videos so you can extract images and understand my style. This new video is just a simple introduction to new ETL software application. It is intended for audiences with not knowledge of ETL. It just conveys a simple message of a new data ...
...would be responsible for ETL development leveraging SSIS against the Marketing and Merchandising Enterprise Data Warehouses. This position reports to the ETL Manager. You must demonstrate a proficiency with data modeling and Microsoft SSIS. You will work closely with the peer members of the enterprise data and other IT teams to ascertain expert understanding
Should be able to develop code for ETL Processing using Pyspark, Spark Sql, Scala. We have existing ETL jobs. We are planning to replace the ETL tool with Spark. For that purpose we are implementing Spark to do ETL process. Skill Set required : Spark, Python, Sql, Scala, AWS, Git, Github
...do at BluePi: BluePi is a born in cloud organization with one of the main focus area being Big Data and Analytics. At BluePi we solve different business problems for clients from different industrial domains like Media, Retail, Logistics, Clothing, etc. At Bluepi we build, develop and deliver big data solutions including ETL, Data warehousing, DataLake
We need remote ETL developer in DWH for long-term work. Requirements: 5 years of experience on Informatica. Ability to translate business requirements into technical requirements Knowledge of Data warehousing(Architecture, Data modelling) Shell scripts and stored procedures in Informatica Good knowledge in Unix shell scripting Experience in debugging
Need a 15+ years of resource as DW solution architect with strong skills sets in Architectural Data Warehousing and BI solutions. Data Modeling, Dimensional Modeling, ETL on contract basis. Location : Bangalore ( Hosur road )
Need a freelancer to build an ETL process using Google Cloud DataFlow (Apache Beam) project. The Pipeline would begin with an ERP Source residing in IBM DB2 or MSFT SQL Server and would end in a Data Warehouse in Google BigQuery. It would be a fix cost project and all the cloud platform services would be made available to the Freel...
We have a .NET project currently that is part of an ETL pipeline. The program extracts data from various sources and sends it to AWS. This is a pretty simple project, source code will be provided once the project is accepted. The scope of the project 1. Convert project to .NET Core (highly desired) 2. Currently, the project uses a standalone [login...
This project is to re-engineer current windows based python project to linux, using dockers, change some ETL routines, and develop plotly dashboards to deliver daily data to clients. Depending on capability and skills, this could be a long term position to continually manage and improve operations once the new platform is running. Cassandra and spar...