Data Architecture Jobs

I need a data entry specialist to input and clean data for a CMMS system. The data is in a database format and needs thorough validation and cleaning. Key Tasks: - Entering equipment records - Inputting maintenance schedules - Adding inventory information - Validating and cleaning existing data Ideal skills and experience: - Experience with CMMS systems - Basic understanding of P&ID's, equipment drawings, and schematics. - Proficiency in database management - Strong attention to detail - Prior experience in data validation and cleaning Looking for someone who can ensure accuracy and completeness in the data entry process.

Responsible for designing and implementing large-scale data migration and ingestion pipelines to move high-volume data from diverse sources into cloud platforms. Sources include HDFS, relational databases such as MySQL and PostgreSQL, and real-time streaming systems like Kafka. Develop and maintain robust data pipelines using PySpark, ensuring efficient processing of batch and streaming data. Implement automated scheduling mechanisms to orchestrate data workflows on daily and monthly intervals, ensuring reliability and timely data availability. Optimize data ingestion and storage through advanced performance tuning, partitioning, and compaction strategies to handle large-scale datasets efficiently. Ensure data quality, consistency, and fault tolerance across all pipelines. Deploy and ma...

I want to create Data governance tool

We are hiring an experienced Databricks Developer for support our data platform initiatives. This is a remote role aligned with US working hours. Location: Remote (Offshore – US Timings) Experience Required: 5+ Years (Data Engineering), 3+ Years (Databricks) Note: Budget is fixed. Please do not apply if you are looking to negotiate. Key Responsibilities Develop and optimize data pipelines using Databricks, PySpark, and Spark SQL Design and implement Delta Lake architecture (Bronze / Silver / Gold layers) Work on Lakehouse architecture and manage Unity Catalog Apply DataOps practices for scalable and reliable data workflows Optimize Spark jobs for performance and cost efficiency Required Skills Strong hands-on experience with Databricks Proficiency in PySpark and Spark SQL Experie...

I need a streamlined way to connect several data sources and set up an automated backup-and-recovery workflow. The focus is strictly on the data itself—databases, spreadsheets, APIs or any other source I might add later—so the solution you build has to pull from each location reliably, store copies securely, and make restoration straightforward. My ideal setup runs on a schedule, monitors for failures, and keeps versioned archives so I can roll back to any point in time. A lightweight dashboard or log that shows last-run status and any errors would be helpful, but command-line reporting is fine if that keeps things simpler. Deliverables • A working integration script, service, or low-code flow that connects the chosen data sources • Automated backup routine with...

Senior Data Pipeline / ETL Engineer (FastAPI, PostgreSQL, OpenSearch) – Build MVP Data Ingestion Pipeline for Financial Intelligence Platform Overview We are building a financial intelligence infrastructure platform that aggregates global corporate registries, sanctions lists, and ownership data to produce compliance-grade investigative reports. The GitHub repository, architecture documentation, and core backend services already exist. What we need now is a senior data pipeline engineer who can complete the data ingestion and normalization pipeline so that our MVP feature works perfectly. The primary MVP workflow is: Search a person or company → resolve the entity → check sanctions exposure → reconstruct ownership relationships → produce an evidence-backed re...