Hadoop Jobs

I am ready to stand up a production-grade OpenSearch environment and want your help wiring the full ingest pipeline. The core stack will be OpenSearch with FluentD as the log-management and routing layer, together with Data Prepper (or an equally effective OpenSearch component) for enrichment and trace analytics. Source variety is high: classic log files from several services, change-data-capture streams coming off our databases, and a real-time event feed. Most of what flows in is semi-structured—think JSON snippets with the occasional free-form field—so careful parsing, field mapping, and transformation will be essential before anything lands in an index. What I need from you • A reproducible configuration (YAML / Docker-Compose or Helm is fine) that spins up OpenSea...

Hello, My name is Víctor and I am looking for a reliable freelancer who can assist me on an ongoing technical project. I want to explain clearly the context, the technologies, and what I need, so you can evaluate if the collaboration fits you. About the Project I’m currently working on a large and complex data-engineering / backend automation project involving: Technologies involved Google Cloud Python testing. Python (ETLs, tests, refactoring, data pipelines) CI/CD tools: Jenkins, Bitbucket, Artifactory Cloud environments: Google Cloud Platform, Azure Containers and dependencies SQL / PL-SQL Data validation and business logic implementation The project has multiple ETLs and dashboards, and some modules were already developed by previous freelancers. I need help valid...

I need a robust big-data workflow that will let me uncover how customers across Latin America actually buy. The primary aim is to analyze customer behavior, zeroing in on purchase patterns. For phase one the only source on hand is a wide set of structured and unstructured customer surveys gathered in Spanish and Portuguese markets. The job is to clean, standardize, and load these surveys into an environment that supports high-volume processing (Spark, Hadoop, or an equivalent cloud stack you favour). Once the data is stable, I want exploratory analysis, clustering, and predictive modelling that highlight: • Which demographic segments purchase which categories most frequently • Seasonal or regional spikes in demand • Correlations between stated preferences and actual...

Bank Loan ETL & Visualization Project Report 1. Abstract This project builds a complete ETL (Extract, Transform, Load) pipeline for bank loan analytics using PySpark and Python. It cleans, validates, and integrates branch, customer, and loan datasets into a unified master table. The pipeline standardizes financial data, generates analytical insights, and prepares the output for reporting and automated financial analysis. 2. Technologies Used Python PySpark Pandas Matplotlib CSV Files Java JDK (required for Spark) 3. Dataset Description This project uses three CSV datasets: – Branch details (branch_id, branch_name, branch_state) – Customer demographic information – Loan records linked to customers and branches 4. ETL Workflow The pipeline includes the following st...

I need a Python-based, fully documented solution that runs completely on my machine: an MCP server that talks to a FHIR R4 endpoint (the FHIR server itself may be remote), an agent using Ollama, plus a small React chatbot UI that demonstrates three key features: • Retrieve patient data – specifically medical history, current medications, lab results, medical images and radiology findings. • Display and download any related attachments. • Perform free-text searches inside those attachments. The server component should expose these capabilities through clean REST endpoints and log each FHIR call for traceability. Please follow FHIR best practices (paging, terminology handling, proper resource references) and keep the codebase test-driven. I am comfortable with Fast...

Project Overview: We are looking for an experienced Machine Learning Engineer specialized in audio processing and deep learning. The goal is to design, train, and deploy a high-performance AMD (Answering Machine Detection) model for telephony, using an existing dataset of approximately 67,000 labeled audio samples. The model must operate in real-time with low latency, and integrate into our existing calling infrastructure (Drachtio / Asterisk / FreeSWITCH / Vicidial). Mission Responsibilities: Analyze and preprocess the existing dataset (cleaning, balancing, train/val/test split) Extract audio features such as Mel-spectrograms, MFCC, STFT, normalization Design and train a CNN/CRNN model for AMD classification (Human / Voicemail / Silence / Fax / Other if needed) Optimize the model for...

I have an existing SAS program that handles end-to-end data processing for a single SQL Database source. The code cleans raw tables, applies a series of transformations, then produces several aggregated outputs that feed downstream reports. I now need the entire workflow re-implemented in PySpark running on Azure Databricks so I can retire the SAS environment and take advantage of Databricks’ scalability. You will receive: • The original .sas files with inline comments that explain each step • A data-dictionary of the SQL tables involved • Sample input/output datasets to verify parity What I’m expecting from you: 1. A well-structured Databricks notebook (or .py files) that reproduces the SAS logic for data cleaning, transformation, and aggregation. ...