Senior Data Engineer

Techgene Solutions LLC Logo
Techgene Solutions LLC
Alpharetta, GA, United States
  • Construction
  • FullTime
  • PartTime
  • Applications have closed

Position: Senior Data Engineer (PySpark Databricks Oracle/PostgreSQL Migration)

Location: 5800 Windward Parkway, Alpharetta, GA 3000 ++(Only Locals) 3 Days onsite++

Job type: Contract

Exp Level: Min 9 Years

  • Expert-level proficiency in PySpark and Databricks.
  • Primary Skill Focus: PySpark, Databricks (Hands-on coding test expected)

Skills Required:

  1. RDBMS Expertise:
  • Strong hands-on experience with Oracle and PostgreSQL databases.
  • Deep understanding of database architecture, design, indexing, and performance tuning.
  • Expert-level SQL skills: queries, stored procedures, functions, triggers, and views.

2. Database Migration

  • Proven experience in end-to-end database migration projects (preferred: Oracle PostgreSQL).
  • Strong ability to perform data mapping, transformation, validation, and reconciliation.
  • Experience using migration tools, scripts, and automation frameworks.

3. Data Engineering & Analysis

  • Advanced proficiency in Databricks for large-scale data processing.
  • Expert in PySpark and Python for data transformation and analytics.
  • Ability to build, enhance, and optimize complex ETL/ELT data pipelines.

4. Job Scheduling & Automation

  • Experience creating and maintaining Databricks jobs for scheduled reporting.
  • Familiarity with workflow orchestration tools (Airflow, ADF, Step Functions, etc.).

5. Performance Optimization

  • Strong background in performance tuning for Oracle and PostgreSQL.
  • Experience with index strategies, query optimization, execution plan analysis, and caching.

6. Cloud Platforms

  • Good understanding of AWS Cloud, including RDS, S3, EMR, Lambda, Glue, or similar services.

Responsibilities:

  1. Database Design & Development:
  • Design and develop robust database solutions that meet data storage and retrieval requirements.
  • Create scripts and procedures to automate routine database tasks.
  1. Migration & Implementation:

Lead the migration process from Oracle to PostgreSQL, ensuring data integrity and minimal downtime.

Develop comprehensive migration plans and execute them proficiently.

  1. Support & Maintenance:

Monitor database performance and implement necessary improvements.

  1. Reporting & Analytics:

Develop and maintain Databricks jobs for generating business reports and analytics.

Provide insights from data analysis to support decision-making.

Apply for job