engineering
Posted May 5Commercial IT Senior Data Solution Developer
Houston, United StatesOn-site
Responsibilities
- responsibilities may include: Design and develop robust data pipelines, ETL/ELT processes, and workflow orchestration systems to automate data ingestion, transformation, and distribution across multiple platforms for Commercial Collaborate with data architects and analysts to translate business
- requirements Build and maintain scalable data infrastructure including data warehouses, data lakes, and streaming systems that support high-volume data processing and analytics workloads Implement data quality validation, monitoring frameworks, and error handling mechanisms to ensure reliable data delivery and maintain system performance standards Monitor system health, troubleshoot data pipeline failures, and implement automated recovery procedures to maintain operational reliability and minimize downtime
Requirements
- experience to join our Commercial IT team.
- Qualifications: Legally authorized to work in the United States Bachelor's degree or higher in Computer Science, Information Systems, Information Science, other related field, or foreign equivalent 5 or more years of
- experience in full stack Python development 5 or more years of
- experience in working with commercial or front office users in energy trading or commodity trading organization Advanced level of knowledge in SQL and cloud-based data platforms (snowflake,Databricks) Advanced level of knowledge of Github actions, CI/CD workflows, Docker container apps, and IAC (Terraform) Intermediate level of knowledge of building and orchestrating pipelines using Airflow, Databricks Jobs or ADF Intermediate level of knowledge of developing REST APIs (Flask, FastAPI, or similar)
- Qualifications: Excellent verbal and written presentation skills, with the ability to communicate clearly and persuasively Follow best practices for coding, testing, version control, peer review, and CI/CD automation Intermediate knowledge of application development capabilities and ability to swiftly grasp new concepts and technologies Intermediate knowledge of project processes and methodology to support project management initiatives and delivery Ability to work as a Team player, and self-driven
- experience in large distributed systems, data warehousing, data lake at scale Hands on
- experience and proficiency in Data Management tools – SQL, SDE, JSON, XML, scripting languages such as Python Big Data technology stack including NoSQL, Spark, Hive, Kafka, StreamSets, IICS, ADF etc. Intermediate knowledge of application development capabilities and ability to swiftly grasp new concepts and technologies