data
Posted YesterdaySenior Big Data Engineer
at Qualys
Pune, IndiaOn-site
Responsibilities
- Drive technology explorations and roadmaps.
- Ensure the quality of architecture and design of systems.
- Effectively research and benchmark cloud technology against other competing systems in the industry.
- Be the thought leader in data platform and pipeline along with Risk Evaluation.
Requirements
- Possess expert knowledge in performance, scalability, enterprise system architecture, and engineering best practices.
- Leverage knowledge of internal and industry prior art in design decisions.
- Qualifications : Bachelor’s degree in computer science or equivalent 5+ years of total experience.
- Required Skills: Java/Scala, Apache Spark, Kafka, Hadoop, RDBMS (Oracle or equivalent), and NoSQL databases (Cassandra or equivalent).
- experience in design and architecture Big Data solutions using Spark 3+ years
- experience in working with engineering resources for innovation. 4+ years
- experience in understanding Big Data events flow pipeline. 3+ years
- experience in performance testing for large infrastructure. 3+ In depth
- experience in understanding various search solutions solr/elastic. 3+ years experience in Kafka In depth
- experience in Data lakes and related ecosystems. In depth
- experience of messing queue In depth experience in giving
- experience in understanding caching components or services Knowledge in Presto technology. Knowledge in Airflow. Hands-on
- experience in scripting and automation In depth understanding of RDBMS/NoSQL, Oracle , Cassandra , Kafka , Redis, Hadoop, lambda architecture, kappa , kappa ++ architectures with flink data streaming and rule engines
- Experience in working with ML models engineering and related deployment. Design and implement secure big data clusters to meet many compliances and regulatory requirements.
- Experience in leading the delivery of large-scale systems focused on managing the infrastructure layer of the technology stack. Strong
- experience in doing performance benchmarking testing for Big data technologies. Strong troubleshooting skills.
- Experience leading development life cycle process and best practices
- Experience in Big Data services administration would be added value.
- Experience with Agile Management (SCRUM, RUP, XP), OO Modeling, working on internet, UNIX, Middleware, and database related projects.
Experience
- Experience building scalable, enterprise-grade big data solutions using either Java or Scala. 2+ years of relevant
Additional details
- Come work at a place where innovation and teamwork come together to support the most exciting missions in the world! Description : We are seeking a talented Sr. Big Data Engineer to deliver roadmap features of Enterprise TruRisk Platform which would help customers to Measure, Communicate and Eliminate Cyber Risks.
- Working with a team of engineers and architects, you will be responsible for prototyping, designing, developing and supporting a highly scalable, distributed SaaS based Security Risk Prioritization product.
- This is a fantastic opportunity to be an integral part of a team building Qualys next generation platform using Big Data & Micro-Services based technology to process over billions of transactions data per day, leverage open-source technologies, and work on challenging and business-impacting initiatives.
- Provide technical leadership to the engineering organization on data platform design, roll out and evolution.
- Liason to product teams, professional services and sales engineers on solution and trade-off reviews and represent engineering in such conversations.
- Serve as a technical lead on our most demanding, cross-functional departments.
- Functionally decompose complex problems into simple, straight-forward solutions.
- Fully and completely understand system interdependencies and limitations.
- Able to document the details so it will be easy for developers to understand the requirements.
- Assisting developers with proper requirements and directions.