Software Development Engineer III

Job Description

Coding
- Develop; test; and implement data solutions based on functional / non-functional business requirements.
- Very good knowledge and hands-on in PySpark and dataframe APIs.
- Build Data Models to store big data in a most optimized manner.
- Implementing optimal data pipeline architecture.
- Identify patterns in code and refactor the code towards them where it increases understanding and/or maintainability.
- Produce proof of concepts to evaluate new tools.
- Very strong understanding of distributed computing concepts.
- Proven ability to write clean code that’s maintainable and extensible (design patterns; OOPs). Proven ability to write unit test cases.
- Comfortably debug/find complex problems.
- Consistently lead delivery of valuable features.
- Good working experience on Spark application performance tuning and optimisations.
- Good understanding of Hive and usage with Spark as well as any other open table format like iceberg.
- Good knowledge on data integration tools like Sqoop.
- Experience on orchestration tools like oozie; airflow etc.
- Basic knowledge on shell scripting.
- Comfortable with Git and GitHub.
- Experience and knowledge of Open-Source Platform for streaming data like Apache Kafka etc.
- Need minimal directions or guidance and collaborate with SDEs across teams working on the same or similar business problem.
Development Practise
- Automate everything by default
- Build a CI/CD pipeline
- Automate security scanning and performance testing as part of build.
Design
- Identify and apply appropriate design patterns to problems. Able to come up with multiple design solutions and justify the reason for chosen solution.
- Actively practice the boy scout principle leaves design code in a better state than when arrived.
- Demonstrate a good understanding of SOLID principles and how to apply.
- Able to understand the impact on the interface across applications.
- Able to identify risks and come up with mitigation plan.
- Able to come up with High and Low-Level Design (HLD) for a given requirement.
- Able to identify / highlight non-functional requirements and the design should cater for the same.
- Able to optimize the design to achieve maximum performance consuming minimal resources.
- Design should cater for the future considerations.
- Understands Requirement Traceability and ensures that all design components are traced back to requirements.
- Able to design re-usable and scalable modules.
- Able to anticipate common exceptions and design defensive mechanisms.
- Considers SLA (Service Level Agreement) and OLA (Organization Level Agreement) before designing Job schedule.
Security
- Write secure code by default.
- Use tools to perform deep security probes.
- Integrate security in DevOps.
Behaviours & Collaboration
- Lead group discussions on design and understand domain well enough to work with Product Managers to drive value.
- Work effectively with product stakeholders to communicate and translate their needs into improvements.
- Support production systems; resolve incidents and perform root cause analysis.
- Comfortably present information.
- Mentor and develop those around you.
- Share knowledge with the wider engineering community.