Data Platform Software Engineer / Senior Backend Engineer Data & Streaming
Posted Today by Apetan Consulting
Negotiable
Undetermined
Remote
Remote or New Jersey
Summary: This role is for a mid-level Software Development Engineer focused on backend engineering, specifically in real-time data pipelines and distributed systems using technologies like Python, Kafka, Flink, and AWS. The position involves migrating workloads from Snowflake to enhance cost efficiency and reduce latency, while collaborating closely with a Staff Engineer. The ideal candidate should possess strong coding skills and experience in data processing. The role emphasizes building scalable systems and applying modern engineering practices.
Key Responsibilities:
- Build highly scalable distributed systems that leverage event-based and streaming data pipelines to handle ingestion and processing of robot, manufacturing, and clinical data
- Enable users by providing self-service APIs and applications to access and interact with data
- Work closely with core engineering teams to consistently evolve data models based on growing business needs
- Apply software development best practices such as CI/CD, automated testing, infrastructure-as-code, and microservice architectures
- Effectively participate in the team's planning, code reviews, KPI reviews, and design discussions leading to continuous improvement in these areas.
Key Skills:
- Exceptional quantitative background (Computer Science, Math, Physics, and/or Engineering), or at least 5+ years of industry experience in a quantitative role
- Fluent coding with Python and Java
- Proven experience building data pipelines and working with distributed systems using technologies such as Kafka/Flink, Snowflake, AWS Lambdas
- Excellent written and verbal communication skills
- Proven understanding of best engineering practices, such as unit testing and integration testing, and deployment patterns.
- Experience with Kubernetes
- Experience with SQL and relational databases
- Ability and enthusiasm to work collaboratively and cross-functionally, and take end-to-end ownership to deliver results for customers
Salary (Rate): undetermined
City: undetermined
Country: undetermined
Working Arrangements: remote
IR35 Status: undetermined
Seniority Level: Mid-Level
Industry: IT
Title: Software Development Engineer 3
Locations: Remote
Project Duration: 12 Months
This is a midlevel backend engineering role focused on real time data pipelines and distributed systems, using Python, Kafka, Flink, and AWS.
Project is migrating workloads out of Snowflake to reduce cost and latency, and they're looking for a strong coder with data processing experience who can grow under a Staff Engineer.
Software Development Engineer 3 (Sunnyvale, CA)
This role is a senior-level backend/data platform engineering position supporting Intuitive Surgical's robot, manufacturing, and clinical data ecosystems. The engineer designs and builds highly scalable, event-driven and streaming data platforms that power ingestion, processing, and self-service access to large volumes of robotic and clinical data. The role partners closely with core engineering teams to evolve data models, APIs, and platform capabilities while applying modern engineering best practices including CI/CD, automated testing, infrastructure-as-code, microservices, and Kubernetes-based deployments. The ideal candidate is a strong Python/Java engineer with hands-on experience building distributed data pipelines using technologies such as Kafka/Flink, Snowflake, AWS Lambda, Kubernetes, and SQL, with bonus strengths in Apache Iceberg, Terraform, GitLab CI/CD, and CNCF-native cloud platforms.
Responsibilities
- Build highly scalable distributed systems that leverage event-based and streaming data pipelines to handle ingestion and processing of robot, manufacturing, and clinical data
- Enable users by providing self-service APIs and applications to access and interact with data
- Work closely with core engineering teams to consistently evolve data models based on growing business needs
- Apply software development best practices such as CI/CD, automated testing, infrastructure-as-code, and microservice architectures
- Effectively participate in the team's planning, code reviews, KPI reviews, and design discussions leading to continuous improvement in these areas.
Qualifications
Skills, Characteristics, and Technology:
- Exceptional quantitative background (Computer Science, Math, Physics, and/or Engineering), or at least 5+ years of industry experience in a quantitative role
- Fluent coding with Python and Java
- Proven experience building data pipelines and working with distributed systems using technologies such as Kafka/Flink, Snowflake, AWS Lambdas
- Excellent written and verbal communication skills
- Proven understanding of best engineering practices, such as unit testing and integration testing, and deployment patterns.
- Experience with Kubernetes
- Experience with SQL and relational databases
- Ability and enthusiasm to work collaboratively and cross-functionally, and take end-to-end ownership to deliver results for customers
Bonus points:
- Experience on a Platform team
- Experience with Gitlab CI/CD or other CI tooling
- Experience with Apache Iceberg
- Experience with Terraform and general IaC best practices
- You're familiar with CNCF projects and have successfully used them in the past
Project is Refactor existing Data Pipelines and focus on Distributing processing and data pipelines. Real-time data processing on the cloud (AWS, Kubernetes).
Move some workloads out of Snowflake into real-time frameworks using Kafka + Flink on AWS EKS (Kubernetes)
Goals: reduce cost and improve latency for end customers
- It's a Mid level backend engineer who builds the systems that move, process, and expose large volumes of data in real time.
- Design, build, and maintain backend services and distributed data processing pipelines
- Work on real-time streaming pipelines using Kafka and Flink
- Help migrate workloads from Snowflake to real-time systems
- Develop and deploy services running on Kubernetes (AWS EKS)
- Collaborate closely with a Staff Engineer who will lead the project
- Use DevOps tooling (CI/CD, Terraform) as needed to support development and deployment
Candidate must have done project related to distributed systems work, especially data processing pipelines.
Top Required Skills (Non Negotiable / Highest Priority)
- Python (most important)
- Java (important, but secondary to Python)
- Must be fluent, not just basic familiarity.
- Cloud development experience: Ideally AWS, Google Cloud Platform
- Data pipelines / data processing experience:
- Ideally with Apache ecosystem:
- Kafka
- Flink
- (Iceberg as a plus)
- Kubernetes (doesn't need to be an expert; exposure + ability to learn is fine)
- CI/CD + Terraform is a bonus