Senior Engineer (Agentic + Data/ML) - Remote

Senior Engineer (Agentic + Data/ML) - Remote

Posted 3 days ago by Lorven Technologies, Inc.

Apply
Negotiable
Undetermined
Remote
Remote
Apply
Amazon Web Services (AWS) Anomaly Detection Broadcast Advertising Closed Loop Cloud Computing Cloud Technology Data Centers Data Quality Device Tracking Software Machine Learning Metadata Public Cloud Python (Programming Language) Scheduling Software Engineering SonarQube Workflows

Summary: The Senior Engineer (Agentic + Data/ML) role involves joining a core engineering team at a leading U.S. video content provider, focusing on enhancing engineering productivity through the development of agentic retrieval workflows, data pipelines, and ML-driven scoring systems. The position requires strong expertise in Python and modern data/ML tools, along with experience in building production-grade orchestration systems on AWS. The engineer will design and optimize retrieval agents and integrate backend services to process code quality metadata at scale. This remote position is ideal for candidates with 5-10+ years of relevant experience in software, data, or ML engineering.

Key Responsibilities:

  • CES improvements: PR-level scoring, historical context, context-retrieval agent, automated score correction, training data prep from feedback.
  • Data/PR tracking: ingest PRs, link to commits/branches, track merges to main, backfills.
  • Orchestration & reliability: Dagster migration, retries, scheduling, monitoring, data quality alerts.
  • LLM/prompt optimization: DSPy-based prompt tuning, eval set creation, feedback-driven corrections, prompt cost/quality tradeoffs.
  • Metadata & classification: role/work-type classification, epic linking improvements, filters/explorer features.

Key Skills:

  • Agentic coding: build retrieval agents that pull code context/diffs/history to improve scoring.
  • Data pipelines: Dagster (preferred) or equivalent orchestrator; AWS-native pipelines experience.
  • LLM/Prompting: DSPy or similar prompt/policy optimization; prompt chaining; context packing.
  • Data modeling & analytics: PR/commit linking, branch/main tracking, epic linking; quality/effort scoring signals.
  • Backend integration: building services that ingest PR/commit metadata and compute metrics.
  • Eval/feedback loops: design and run eval sets; collect user feedback; automate score correction.

Salary (Rate): undetermined

City: undetermined

Country: undetermined

Working Arrangements: remote

IR35 Status: undetermined

Seniority Level: undetermined

Industry: IT

Detailed Description From Employer:

Role: Senior Engineer (Agentic + Data/ML)

Location: Remote

Project description

You will join a core engineering team within one of the leading U.S. video content providers, supporting hundreds of distributed software engineering teams and operating across a large scale infrastructure that spans internal data centers, broadcast facilities, and public cloud (AWS).
The project focuses on building agentic retrieval workflows, data pipelines, and ML/LLM driven scoring systems that enhance engineering productivity and code quality metrics. You will design retrieval agents, optimize LLM prompting strategies, build robust orchestration, and integrate backend services that process PR, commit, and code quality metadata at scale.

Target Background:
5-10+ years as a software/data/ML engineer.
Strong Python expertise and familiarity with modern data/ML tooling.
Proven delivery of production-grade orchestration systems (Dagster/Airflow) on AWS.
Practical experience with prompt/LLM optimization (DSPy preferred).
Comfortable working with Git/PR metadata, code diffs, backfills, and performance critical data pipelines.

Responsibilities

CES improvements: PR-level scoring, historical context, context-retrieval agent, automated score correction, training data prep from feedback.

Data/PR tracking: ingest PRs, link to commits/branches, track merges to main, backfills.

Orchestration & reliability: Dagster migration, retries, scheduling, monitoring, data quality alerts.

LLM/prompt optimization: DSPy-based prompt tuning, eval set creation, feedback-driven corrections, prompt cost/quality tradeoffs.

Metadata & classification: role/work-type classification, epic linking improvements, filters/explorer features.

Must have Skills

Agentic coding: build retrieval agents that pull code context/diffs/history to improve scoring (aligns with CES Phase 3 and quality metric work).

Data pipelines: Dagster (preferred) or equivalent orchestrator; AWS-native pipelines (ECS/EventBridge/Lambda/S3/Glue experience a plus); robust backfills and retries.

LLM/Prompting: DSPy or similar prompt/policy optimization; prompt chaining; context packing; eval design and execution.

Data modeling & analytics: PR/commit linking, branch/main tracking, epic linking; quality/effort scoring signals; anomaly detection for data quality.

Backend integration: building services that ingest PR/commit metadata, compute CES/quality metrics, and expose them to UI/API.

Eval/feedback loops: design and run eval sets; collect user feedback; close the loop with automated/manual score correction.

Nice to have

Experience with code-scoring/effort or quality metrics; SonarQube integration.

Graph-ish linking across artifacts (Jira/PR/commit/branch) for epic/work-type classification.

Experience tuning performance/cost for LLM calls (OSS LLM evals, batching, caching).

Observability for pipelines (metrics/traces/logs) and data quality alerting.

Apply
Similar Jobs
Loading data...