We are seeking a motivated and detail-oriented professional to join our team as an LLM Expert & Strategy Engineer. You will be responsible for managing the full lifecycle of Large Language Models (LLMs) and their applications.
Responsibilities
- Maintain broad knowledge of LLM capabilities, limitations, and emerging best practices.
- Recommend and implement LLM consolidation strategies (reducing model sprawl while maintaining coverage).
- Stay up to date on new model releases and assess cost/performance tradeoffs.
- Prompt & Pipeline Management.
- Design, optimize, and manage production prompt pipelines.
- Monitor and recommend changes for prompt drift across use cases.
- Maintain a versioning system for prompts and ensure best-performing prompts are deployed.
- Collaborate with product and engineering on scaling prompt/inference pipelines.
- Monitoring & Observability (Arize AI).
- Configure and monitor LLM performance dashboards with Arize AI (or similar platforms).
- Detect drift, anomalies, and regressions in model and prompt behavior.
- Build feedback loops to continuously improve prompt and model reliability.
- QA & Issue Investigation.
- Partner with QA to isolate whether issues originate from prompts, models, or infrastructure.
- Run controlled experiments to validate hypotheses about failures.
- Provide actionable root cause analysis and fixes.
- Model & Infrastructure Lifecycle.
- Manage and maintain optimal versions of all deployed LLMs.
- Ensure infrastructure is configured for cost efficiency and performance.
- Oversee safe rollout of model upgrades or replacements.
- Document and enforce standards for LLM deployment, evaluation, and rollback.
- Cross-Team Collaboration
- Work closely with engineering to align model changes with product requirements.
- Support product managers in understanding LLM capabilities/limitations.
- Share insights with leadership to guide AI roadmap and resourcing.
Requirements
- Minimum of 7 years of experience in software development.
- Minimum of 3 years of experience in LLM and AI model development.
- Good communication and team collaboration abilities.
Other Information
Reporting to: Project Manager
CTC: as per industry standards
Joining timeline: Immediate to 30 days
Location: Remote
Job Code: Moonpeak
About Us
Development Logics is an IT services company. We are headquartered in Dharamshala, Himachal Pradesh.
Our current offering includes Product Engineering, Digital Marketing, BPO and Staffing solutions. We are currently offering services to clients in the US, Europe, India and Australia.