Log In
Sign Up
Responsibilities:
- Design, develop, and oversee the implementation of processes and tools to assess the quality of data used in LLM model training.
- Create metrics and KPIs to evaluate data accuracy, consistency, and relevance.
- Work with Engineering teams to develop automated logic checks that will identify inconsistencies and potential issues in the training data.
- Lead the integration of quality processes into existing data pipelines.
- Collaborate with Data Scientists to scrutinize annotation data and develop strategies for continuous data quality improvement.
- Provide feedback loops and ensure alignment of data quality with annotation guidelines.
- Engage with Machine Learning Engineers to determine how data quality variations influence LLM model performance.
- Recommend adjustments to data collection, preprocessing, and utilization based on model performance analysis.
- Keep abreast of the latest trends and advancements in data quality management.
- Recommend and implement enhancements to our quality processes, tools, and methodologies based on industry best practices.
Requirements:
- 7+ years of design/test/implementation/consulting experience in data quality management for machine learning model training
- Understanding of machine learning principles, especially in the context of NLP and LLMs
- Fundamental knowledge in relevant programming and tools (e.g., Python, SQL)
- Demonstrated experience in project management and cross-functional collaboration
- Exceptional analytical, problem-solving, and organizational skills
- Proven ability to think strategically about business, product, and technical challenges
