Senior Data Scientist — AI Evaluation & Quality (Remote)

Job description

You'll join the AI Team — the group driving all AI products and technology at Finom

We build and ship AI across the company: AI financial co-pilot, voice agent, and internal AI-powered processes

Our belief: your AI agent is only as good as your eval loop — we can build AI as good as the evals we run on it

Your mission: own that eval loop across every AI product we ship — pre-launch quality gates, post-launch monitoring, continuous improvement

You'll work directly with our AI Quality lead, Igor Kolodkin

Close collaboration with AI engineers, Product, and domain experts across the company

Core stack: Databricks, DeepEval, Claude Code

What You Will Be Doing

• Own and extend our offline eval suite across products — datasets (capability + regression), judges, metrics
• Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency
• Close the production feedback loop: mine failure patterns from real traffic → turn them into regression cases → propose fixes to Product and domain experts
• Harden methodology: judge stability, non-determinism handling
• Translate numbers into decisions – weekly syncs, clear trade-offs, no dashboards for their own sake

Must-Haves

• Python and SQL — you can build an analysis end-to-end
• Solid foundation in statistics — sampling, hypothesis testing, variance, understanding what a noisy metric is
• Analytical mindset — you start from the business question, not from the tool
• 3+ years in analyst / data scientist roles, at least one in a product context

Nice-to-Haves

• Experience in quality analytics for ML systems — ranking, recommendations, classification, etc.
• Hands-on experience evaluating LLM applications (RAG, agents, tool use, judges)
• Experience building LLM agents — side projects, toy builds, personal experiments all count

How we work — one thing we mean seriously

• AI-assisted coding is our default authoring environment, not a bonus
• Claude Code is our main tool — you'll reach for it for SQL, Python, analyses, dashboards, and internal scripts
• We're looking for analysts who are already curious and fluent with AI coding — or genuinely excited to become fluent fast
• We care about what you ship and how clearly you think
• If this idea excites you rather than worries you, you'll feel at home here

What You Will Get In Return

Make a genuine impact on the product

Join our upward trajectory, and grow with us. We provide the resources and opportunities for continuous personal and professional development, empowering you to make a genuine impact on our evolving product.

Work in the EU

Embark on this exciting journey with us and enjoy the flexibility of traveling and working remotely or in a hybrid model across Europe.

Become a stock options holder

Unlock your inner entrepreneur and align your aspirations with ours through our Stock Options Program. This exciting opportunity is available to every team member, from junior team members to our founders.

Receive unwavering support and care

Finom stands by you at every step, embodying our commitment to your well-being and success reflected in our modern, friendly, and eco-conscious corporate culture. We offer constant support and care to ensure your Finom experience is successful and fulfilling.

Work & Swim program

Immerse yourself in our exclusive Work & Swim Program. Spend one month in a comfortable corporate apartment in enchanting Cyprus. It's the ideal opportunity to strike the perfect work-life balance while enjoying breathtaking Mediterranean views.

Equal Opportunity Statement

At Finom, we're an equal opportunity employer and value diversity at our company. We embrace diversity and invite applications from all walks of life. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, disability status, or other applicable legally protected characteristics.