Pages

Fin's own interview guide describes this round as: "We will ask you a series of rapid-fire questions related to different aspects of ML. These won't be too deep: as a ballpark, we expect them to be around three minutes each." That's roughly 8-10 questions in 30 minutes, for a Staff/Senior ML Scientist role at an AI customer-support company.

This round tests breadth, not depth: working knowledge over academic theory, the ability to connect concepts into a system, and clarity under time pressure. Fin builds an AI support agent, so expect the panel to weight questions toward LLM and RAG system design more heavily than classical ML. This guide has five parts: a tiered concept checklist to scan the night before, worked 3-minute answers for the questions most likely to come up, the follow-up questions each worked answer tends to trigger, a rehearsal plan for practicing out loud, and a breakdown of how much prep time to spend where.

Section 1: Concept Checklist

Each pillar is split into tiers. Tier 1 is must-know for any ML basics round. Tier 2 is Fin-specific: topics an enterprise support AI company is likely to probe given its product. Tier 3 is nice-to-have depth that signals seniority but is less likely to come up directly. Under time pressure, drill Tier 1 first, then Tier 2, and treat Tier 3 as a bonus.

Pillar A: Classical ML Fundamentals (~30% of questions)

Tier 1 (must-know)

Bias-variance tradeoff: expected error breaks into bias squared plus variance plus irreducible error; simple models underfit (high bias), complex models overfit (high variance); regularization, ensembling, and cross-validation navigate the tradeoff (deeper dive: your interview-preparation and fundamental-statistics posts).
Overfitting and regularization: train loss falling while validation loss rises is the tell; fix with L2/L1, dropout, weight decay, early stopping, or data augmentation; L2 suits linear models, dropout suits neural nets.
Evaluation metrics: accuracy is misleading on imbalanced data; precision matters when false positives are costly, recall when false negatives are costly, F1 balances both, and AUC-ROC is threshold-independent; for LLM outputs add ROUGE, BERTScore, and LLM-as-judge, each with its own bias.
Handling imbalanced data: stratified splits, class weights, over/undersampling, threshold tuning, and cost-sensitive loss all correct for skewed classes.
Decision trees vs linear models: trees capture non-linearity natively but overfit easily; linear models are interpretable but need manual feature engineering; ensembles like Random Forest and XGBoost usually beat either alone.
Cross-validation and data leakage: k-fold CV estimates generalization; use stratified folds for imbalanced data and time-series splits for temporal data; never run random CV on sequential data, it leaks the future into training.
Hyperparameter tuning: grid, random, or Bayesian search, with early stopping on a validation set; watch for overfitting to the validation set itself if you tune too long.
Learning rate and optimizers: SGD is slower but often generalizes better; Adam converges faster; warmup followed by cosine decay is the standard schedule; AdamW decouples weight decay from the gradient update.
Activation functions: ReLU is the standard, no saturation; GELU is the smooth variant used in transformers; sigmoid and tanh saturate and cause vanishing gradients.
Batch norm vs layer norm: batch norm normalizes across the batch and needs a large batch size; layer norm normalizes across features and works even at batch size 1, which is why transformers use it.
Residual connections: skip connections prevent vanishing gradients and are what let networks go genuinely deep.

Tier 2 (Fin-specific depth)

Feature engineering vs deep learning: engineering wins on small data or when interpretability matters; deep learning wins once you have volume of unlabeled data to pretrain or fine-tune on.
Batch size effects: larger batches train faster and more stably but can generalize worse; smaller batches are noisier but often generalize better; 32-256 is the typical practical range.
Double descent: classical theory says error falls then rises with model capacity; modern overparameterized models often show a second descent past the interpolation threshold, thanks to implicit regularization.

Pillar B: Applied Statistics (~25% of questions)

Tier 1 (must-know)

Bayes' theorem: P(A|B) = P(B|A) * P(A) / P(B); prior belief before data, posterior belief after; underpins Naive Bayes, RLHF reward modeling, and MAP-style regularization (deeper dive: your interview-preparation and fundamental-statistics posts).
P-values and hypothesis testing: a p-value is the probability of data this extreme given the null is true, not the probability the null is true; reject below 0.05 by convention; Type I is a false positive, Type II a false negative, and power is 1 - beta.
Confidence intervals vs credible intervals: a frequentist 95% CI means 95% of intervals built this way across repeated experiments contain the true parameter; a Bayesian 95% credible interval means, given the data observed, there's a 95% probability the parameter falls inside it.
Multiple testing problem: testing many hypotheses inflates false positives; Bonferroni correction is simple but overly conservative; Benjamini-Hochberg controls the false discovery rate and is the more practical choice.
A/B testing pitfalls: novelty effects, peeking or p-hacking, uncorrected multiple comparisons, unequal variance across arms, and underpowered sample sizes are the usual culprits.
Correlation vs causation: confounders, reverse causality, and selection bias all produce correlation without causation; establishing causality needs an RCT or a strong identification strategy like instrumental variables or difference-in-differences.
Sampling and selection bias: a non-random sample doesn't represent the population; for example, historical support tickets overrepresent edge cases relative to the full customer base.

Tier 2 (Fin-specific depth)

Power analysis and sample size: for a given effect size at 80% power and alpha 0.05, you need roughly (16 / effect size)^2 samples; a lot of real A/B tests are quietly underpowered.
KL divergence and mode collapse: KL divergence is asymmetric; forward KL is mode-covering (used in VAE posteriors), reverse KL is mode-seeking (used in RL); DPO uses a KL penalty to keep the policy from drifting too far from the reference model (deeper dive: your reinforcement-learning-in-llms post).
Expected calibration error: a model that says "80% confident" should be right 80% of the time; measure by binning predictions against actual accuracy; temperature scaling recalibrates without retraining.
PCA and dimensionality reduction: PCA takes the top-k eigenvectors of the covariance matrix to capture the most variance; it's linear only, so it fails on nonlinear manifolds.
Mutual information: captures nonlinear dependence that correlation misses; used in feature selection and information-bottleneck style analysis.

Pillar C: LLM / Modern AI Fundamentals (~45% of questions, for Fin)

Tier 1 (must-know)

Prompting vs fine-tuning: prompting (zero-shot, few-shot, chain-of-thought) is cheap and generic with no training cost; fine-tuning (SFT, preference tuning) costs more but is domain-specific and performs better on a repeated use case (deeper dive: your ml-system-design-enterprise-support-ai post).
RAG basics: retrieve relevant context from a knowledge base and ground generation in it; reduces hallucination and staleness risk in principle; helps on knowledge-heavy, separable-fact tasks, hurts when retrieval is noisy, the KB is stale, or the task needs deep multi-document reasoning (deeper dive: your ml-system-design-enterprise-support-ai post).
Why LLMs hallucinate and how to mitigate it: models assign high probability to plausible but false continuations because of weak MLE priors, reward hacking, or out-of-distribution inputs; mitigate with RAG grounding, confidence filtering, LLM-as-judge, and process supervision that rewards correct steps, not just correct final answers.
LLM-as-judge and automatic metrics: a strong LLM scoring another model's output has known biases toward length, position, and its own model family; ROUGE and BLEU are word-overlap metrics that are easy to game, BERTScore is more semantic, and human eval remains the gold standard despite being expensive.
Attention mechanism: softmax(QK^T / sqrt(d_k))V, where the sqrt(d_k) scaling stabilizes gradient variance and multiple heads let different heads specialize in different relationships; cost is quadratic in sequence length, which bites hard past 100k tokens (deeper dive: your attention-everything-you-need post).
RLHF vs DPO vs GRPO: RLHF is a three-stage pipeline (SFT, reward model, PPO) that's flexible but complex and prone to reward-model overfitting; DPO collapses this into one supervised loss, simpler and more stable; GRPO uses group-relative RL with verifiable rewards, suited to math and code reasoning (deeper dive: your ml-system-design-enterprise-support-ai and reinforcement-learning-in-llms posts).

Tier 2 (Fin-specific, heavily weighted)

Query rewriting: a small fine-tuned model (1-3B params, 10-50k examples) turns a raw colloquial user query into a structured retrieval query to improve recall; watch for over-rewriting hurting queries that were already short and precise.
LoRA: adds low-rank adapters (rank 16-64) and freezes the base weights, roughly 100x fewer trainable parameters than full fine-tuning, enabling fast iteration and multi-adapter composition.
SFT data quality for support tickets: survival filtering removes short, unresolved, or spam examples (often 40-60% of raw data); quality scoring that keeps only the top 30% beats training on everything; CSAT-weighted training helps continual improvement.
Synthetic data augmentation: a teacher model paraphrases, backtranslates, or self-instructs to expand training data, but risks inheriting teacher bias and topic drift, so weight synthetic examples lower than real ones.
Chunking strategies: fixed-size chunks (512 tokens, 128 overlap) are fast but naive; semantic chunking on topic shifts is more coherent but slower; hierarchical chunking (paragraph plus section) works best on heterogeneous sources; watch for garbled table extraction and preserve breadcrumbs in nested documents.
Hybrid retrieval: BM25 nails exact matches like error codes, dense embeddings handle paraphrase, and a cross-encoder reranker on top improves recall at 10 by 10-15% over dense retrieval alone.
Embedding models: sentence-transformer, E5, and BGE are standard; fine-tune on task-specific pairs for best results; in-batch negatives are critical for contrastive training.
Retrieval evaluation: Recall at k asks whether the gold document made the top k, MRR asks how highly it ranked, NDCG weights graded relevance; high recall at 100 can still mean low precision at 5.
Knowledge base rot and retrieval concentration: policies change and docs go stale; mitigate with temporal decay scoring, contradiction detection, and a refresh queue; if 5% of chunks dominate every query, that signals an embedding or chunking bias.
Multi-hop reasoning as a RAG weakness: RAG assumes a single-document answer, so cross-document reasoning is a known weak spot; long-context models can sometimes outperform RAG here.
Constitutional AI: explicit principles (harmless, honest, helpful) drive a self-critique-and-revise loop that feeds an SFT stage, followed by RLAIF with an AI judge; scales past the human-feedback bottleneck and is more transparent than pure human preference data (deeper dive: your reinforcement-learning-in-llms post).
Cold-start reasoning training: pure RL from scratch can cause format collapse (language mixing, unreadable outputs); cold-starting with a small set of high-quality chain-of-thought examples before RL stabilizes format first; rule-based rewards for format, language, and safety can augment outcome rewards (deeper dive: your reinforcement-learning-in-llms post).
Iterative DPO: addresses offline DPO's distribution-shift weakness by generating new responses from the current policy, scoring them with a reward model, then running DPO again on the fresh data; closes much of the gap with PPO while staying more stable.

Tier 3 (nice-to-have depth)

KV caching and grouped-query attention: KV caching avoids recomputing key/value pairs during generation; GQA shares KV heads across groups, cutting cache memory roughly 4x for under 0.1% quality loss, which matters for low-latency streaming agents (deeper dive: your attention-everything-you-need post).
Mixed precision and quantization: BF16 gives roughly 2x memory savings and 2-3x speedup; FP8 on H100-class hardware doubles that again; INT8/INT4 quantization shrinks inference models 2-4x for under 1% accuracy loss if calibrated well.
Inference optimizations: PagedAttention (used by vLLM) cuts memory fragmentation from roughly 60% waste to 20%; speculative decoding with a small draft model gives a 2-3x speedup; prompt caching can cut cost up to 90% for stable prompt prefixes.
Agentic and tool-use evaluation: tracks whether the model picked the right tools in the right order, recovered from a tool failure, and how often it escalated to a human.
Reasoning model evaluation: process supervision (grading intermediate steps) beats pure outcome supervision (grading only the final answer) for math and code tasks.

Section 2: Worked Examples

Eight worked answers, weighted toward the RAG, fine-tuning, and alignment topics Fin is most likely to ask about, each written to the 3-minute template: a crisp definition (15 seconds), why it matters with the core tradeoff (30 seconds), a mental model (60 seconds), a concrete example (30 seconds), then a pause, ready for follow-ups (15 seconds).

1. Can you explain DPO and why you'd use it over RLHF?

Definition. DPO is a one-stage supervised learning approach to preference alignment. Instead of training a separate reward model, it derives the optimal policy in closed form and reduces alignment to a binary classification loss.

Why it matters. Traditional RLHF requires three stages (SFT, reward model training, PPO), which is complex and requires tuning the reward model separately, and that reward model can overfit or drift. DPO solves this with a single unified objective, which is why it's become the default first alignment pass for teams without a large RL infrastructure investment.

Mental model. The optimal RLHF policy is pi* proportional to pi_ref * exp(reward / beta). Rearranging to express reward in terms of the policies, then substituting into the Bradley-Terry preference model, gives the DPO loss, which is just binary cross-entropy on the ratio of policy to reference-policy log-probabilities for the winning versus losing response. No reward model needed.

Example. Fine-tuning a 7B model on 5k preference pairs took about 4 hours on a single A100, versus PPO needing roughly 2 weeks with a critic network, for comparable quality.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai and reinforcement-learning-in-llms posts.)

2. How would you design a retrieval system for a support knowledge base?

Definition. Retrieval for support means turning a heterogeneous knowledge base, tickets, Confluence pages, Slack threads, into a system that reliably surfaces the right passage for a user's question, fast enough for a live chat.

Why it matters. Get retrieval wrong and everything downstream, grounding, hallucination rate, resolution rate, degrades no matter how good the generation model is. The core tradeoff is exact match versus semantic match: users ask about specific error codes and policy names, but also paraphrase the same question a dozen different ways.

Mental model. Hybrid retrieval is the answer: BM25 catches exact-match tokens like error codes and product names that dense embeddings often miss, dense embeddings catch paraphrase and semantic similarity, and a cross-encoder reranker sits on top of both candidate sets to pick the final passages. This combination typically improves recall at 10 by 10-15% over dense retrieval alone. Chunking strategy matters just as much as the retrieval method: fixed-size chunks are fast to build but naive, semantic chunking respects topic boundaries, and hierarchical chunking (paragraph plus section) works best across a heterogeneous source mix. Tables inside support docs are a specific failure point, they tend to get garbled during text extraction, so table-aware parsing is worth the extra engineering.

Example. On a support KB mixing tickets and internal docs, moving from dense-only retrieval to BM25-plus-dense-plus-reranker is a realistic path to a double-digit recall at 10 improvement, with the reranker doing a lot of the heavy lifting on precision.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai post.)

3. How do you stop a support agent from hallucinating policy details?

Definition. Hallucination here means the model states a policy, refund window, or procedure with confidence when it isn't actually grounded in a real, current source document.

Why it matters. For an AI that answers real customers, a confidently wrong policy statement is worse than a slow answer or an escalation. The tradeoff is coverage versus safety: the more the model is allowed to answer from parametric memory instead of retrieved evidence, the more fluent it sounds and the more it risks being wrong.

Mental model. The fix is layered, not a single trick. RAG grounding forces the model to cite retrieved passages rather than recall from weights. A faithfulness check, either a smaller model or a rule-based verifier, compares the draft answer against the retrieved source and flags unsupported claims before the answer ships. Confidence-based abstention lets the model say "I don't have a confident answer" and hand off, rather than guess. Underneath all of this, knowledge base rot is a root cause of hallucination that's easy to miss, if the policy the model is grounded in is six months stale, grounding doesn't help, so temporal decay scoring and contradiction detection on the KB itself matter as much as anything at generation time.

Example. A faithfulness classifier that flags claims not supported by the retrieved passage, run before the answer is shown to the customer, catches a meaningful share of hallucinated policy details that pure RAG grounding alone lets through, because retrieval being correct doesn't guarantee the generated text stayed faithful to it.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai post.)

4. How would you fine-tune on years of support tickets without inheriting bad agent habits?

Definition. Historical support tickets are a huge, free source of SFT data, but they also contain years of shortcuts, wrong answers, and rude or unresolved conversations that a model would happily learn to imitate if trained on everything.

Why it matters. The naive approach, dump every ticket into SFT, actively hurts quality because the model learns the average behavior of past agents, not the best behavior. The tradeoff is data volume versus data quality: more data is usually better, except when a meaningful fraction of it is bad behavior you don't want reproduced.

Mental model. Filter before you fine-tune. Survival filtering removes tickets that were short, unresolved, or spam, which alone often removes 40-60% of raw volume. Quality scoring, using a reward model or heuristics like resolution time and follow-up rate, ranks the remainder and keeps only the top slice, training on the top 30% by quality typically beats training on everything. CSAT-weighted training goes a step further and up-weights examples tied to conversations the customer actually rated well, so the signal keeps improving as new, better-rated conversations come in. Where real data is thin for a specific scenario, synthetic data from a teacher model can fill the gap, but it should be down-weighted relative to real examples since it risks inheriting teacher bias and topic drift.

Example. A quality-scored subset of historical tickets, roughly 30% of the raw corpus, trained a model that outperformed one trained on the full unfiltered corpus, because the unfiltered version had learned to imitate agents' bad shortcuts along with their good ones.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai post.)

5. How do you decide when a support agent should escalate to a human?

Definition. Escalation routing is the decision layer that sits above generation: given a user's message and the model's draft response, decide whether to send the answer, ask a clarifying question, or hand off to a human agent.

Why it matters. Every wrong escalation costs something: escalate too often and you lose the automation value the whole system exists for, escalate too rarely and low-confidence answers reach customers unchecked. The tradeoff is precision of automation versus safety net coverage, essentially the same precision-recall tradeoff from classical ML, just applied to a routing decision instead of a classifier.

Mental model. Think of it as multi-model routing with a confidence threshold. A cheap, fast model or a set of features (retrieval confidence, generation model's own log-probability, whether the query matches a known unresolved category) scores each conversation, and anything below the threshold routes to a human or to a bigger, slower model. The threshold isn't fixed, it should reflect the actual cost asymmetry: a wrong answer on a billing question costs more than a wrong answer on a documentation lookup, so thresholds can vary by topic. Latency budget matters here too, escalation decisions need to happen fast enough not to stall the conversation.

Example. A confidence threshold tuned per ticket category, rather than one global threshold, routes billing and account-security questions to humans more conservatively than general documentation questions, which better matches the real cost of getting each wrong.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai post.)

6. How would you design this pipeline under a strict cost-per-query budget?

Definition. Cost optimization for a support AI pipeline means hitting a target cost-per-resolved-query without giving up so much quality that resolution rate or CSAT drops.

Why it matters. At Fin's scale, a few cents per query compounds fast, and the naive answer, always call the biggest, best model at every stage, is usually not affordable. The tradeoff is uniform quality versus targeted spend: not every step in the pipeline needs the frontier model.

Mental model. Spend compute where it actually buys quality, and cut it everywhere else. Query rewriting only needs a small 1-3B model, not the main generation model. Retrieval reranking should run on a shortlist, rerank only the top handful of candidates the first-stage retriever returns, not the whole corpus. Quantization (INT8, sometimes INT4) shrinks the inference footprint 2-4x for under 1% accuracy loss when calibrated well, which is close to free money on cost. Batching and prompt caching, caching the stable system-prompt prefix rather than the whole conversation, can cut cost up to 90% on the shared portion of every request. The overall pattern is a cascade: cheap components filter and narrow the problem before an expensive model ever sees it.

Example. Moving query rewriting from the main generation model to a dedicated small fine-tuned model, and reranking only the top-20 retrieved candidates instead of the full result set, are both changes that cut cost meaningfully without touching the final generation model at all.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai and attention-everything-you-need posts.)

7. How do you evaluate the quality of an end-to-end support AI system?

Definition. End-to-end evaluation here means measuring both retrieval and generation, and tying both back to a business outcome, not just a model-level score.

Why it matters. No single metric captures whether a support agent is good. A model can score well on a generation metric while still failing customers because retrieval fed it the wrong document, or it can retrieve the right document and still fail because the generation step misrepresented it.

Mental model. Layer the evaluation by pipeline stage. Retrieval gets its own metrics: Recall at k for whether the gold document made the shortlist, MRR for how highly it ranked, NDCG for graded relevance, since a system can look fine on recall at 100 while precision at 5 is poor. Generation gets LLM-as-judge against a rubric (correctness, tone, completeness) plus periodic human review, being explicit about LLM-as-judge's known biases toward longer, more-confident, and same-family outputs, and calibrating it against human labels rather than trusting it blind. Underneath both, the metric that actually matters for the business is production behavior: resolution rate, escalation rate, CSAT, and how often a human has to correct the model after the fact.

Example. A system can post a strong LLM-as-judge score while its resolution rate stays flat, that gap is usually the tell that the judge is rewarding verbose, confident-sounding answers rather than answers that actually solved the customer's problem.

Ready for follow-up. (Deeper dive: your ml-system-design-enterprise-support-ai post.)

8. Can you explain the bias-variance tradeoff and how it shows up in a support model?

Definition. Bias is error from a model too simple to capture the real pattern (underfitting); variance is error from a model too sensitive to noise in its training data (overfitting). Total generalization error breaks down roughly into bias squared plus variance plus irreducible error.

Why it matters. It's the single tradeoff underneath almost every modeling decision: model size, regularization strength, how much data to collect, when to stop training. Getting it wrong in either direction has a concrete cost: too much bias means missing real cases, too much variance means memorizing noise that doesn't generalize.

Mental model. Picture two support models. A small model trained on too little ticket data underfits: it misses edge cases and gives generic answers because it never saw enough variety to learn the nuance. A large model fine-tuned too long on a narrow slice of tickets overfits: it starts reproducing very specific, sometimes bad, agent habits from its training set instead of generalizing to new questions. The fix set is the same one used everywhere else: regularization (LoRA rank control, weight decay), more or better data (quality-filtered tickets, not just more volume), ensembling, and validation on held-out, representative traffic to pick the right stopping point. Worth a sentence on the modern wrinkle: double descent shows that pushing capacity well past the point of fitting training data perfectly can sometimes lower test error again, so the classic U-shaped curve isn't universal in overparameterized regimes.

Example. A support model fine-tuned for too many epochs on a narrow set of billing tickets started reproducing specific phrasing quirks from a handful of training conversations, a variance failure, while a model with too little data defaulted to generic non-answers on anything outside common cases, a bias failure.

Ready for follow-up. (Deeper dive: your interview-preparation and fundamental-statistics posts.)

Section 3: Follow-Up Patterns

Every worked answer above tends to draw one of these follow-ups. Prepare the short version of each so a follow-up doesn't turn into rambling.

1. DPO

"When would you use PPO over DPO?" Complex domains needing fine-grained reward shaping, safety-critical systems, or very large models where the added training stability of a separate reward model and critic is worth the complexity.
"What's the KL penalty doing?" It keeps the policy from drifting too far from the reference/SFT model, which preserves instruction-following ability that pure reward maximization would otherwise erode.
"How does this compare to Constitutional AI?" Constitutional AI generates its preference data from AI feedback instead of human labels; DPO is agnostic to where the preference pairs came from, so the two combine naturally, use AI feedback to build the preference dataset, then train DPO on it.

2. Retrieval system design

"What if retrieval quality is poor and the model still hallucinates?" That's a signal to go back a step, poor retrieval isn't a generation problem, check chunking, embedding model fit, and whether the reranker is actually improving precision at the top ranks before touching the generation model at all.
"How do you evaluate retrieval separately from generation?" Recall at k and MRR against a labeled query-to-gold-document set, run offline, before any end-to-end evaluation; this isolates whether failures come from retrieval or generation.

3. Hallucination mitigation

"How do you measure hallucination rate without labeling every response by hand?" A faithfulness classifier or LLM-as-judge scored against retrieved sources, sampled and calibrated against a smaller human-labeled set, gives a scalable proxy.
"Isn't a stale knowledge base a different problem from hallucination?" They compound each other. A model can be perfectly faithful to a stale document and still give a wrong, confident answer, so KB freshness (temporal decay, contradiction detection) is part of the hallucination story, not a separate one.

4. SFT from tickets

"Would you use LoRA or full fine-tuning here?" LoRA for most iteration cycles, it's cheap, fast, and composable across use cases; reserve full fine-tuning for cases where you need to shift the base model's behavior more broadly than a low-rank adapter can reach.
"How do you avoid catastrophic forgetting when fine-tuning on a narrow ticket slice?" Mix in general instruction data alongside the ticket data, and keep LoRA rank modest so the adapter can't overwrite too much of the base model's broader capability.

5. Routing and escalation

"What signals feed the confidence threshold?" Retrieval confidence score, the generation model's own log-probability or self-reported uncertainty, and category-level historical error rate, combined rather than any single signal alone.
"Doesn't routing add latency?" Yes, so the routing model itself has to be small and fast, the whole point is spending a little latency upfront to avoid a much worse outcome downstream.

6. Cost optimization

"Where's the biggest cost driver in a RAG pipeline?" Usually the final generation call, so the highest-leverage cuts are cheap upstream stages (query rewriting, reranking a shortlist) that reduce how much has to go through the expensive model, plus prompt caching on the stable prefix.
"How do you validate quantization didn't hurt quality?" Run the full offline evaluation suite, retrieval and generation metrics, before and after quantization on a representative traffic sample, not just a handful of spot checks.

7. Evaluation

"How do you handle LLM-as-judge's known biases?" Explicit rubrics instead of open-ended scoring, ensembling more than one judge model, and periodic calibration against a human-labeled sample to catch drift.
"What's your primary production metric versus your offline metric?" Offline metrics (recall, judge scores) gate releases; production metrics (resolution rate, escalation rate, CSAT, human correction rate) are the actual source of truth and should be monitored continuously, not just at launch.

8. Bias-variance

"How does this show up differently in LLM fine-tuning versus classical models?" Same tradeoff, different levers, instead of model complexity you're tuning LoRA rank, data quality and volume, and number of training epochs, and "variance" often looks like memorizing specific training-set phrasing rather than classic numerical overfitting.
"What's double descent, and does it break this framework?" No, it extends it, in heavily overparameterized models, error can fall, rise, then fall again past the interpolation threshold, which is why very large models can generalize well despite fitting their training data almost perfectly.

Section 4: Rehearsal Plan

The 3-minute structure. Crisp definition (15 seconds), why it matters with a concrete scenario and the core tradeoff (30 seconds), your mental model or intuitive explanation (60 seconds), a real example or benchmark number (30 seconds), then pause, ready for follow-ups (15 seconds). This is the template every worked example above follows.

Daily drill methodology.

Explain each core concept out loud daily, not silent review. Saying it out loud surfaces the gaps that reading past.
Time yourself. Aim for a crisp 3 minutes on the main answer, plus roughly 1 minute per likely follow-up.
Self-critique after each attempt: did you ramble, did you skip the intuition and jump straight to jargon, did you forget a concrete example.
Drill out of order. Have someone (or a random list) fire concepts at you in random order, not the order in this document, since the real round won't follow this document's order either.

Common failure modes

| Failure mode | Fix | | --- | --- | | Rambling without structure | Use the 5-part template every time, even under pressure | | Too mathematical, no intuition | Always pair the formula with a concrete example | | Forgetting to state assumptions | Say "assuming X" explicitly before diving in | | No answer ready for "when would you NOT use this" | Prepare the inverse case for every concept in advance | | Over-explaining basics the interviewer already knows | Assume the basics are known, go straight to the subtlety |

What "not too deep" actually means.

Don't launch into an unprompted 15-minute deep dive citing the original paper and its history.
Do give a crisp explanation with one genuinely intuitive detail (for example, what multi-head attention buys you over single-head) and a concrete tie back to the product.
Don't derive the bias-variance formula from scratch unprompted.
Do state it in one sentence, name the tradeoff, and tie it to a concrete scenario, like a small support model underfitting on edge cases versus a large one memorizing bad agent habits.

Section 5: Fin-Specific Weighting

Given Fin's product (a text-based AI support agent) and this being a Staff/Senior role, prep time is best spent where the questions are most likely to land.

Most likely, roughly 60% of questions

Designing a retrieval system for support (BM25 vs dense vs hybrid, and why, given a heterogeneous KB of tickets, Confluence, and Slack, with tables as a high-risk parsing case).
Preventing a support agent from hallucinating policy details (RAG grounding, faithfulness checks, process supervision).
Training on years of support tickets without inheriting bad agent habits (survival filtering, quality scoring, CSAT weighting).
Deciding when to escalate versus let the agent handle it (multi-model routing, confidence thresholds, latency budget).
Designing the pipeline under a cost-per-query constraint (small-model query rewriting, shortlist-only reranking, quantization, batching, prompt caching).

Possible, roughly 30% of questions

DPO or RLHF for tone or brand-voice alignment.
Cold-start reasoning training for complex multi-ticket resolution.
Detecting and fixing knowledge-base rot.
A/B testing a model change in production.
An end-to-end evaluation framework spanning retrieval through generation.

Unlikely, roughly 10% of questions

Speech recognition specifics, autonomous driving, or computer vision. Fin's product is text-based, so these are low-probability unless the interviewer is probing general breadth rather than product fit. If they come up, a short, honest answer is fine; this isn't where the prep budget should go (background material lives in your autonomous-driving-foundations and speech-architecture posts, but a deep review of either is low priority for this round).

Read the checklist once the night before, run the daily drill on the Tier 1 and Tier 2 concepts, and go into the round ready to hit the 5-part structure on whatever comes up first.

Basic ML Speed Round Prep