11 Perplexity AI Engineer Interview Questions & Answers

At a high level, the goal is to stop work that depends on a failed task. The main challenge is finding every downstream task, including indirect dependents. I would explain this in three steps: identify the failed task, visit each direct dependent, then continue recursively through its descendants. In the diagram, C fails, so E, F, and H also become FAILED. The trade-off is that we must traverse the dependency graph, but unaffected branches stay unchanged.

The system must make sure a task cannot continue when something it depends on has failed. The difficult part is that the affected task may be several steps away from the original failure. For example, C fails first. E and F depend directly on C, so they fail too. H depends on E and F, so H must also fail. The other branch, B to D to G, does not depend on C and stays unchanged. The solution follows dependency arrows downstream until there are no more descendants to visit.

1. Can the dependency graph contain shared descendants, like H depending on both E and F?
2. Should an already FAILED task be skipped if another path reaches it again?
3. Can we assume the dependency graph has no cycles?

I would first make the arrow meaning clear. An arrow points from one task to another task that depends on it. In the diagram, C points to E and F. That means E and F depend on C. If C fails, both must become FAILED. This direction matters because failure moves only toward descendants, not toward tasks above C.

When C fails, I mark C as FAILED. This is the starting failure shown in the diagram. I do not change A, B, D, or G because they are not descendants of C. This keeps the update limited to tasks that actually depend on the failed work.

Next, I look at the tasks that depend directly on C. Those tasks are E and F. I mark both as FAILED. Then I apply the same rule to each of them. This repeated step is recursion. In simple words, the same failure rule is applied again to each dependent task.

E and F both lead to H. H does not depend directly on C, but it is still a descendant of C. Therefore H must also become FAILED. A depth-first or breadth-first walk can find every reachable descendant. Either approach works here. The important rule is to keep following dependency arrows until no new descendant remains.

The B to D to G branch stays unchanged. None of those tasks depends on C. This matters because a failure should not spread into unrelated work. H can be reached through both E and F, so it should be marked FAILED only once. A visited check, or an equivalent already-FAILED check, prevents repeated work. The benefit is correct failure handling. The downside is that the system must walk the affected part of the graph whenever a task fails.

The benefit is that every task depending on failed work is stopped correctly. Unrelated tasks keep their current state, so one failure does not cancel the whole graph. The downside is that the system must visit the affected descendants. If many tasks depend on one failed task, this walk can touch many tasks and dependency edges. A visited check also matters when two paths reach the same task, such as H through E and F. This prevents repeated work. Depth-first and breadth-first search both work because the goal is simply to reach every downstream descendant.

I would use Logistic Regression as a simple binary classifier. I would split the provided X and y into training and test data, fit the model on the training part, predict the test part, and report the runtime accuracy, confusion matrix, and classification report. The important point is that I would report the values produced from the provided data instead of assuming a score in advance.

The task is to teach a program to choose between two possible labels using examples that already have the correct answer. I would keep most examples for learning and save some examples for checking the result. The program learns from the first group and then makes choices for the saved group. I would compare those choices with the real answers and report what happened. I would not make up a score because the real result depends on the supplied rows and labels.

1. Are X and y already provided as Python objects?
2. Should I use the requested split settings exactly, including 20 percent for testing and stratification?
3. Do you want only accuracy, or should I also show the confusion matrix and classification report?

I would start with the provided feature matrix X and binary labels y. A binary label means each example belongs to one of two classes, such as 0 or 1.

I would use Logistic Regression. It is a simple supervised classifier that learns a linear decision boundary from labeled training examples. I would split the supplied data so that 80 percent is used for training and 20 percent is kept for testing. I would use random_state equal to 42 so the split can be repeated. I would also use stratify equal to y so the split tries to keep the class proportions similar in both parts.

Next, I would call fit on X_train and y_train. This is the training step. The model learns parameters from those labeled examples. I would then call predict on X_test. The predict method returns class 0 or class 1. If I needed class probabilities, I could use predict_proba instead.

For the initial evaluation, I would calculate accuracy, a confusion matrix, and a classification report. Accuracy shows the fraction of test predictions that are correct. The confusion matrix shows how predictions are distributed across the two true classes. The classification report gives measures such as precision and recall for each class.

The exact result must come from running the code on the provided dataset. The supplied prompt does not include the actual toy rows, so a numeric result is not supported here. I would report the runtime values produced by print(acc), print(cm), and print(report). One limitation is that a small test set can give an unstable estimate. Stratification can also fail when a class has too few examples. In production, I would use a stronger validation process before treating this initial result as reliable evidence.

1. Use the supplied X feature matrix and y binary labels.
2. Split the data into training and test parts with test_size equal to 0.2, random_state equal to 42, and stratify equal to y.
3. Create a LogisticRegression classifier.
4. Fit the classifier with X_train and y_train.
5. Predict labels for X_test with predict.
6. Compute accuracy_score, confusion_matrix, and classification_report from y_test and the predictions.
7. Print and report the runtime results. This task uses standard supervised classifier training. Prompting, retrieval, preference tuning, and language model fine tuning are not needed for this toy classifier.

A common mistake is training and testing on the same rows. That gives an overly optimistic result because the model is checked on data it already saw. Another mistake is inventing an accuracy value before running the provided dataset. It is also incorrect to say that predict directly returns probabilities. Predict returns class labels, while predict_proba can return class probabilities. A candidate should also remember that stratified splitting needs enough examples from each class. Finally, using only accuracy can hide class specific errors, so the confusion matrix and classification report give useful extra evidence.

Explain the flow in order: provided data, split, fit, predict, then evaluate. State clearly that the final numbers must come from running the supplied dataset. Also distinguish predict, which returns class labels, from predict_proba, which can return probabilities.

I would keep the same task, data split, and ROC AUC metric, then make one defensible change and measure whether it helps. The baseline is logistic regression with ROC AUC of 0.72. I can try Gradient Boosted Trees with XGBoost, or keep logistic regression and tune its regularization value C using validation data. I would choose the best validated option, then evaluate it on the same test set. In the diagram, the improved ROC AUC is 0.86, so the absolute improvement is 0.14.

The goal is to improve the classifier without making the comparison unfair. I keep the same binary classification task, the same train, validation, and test split, and the same ROC AUC metric. The baseline is logistic regression with ROC AUC of 0.72. I then change only the model choice or its tuning. I can try Gradient Boosted Trees with XGBoost, or tune the logistic regression regularization value C. I use validation data to make that choice. Finally, I evaluate the selected model on the same test set and compare the result with the baseline.

1. Should ROC AUC remain the main metric for the comparison?
2. Is the test set fixed and untouched until the final evaluation?
3. May I change the model family, or should I tune only the original logistic regression model?

I would start with the baseline shown in the diagram. The model is logistic regression with L2 regularization and C equal to 1.0. Its ROC AUC is 0.72 on the test set.

Next, I would make one controlled change. One option is Gradient Boosted Trees with XGBoost. The diagram shows this as a stronger model choice using its default settings. Another option is to keep logistic regression and tune C, which controls the strength of regularization. The candidate values shown are 0.01, 0.1, 1, 10, and 100.

I would use the validation set to compare choices. If I need a more stable tuning estimate, I can use cross validation on the training data. I would not repeatedly use the test score while selecting the model or C value. Doing that would let test information influence the choice and make the final result less trustworthy.

After selection, I would evaluate the chosen model on the same test set with the same ROC AUC metric. The diagram shows the improved model reaching 0.86. Compared with the baseline of 0.72, the absolute improvement is 0.14.

If the metric does not improve, I would change the model choice or tuning and repeat the validation step. I would also watch for overfitting. If a simpler model gives nearly the same result, I would usually prefer it because it is easier to operate and explain.

1. Keep the same binary classification task, data split, and ROC AUC metric.
2. Measure the logistic regression baseline. The diagram shows ROC AUC of 0.72.
3. Make one controlled adaptation choice. Try Gradient Boosted Trees with XGBoost, or keep logistic regression and tune C.
4. If tuning C, compare values such as 0.01, 0.1, 1, 10, and 100 using validation data. Cross validation can be run on the training data for a more stable choice.
5. Select the best validated option while watching for overfitting.
6. Evaluate that selected model on the same untouched test set.
7. Compare the final ROC AUC with the baseline. The diagram shows 0.86 versus 0.72, which is an absolute improvement of 0.14.
8. If the metric does not improve, revise the model choice or tuning and repeat the validation process without tuning against the test set.

A common mistake is choosing hyperparameters by repeatedly checking the test score. That allows test information to affect model selection and makes the final comparison less trustworthy. Another mistake is comparing models on different data splits or with different metrics. A candidate may also assume that a more complex model must be better. The stronger model should justify its added complexity with better validated performance. Another mistake is ignoring overfitting when the validation result improves but the final test result does not.

State the fairness rule first: same task, same data split, and same ROC AUC metric. Then explain exactly one model or tuning change, how validation selects it, and why the test set stays outside that selection loop. Finish with the diagram result: ROC AUC improves from 0.72 to 0.86, which is an absolute gain of 0.14.

At a high level, I would build a streaming system that finds queries whose interest is rising compared with a recent baseline. Query events first pass through an ingestion gateway, then a partitioned event stream. Stream processors normalize queries, count them in sliding windows, and create trend features. A scoring service ranks the strongest trends and stores Top-K results for fast serving. The serving API can personalize results using user signals. Abuse controls reduce gaming. The main trade-off is freshness versus processing, storage, and ranking cost.

This system finds search queries that are becoming popular right now. For example, a query may suddenly get much more interest than it had a few minutes ago. We need to notice that change quickly, rank the strongest trends, and return them with low delay. The hard part is separating real interest from normal traffic, spam, and fake bursts. The design also needs regional and personalized results. I would follow the diagram from query collection, through counting and scoring, and finally to serving, feedback, and monitoring.

• How fresh should the trending list feel to users?
• Do we need global trends, regional trends, or both?
• How much personalization should change the public trending list?
• Which kinds of abusive traffic should we block or reduce?

I would start with the real-time query events. Search Apps, Web Search, Mobile Apps, and Other Sources send events to the Ingestion API Gateway. The gateway handles authentication, rate limiting, validation, and PII filtering. PII means personal information that should not enter the trend pipeline unnecessarily. These checks matter because invalid, abusive, or sensitive events should be handled before aggregation. Valid events then move into the Event Stream. The diagram uses Kafka or Pulsar. The stream is shown as partitioned by time and region so processing can be divided across groups of events.

Stream Processors, shown as Flink or Spark Streaming, consume the Event Stream. They first normalize query text. The diagram shows lowercasing, stemming, spelling fixes, and deduplication. The goal is to avoid treating small text differences as unrelated trends. The processors then keep sliding windows such as 1 minute, 5 minutes, and 1 hour. A sliding window is a recent time range that keeps moving forward. For each query, the system creates features such as counts, unique activity, growth rate, entropy, geographic spread, and recency. These features become inputs to trend scoring.

The Trend Scoring Service turns those features into a trend score. The Scoring Model uses weighted signals shown in the diagram. These include growth rate versus a baseline, absolute volume, acceleration, uniqueness and diversity, recency, geographic spread, and category or vertical boosts. Growth helps detect rising interest. Absolute volume helps avoid overvaluing tiny changes. Recency favors fresh activity. The Ranker then orders candidates per region and globally. This produces the Top-K queries, meaning the highest-ranked queries are kept for fast serving.

The Storage Layer has three jobs. The Hot Store keeps Top-K results in Redis for each region and globally. Its time-to-live is measured in minutes, so old results can expire. The Time Series DB, shown as Druid or ClickHouse, stores counts and features over time. This supports comparisons between current and earlier behavior. Cold Storage, shown as S3 or GCS, keeps raw events and aggregates for longer-term analysis. These stores serve different needs. Redis supports fast reads, while the other stores support historical analysis and tuning.

The Serving API gets prepared trend data from the Storage Layer and returns results to Web Clients, Mobile Apps, and Partner APIs. The diagram shows inputs such as geography, time window, category, and result limit. The API can also return a personalized view. The User Profile Service supplies interests, history, and location. The Personalization Service re-ranks the existing trend candidates using user signals and context. This keeps the main trend-detection logic separate from user-specific ordering.

Abuse Mitigation works across the design. It includes per-IP or per-user rate limiting, bot detection, query spam and repetition filters, blacklist rules, and anomaly detection. Its goal is to prevent people or automated systems from gaming the trending list. The Feedback Loop collects click or no-click signals, dwell time, abuse reports, and relevance feedback. Offline Analysis and Model Tuning use historical outcomes to improve scoring weights, evaluate against outcomes, and test changes. Monitoring and Alerts watch pipeline health, latency, lag, throughput, Top-K freshness, and error rates. Governance and Privacy add PII removal, data retention rules, access control, and audit logs. These supporting flows improve quality and safety without becoming the client response path.

The main trade-off is freshness versus cost and stability. Short sliding windows notice new trends quickly, but they can react strongly to noise and sudden bursts. Longer windows are steadier, but they respond more slowly. Keeping Top-K results in Redis makes serving fast, but complete history must live elsewhere. Regional ranking gives more useful local trends, but it increases processing and storage work. Personalization can improve relevance, but it needs user signals and careful privacy controls. Abuse filters reduce manipulation, but aggressive filters can also suppress real breaking events. We accept these costs because a useful trending system must stay fresh, fast, scalable, hard to game, and practical to operate.

At a high level, I would build a search system that learns from user behavior and adapts in near real time. Interaction events flow through Kafka or PubSub, then Flink or Spark Streaming updates short-term and long-term interest profiles. A query is understood, then BM25, vector search, personalization, and content filters retrieve candidates. Learning-to-rank uses relevance, recency, personal fit, diversity, novelty, and quality signals. I would add privacy controls, cold-start handling, caching, scaling, and evaluation. The main trade-off is faster adaptation versus latency, cost, and operational complexity.

The problem is to make search follow what a person wants now, while still remembering useful older interests. A user may usually read programming content but spend today researching headphones. Search should react quickly without forgetting the longer history. It should also stay fast, work for new users, protect personal data, and avoid repetitive results. I would explain the design from user events, to profile updates, to retrieval and ranking, then close the loop with feedback and evaluation.

• How quickly should new behavior change search results?
• What traffic and latency targets matter most?
• Which user events may we store, and for how long?
• How much exploration is acceptable compared with immediate relevance?

I would start with user behavior. The system records queries, clicks, views, dwell time, skips, saves, purchases, feedback, and useful context. Event Ingestion uses Kafka or PubSub as a durable, ordered event log. Stream Processing uses Flink or Spark Streaming to sessionize, enrich, deduplicate, and window events. Processed events also write asynchronously into Persistent Stores for later use.

The Online Interest Profile has short-term and long-term parts. Short-term session interest keeps recent queries, clicks, intent, entities, and recency. Long-term interest keeps topic vectors, an entity graph, preferences, and quality signals. Stream Processing updates this profile in near real time.

Persistent Stores support this flow. The Event Store keeps raw immutable events. The User Profile Store keeps the latest profiles. The Feature Store keeps precomputed user, item, and context features. Content Indexes hold BM25, ANN vector, and metadata indexes.

The User Query goes first to Query Understanding. It can rewrite the query, correct spelling, detect intent, and identify entities. It also reads the Online Interest Profile.

Candidate Retrieval then combines lexical BM25, semantic vector search, personalization from the User Profile, and content filters. For a new user, Cold Start Handling begins with popularity and content quality. It can also use allowed side information and bandit-style exploration to learn faster.

Ranking and Re-ranking use a learning-to-rank model. Features include relevance, recency with time decay, personal fit, diversity, novelty, and quality or trust. Recency helps recent behavior matter more when interests shift. Diversity avoids many nearly identical results.

Offline Model Training learns ranking and retrieval models from time-decayed behavior signals and negative sampling. The Model Registry keeps versioned models, A/B configurations, and feature definitions. Results return the top K items with scores and explanations.

The Engagement and Feedback Loop collects result interactions and turns them into implicit feedback. It updates profiles in near real time, so the next search can reflect the newest behavior.

Cross-cutting services keep the system practical. Query, result, and vector caches lower latency. Sharded indexes, replication, and auto-scaling support growth. Monitoring tracks latency, QPS, errors, CTR, conversion, drift alerts, and model health. Feature and model versioning supports reproducibility, rollback, and canary deployments. Security covers authentication, authorization, tenant isolation, and PII protection.

Privacy and Controls add consent, transparency, per-user controls, delete or export actions, retention policies, anonymization, encryption, and differential privacy for training. Offline evaluation uses nDCG, MAP, Recall, time-aware holdouts, and interest-drift response time. Online evaluation uses A/B testing or interleaving with CTR, conversion, dwell time, satisfaction, novelty, and diversity.

The main design choice is freshness versus cost. Near-real-time profile updates make search react faster, but they require more streaming work and more serving capacity. Caching lowers latency, but stale cached results or vectors can slow adaptation. Combining BM25 and vector search improves candidate coverage, but it costs more than one retrieval method. Personalization can improve usefulness, but it also raises privacy risk. Consent, deletion controls, encryption, tenant isolation, and PII protection reduce that risk. Exploration helps discover new interests, but too much exploration can hurt immediate relevance. Sharding, replication, and auto-scaling improve scale and availability, but they add operational work. We accept these costs because fast interest adaptation is the core goal.

I would use a versioned lifecycle policy. Age alone would not decide anything. I would combine relevance, legal retention, user deletion, drift, quality regression, and minimum evidence, then choose keep, expire, re-index, fine-tune, or retrain. Every model change would pass offline tests, gradual release, monitoring, and rollback gates.

I would not remove information or change the system just because time has passed. First, I would identify what information and system versions are involved and what changed. Then I would collect evidence. I would ask whether the information is still useful, whether a law requires us to keep it, whether a user asked us to remove it, and whether recent behavior is trustworthy. I would also check whether results are getting worse. The goal is to make the smallest safe change and test it carefully before it reaches everyone.

1. Which assets are in scope: user signals, retrieved knowledge, training data, model versions, prompts, and configurations?
2. What legal retention or user-deletion rules must the lifecycle policy follow?
3. Which quality, safety, product, and infrastructure measures are release gates?
4. Which user, language, region, topic, or other data slices are important enough to monitor separately?
5. How much reliable new evidence is required before we allow fine-tuning or retraining?
6. What rollback conditions and previous known-good version are available?

I would start by defining the scope and version lineage. Every important dataset, retrieved-knowledge index, model, prompt, and configuration should have a version. Lineage means I can trace which exact inputs created a model or release. I would also keep timestamps and evaluation results when they matter. This makes builds reproducible and gives me a known-good rollback point.

Next, I would collect evidence before changing anything. I would look at age, relevance, legal rules, user-deletion requests, data drift, quality regression, and the amount of reliable new evidence. The smallest useful first step is to identify which asset is actually stale or failing instead of assuming the model needs retraining.

For age and relevance decay, I would ask whether the information is still useful for the task and user. Old does not automatically mean wrong. If knowledge is still valid, I keep it. If it is stale or no longer relevant, I expire it. This protects long-lived knowledge from being forgotten too early.

For legal retention, I would apply the retention policy. Some records may have to stay for compliance or audit reasons. I would retain them only when required rather than using one unconditional keep rule.

For user deletion, I would remove eligible user data where the policy and system require it while honoring any required legal retention. I would not claim that one deletion request automatically erases information from every historical model artifact. The exact action depends on where the data exists and what the policy requires.

For retrieved knowledge, I would usually refresh the retrieval layer before changing the model. If documents, metadata, embeddings, or the index are stale, I can re-chunk, re-embed, and re-index the current knowledge. This is usually a smaller and safer change than retraining the model to learn changing facts.

For data drift, I would choose the response by asset. Data drift means the incoming data or user behavior has changed. If retrieved knowledge is stale, I refresh the index. If the model's quality is degrading and I have enough trustworthy evidence, I consider adapting the model. Drift by itself is not enough reason to retrain.

For quality regression, I would check whether required quality measures such as accuracy, faithfulness, helpfulness, or safety are getting worse. A narrow, validated behavior change may justify fine-tuning. Fine-tuning means adapting an existing model with a smaller targeted training run. A broad or persistent change in what the model needs to learn may justify retraining. I would require enough reliable evidence before either action.

Minimum evidence protects the system from noisy recent behavior. One unusual day, a small user group, duplicate events, or poor labels should not trigger a model update. I would check duplicates, outliers, label quality, and whether the pattern persists across important slices. I can also use recency weighting and guardrails. If the evidence is weak or unstable, I collect more data and make no change yet.

When a model change is justified, I build a versioned candidate from reproducible data, model, prompt, and configuration inputs. Then I run offline evaluation before any production release. Offline evaluation means testing the candidate on controlled evaluation data without exposing users to it. I compare the candidate with the current baseline against predefined release thresholds.

I would evaluate model quality such as accuracy, faithfulness, and helpfulness; safety risks such as harmful output or personal-data leakage; robustness; and important data slices such as language, region, user segment, or topic. Slice evaluation matters because a good overall average can hide a serious regression for one group.

If the candidate fails an offline threshold, I reject it and investigate the responsible data, model, prompt, or configuration. I do not release it just because one average metric improved.

If it passes offline evaluation, I move to progressive delivery. Shadow testing runs real production inputs through the candidate without letting its output affect users. This gives production-like evidence with no user impact from the candidate output. A canary release then sends only a small share of live traffic to the candidate. An A/B test can compare variants when product outcomes are part of the decision.

I move to full rollout only when the predefined release gates pass. I keep separate gates for model quality, safety, product outcomes, and infrastructure health. A service can be available and fast while its answers are worse, so deployment health and model quality must not be treated as the same thing.

After release, I monitor continuously. Deployment health includes latency, errors, cost, and availability. Model quality includes measures such as accuracy, faithfulness, and safety. Product outcomes include task success, engagement, or other outcomes that the product actually defines. I also watch data drift and user feedback. I break these measures down by important slices so a global average does not hide a local problem.

Rollback criteria are defined before release. If a critical quality, safety, product, or infrastructure threshold is broken, I revert to the last known-good version. The rollback should restore the model, prompt, configuration, and other release artifacts needed to reproduce that known-good state.

Monitoring then feeds the next lifecycle decision. The main tradeoff is stability versus freshness. Updating too slowly leaves stale knowledge or degraded behavior in production. Updating too quickly can make the system learn from noise, remove still-valid knowledge, or introduce regressions. So I prefer the smallest safe action supported by enough evidence: keep valid knowledge, expire stale information, refresh stale retrieval data, fine-tune narrow validated behavior changes, retrain broad persistent changes, or make no change when the evidence is insufficient or noisy.

1. Define the scope and version user-signal data, retrieved knowledge, training data, models, prompts, and configurations. Record lineage so each release can be reproduced.
2. Collect evidence before changing anything. Check age, relevance, legal retention, user deletion, data drift, quality regression, and minimum reliable evidence.
3. Identify the affected asset and choose the smallest justified action. Keep still-valid information. Expire stale or irrelevant information. Apply legal retention rules. Delete eligible user data where required. Re-index stale retrieved knowledge. Fine-tune narrow validated behavior changes. Retrain only for broad or persistent change with enough evidence.
4. Build a reproducible, versioned candidate when a model or configuration change is needed.
5. Run offline evaluation. Compare the candidate with the current baseline against predefined quality, safety, robustness, important-slice, product, and infrastructure thresholds that apply to the release.
6. If offline evaluation fails, reject the candidate and investigate the responsible data, model, prompt, or configuration.
7. If it passes, use progressive delivery. Use shadow testing when useful, then limited canary traffic, and use A/B testing when product comparison is needed.
8. Release broadly only when all required release gates pass.
9. Monitor deployment health, model quality, product outcomes, data drift, and user feedback separately. Break results down by important slices.
10. Roll back to the last known-good version if a critical release or monitoring threshold is violated.
11. Feed monitoring evidence back into the lifecycle policy. Do not make another learning change until enough reliable evidence supports it.

The main costs come from storing versions, rebuilding retrieval indexes, running evaluations, and training model updates. Keeping many versions uses more storage, but it makes rollback and debugging safer. Re-indexing knowledge is usually cheaper than changing the model. Fine-tuning is usually smaller than full retraining, but it still needs clean data and careful evaluation. Shadow and canary testing can increase serving cost because multiple versions may run at the same time. Slice-based evaluation also needs enough examples in each important group. The ongoing maintenance cost includes thresholds, legal rules, deletion workflows, evaluation sets, lineage, alerts, and rollback paths.

A common mistake is deleting data only because it is old. Age is only one signal; still-valid knowledge may need to stay. Another mistake is retraining whenever drift appears. The drift may only require a retrieval refresh, or it may not hurt model quality at all. It is also risky to learn from a small amount of recent behavior without checking whether the evidence is reliable. Other mistakes are using only average evaluation scores, mixing deployment health with model quality, skipping legal or user-deletion rules, releasing directly to all users, using unclear release thresholds, and having no tested rollback path.

Explain this as an evidence-based lifecycle policy, not as a fixed retraining schedule. Start with scope, version lineage, and evidence. Then show the smallest action for each signal. Finish with offline evaluation, shadow or canary release, clear gates, slice monitoring, and rollback. Emphasize that old data is not automatically bad and recent data is not automatically trustworthy.

I would run the reservoir sampler many independent times on the same stream of N positions with reservoir size k. For every position i, I would count how many final reservoirs contain it. Under uniform marginal selection, each position has probability p = k / N, so its expected count over R runs is E = Rk / N. I would standardize each count using its binomial standard deviation and compare all positions with one simultaneous threshold. If every position stays inside that threshold, the results are consistent with uniform marginal selection. If one or more positions exceed it, I would investigate possible positional bias.

This question asks you to check whether a streaming sampler treats every input position fairly. Imagine the stream has N positions and the final reservoir keeps k items. Run the sampler many times with fresh random choices. For each position, count how many runs include it in the final reservoir. If the sampler is fair, every position should appear in about the same fraction of runs, namely k divided by N. The key is to allow normal random variation while still spotting positions that appear too often or too rarely.

1. Should I check only the marginal selection frequency of each stream position?
2. Do you want a fixed significance level for the simultaneous comparison?

I would treat this as a probabilistic evaluation of the reservoir sampler. The system under test is the sampler itself. The input is a stream with positions 1 through N, and the reservoir has size k. The main property I want to check is whether every position has marginal selection probability close to k / N.

First, I create a count array with one entry for every stream position. I set every count to zero. Then I run the sampler R independent times. After each run, I look at the final reservoir. For every position included in that reservoir, I add one to its count.

For position i, let C_i be the number of runs that selected it. If the sampler is uniform, the expected marginal probability is p = k / N. The expected count is E = R p = Rk / N.

Across independent runs, the count for one fixed position follows a binomial model with standard deviation sqrt(R p times (1 minus p)). I can therefore compute a standardized deviation for each position as z_i = (C_i minus E) divided by sqrt(R p times (1 minus p)). This tells me how far the observed count is from the expected count in units of normal random variation.

Because I inspect all N positions, I should not judge every position with an unrelated single position threshold. That would increase the chance of a false alarm. I would use a simultaneous rule. A simple conservative choice is a Bonferroni threshold using alpha divided by N for each position. Another choice is to simulate correct uniform reservoir samples and calibrate the null distribution of the largest absolute z value.

The success result is not that the sampler is proven uniform. The result is that the observed counts are consistent with uniform marginal selection at the chosen threshold. If one or more positions exceed the simultaneous threshold, that is evidence of possible positional bias and I would investigate the sampler implementation.

The repeated runs should use independent random draws. For a repeatable automated test, I can seed the random number generator while still giving each run new random draws from that generator. The count array is reset before each complete evaluation. There is no external service or database in this test boundary, so no mock or integration dependency is needed.

The main tradeoff is R. A larger R gives more precise frequency estimates, but it also increases evaluation time. The standard error of the observed selection frequency for one position is sqrt(p times (1 minus p) divided by R).

An important limitation is that this test checks each position's marginal selection probability. Positions inside one reservoir are not independent because exactly k positions are selected. This evaluation does not test independence between positions within one reservoir. For small R or very small expected counts, I would prefer direct null simulation over relying only on a normal approximation.

1. Fix the stream length N, reservoir size k, number of independent runs R, and significance level alpha.
2. Create counts for positions 1 through N and set every count to zero.
3. Repeat R times. Run the reservoir sampler on the same N position stream. For every position in the final reservoir, increase its count by one.
4. For each position i, record C_i = counts[i].
5. Compute the expected marginal probability p = k / N.
6. Compute the expected count E = R p = Rk / N.
7. Compute the marginal count standard deviation sqrt(R p times (1 minus p)).
8. Compute z_i = (C_i minus E) divided by sqrt(R p times (1 minus p)) for every position.
9. Check all N positions with one simultaneous decision rule. A simple choice is a two sided Bonferroni threshold with alpha divided across N positions. Another choice is to calibrate the largest absolute z value with null simulation.
10. If all positions remain inside the chosen simultaneous threshold, report that the results are consistent with uniform marginal selection. If any position exceeds the threshold, report evidence of possible positional bias.

A common mistake is to expect every position to have exactly the same count. Random sampling naturally creates variation. Another mistake is to compare many positions with separate unadjusted thresholds, which raises the chance of false alarms. A third mistake is to use an ordinary Pearson chi square test with N minus 1 degrees of freedom on reservoir inclusion counts without accounting for dependence between positions within each run. It is also wrong to say that passing this check proves complete uniformity. The test checks marginal selection frequency, not independence between positions inside one reservoir. Finally, using too few runs can make the result too noisy to be useful.

Start with the simple idea: run the sampler many times and count how often each stream position appears. Then state the expected probability k / N and expected count Rk / N. Explain that random variation is normal, so you standardize the deviations and use one simultaneous threshold across all positions. End by saying that a pass means the results are consistent with uniform marginal selection, not that uniformity is mathematically proven.

I would process the stream one string at a time and keep a set called seen. The set stores the strings I have already emitted. For each incoming string, I check the set. If the string is new, I add it to seen and yield it immediately. If it is already present, I skip it. This preserves the first occurrence without storing the full stream. The expected time is O(n), and the extra space is O(u), where u is the number of unique strings.

The input is a stream of strings that arrive one at a time. We need to keep the first occurrence of each string and skip later copies. We should produce each useful result as soon as possible instead of waiting for the whole input. A set gives us a simple way to remember which strings were already emitted. This lets the solution work incrementally while keeping the original order of first occurrences. Memory grows with the number of unique strings, not with a stored copy of the whole stream.

1. Should string comparison be case-sensitive?
2. Can the stream be very large or continue for an unknown amount of time?
3. Should each unique string be produced immediately when its first occurrence arrives?

The input provides one string at a time. The output must contain only the first occurrence of each string. The order of those first occurrences must stay the same. In the diagram, the input is apple, banana, apple, cherry, banana, date, apple, elderberry. The output is apple, banana, cherry, date, elderberry.

Create a set named seen. A set stores unique values. Here, each value in seen means that string has already been emitted. The central invariant is: seen contains exactly the strings that have already passed to the output.

Start with seen empty. Receive the next string. Check whether it is in seen. If it is not present, add it to the set and emit it. If it is already present, skip it. Then receive the next string. The algorithm never needs to collect the complete input stream first.

Start with seen = {}. apple is new, so add it and emit apple. Now seen = {apple}. banana is new, so add it and emit banana. The next apple is already present, so skip it. cherry is new, so add and emit it. The next banana is skipped. date is new, so add and emit it. The next apple is skipped. elderberry is new, so add and emit it. The final output is apple, banana, cherry, date, elderberry.

A string can be emitted only when it is missing from seen. As soon as the first occurrence is emitted, that string is added to seen. Every later copy therefore fails the new-string check and is skipped. Because values are processed in arrival order and emitted immediately, the first-occurrence order is preserved.

The diagram shows asynchronous and synchronous generator versions. The asynchronous version uses async for to receive values from an AsyncIterator[str]. The synchronous version uses a normal for loop over an Iterator[str]. Both keep the same seen set, perform the membership check, add new strings, and yield only first occurrences. The runnable example uses the synchronous version with the exact input from the diagram.

Let n be the number of processed strings and u be the number of unique strings. Set membership and insertion are O(1) on average, so the expected total time is O(n). The seen set stores up to u strings, so auxiliary space is O(u). The stream itself does not need to be stored. A very large or infinite stream can be processed incrementally, but memory can continue growing if new unique strings continue arriving.

The key idea is to remember only whether each string has already been emitted. The seen set provides that state. Its invariant is simple: after every processed item, seen contains exactly the unique strings emitted so far. When the next string arrives, the algorithm first checks membership. A new string is added and emitted. A string already in the set is skipped. This fits streaming input because each item can be handled immediately without waiting for the complete stream.

Let n be the number of strings processed and u be the number of unique strings. Each input string needs one membership check. A new string also needs one insertion. Python set lookup and insertion are O(1) on average, so the expected total time is O(n). The seen set can contain u unique strings, so auxiliary space is O(u). The algorithm does not store the complete input stream, but its memory can keep growing when new unique strings keep arriving.

One mistake is collecting the full stream before removing duplicates. That loses the incremental processing benefit. Another is converting the whole input directly to a set, which does not represent the required first-occurrence output flow and also consumes the input first. A candidate may also forget to add a new string to seen, causing later copies to be emitted again. Using a list for membership checks makes each lookup slower as the stored values grow. Another mistake is claiming guaranteed O(n) time instead of expected O(n) time for a hash-set solution. Finally, claiming O(1) extra space is wrong because seen grows with the number of unique strings.

Explain the invariant first: seen contains exactly the strings already emitted. Then trace one new string and one duplicate. This quickly shows why the first occurrence passes through and every later copy is skipped.

I would use reservoir sampling. I first store the first k items in a reservoir. For each later item at position i, I draw j uniformly from 1 through i. If j is at most k, I replace reservoir slot j. Otherwise, I discard the new item. After n >= k items, each seen item has probability k / n of being selected. The algorithm takes O(n) time for n processed items and O(k) auxiliary space.

The stream arrives one item at a time, and we do not know its final size. We need to keep exactly k sampled items without saving the whole stream. Reservoir sampling fits this problem because its memory stays limited to k items. We first fill the reservoir. After that, each new item gets a fair chance to replace one stored item. For a finite stream, returning exactly k items requires at least k input items. A truly infinite stream needs an external stopping point before this function can return.

1. Can I assume a finite stream contains at least k items?
2. For a truly infinite stream, should I expose the current reservoir when an external stop or snapshot is requested?

The input is a stream and an integer k. Items arrive one by one. The final number of items may be unknown. The output is a list containing k sampled items when a finite stream has at least k items. We only keep the reservoir, so memory does not grow with the length of the stream.

Store the first k items directly. In the diagram example, k = 3. Item A gives [A, –, –]. Item B gives [A, B, –]. Item C gives [A, B, C]. At this point the reservoir is full.

Suppose the current item is at one-based position i. Draw j uniformly from 1 through i. If j <= k, replace reservoir position j with the new item. If j > k, discard the new item. This gives later items a decreasing chance to enter the sample while keeping the reservoir size fixed.

For item D at i = 4, the illustrated random draw is j = 2. Because 2 <= 3, replace slot 2. The reservoir becomes [A, D, C]. For E at i = 5, j = 5. Because 5 > 3, discard E, so the reservoir stays [A, D, C]. For F at i = 6, j = 1. Because 1 <= 3, replace slot 1. The shown reservoir becomes [F, D, C]. These j values are one valid random outcome, not fixed results.

After seeing n >= k items, every seen item has probability k / n of being in the reservoir. The first k items start inside the reservoir, but later replacement steps reduce their survival probability correctly. A later item enters with probability k / i when it arrives. The same rule keeps all seen items equally likely. Therefore, after any finite prefix of length n >= k, the reservoir is a uniform sample over all k-item subsets of those n items.

The Python code counts items with one-based positions. It appends the first k items. For every later item, randbelow(i) produces a value from 0 through i - 1, so adding 1 gives j from 1 through i. A replacement uses reservoir[j - 1] because Python lists use zero-based indices. Negative k is invalid. k = 0 returns an empty sample. A finite stream shorter than k raises an error. For k > 0, a truly infinite iterator never reaches the final return unless processing is externally stopped.

If n items are processed, the running time is O(n) because each item is handled once. The reservoir stores at most k items, so auxiliary space is O(k). The algorithm does not need to know n in advance.

The key idea is to keep only k items while giving every item seen so far the same chance to be selected. The invariant is: after processing n >= k items, the reservoir is a uniform sample of size k from those n items. The first k items fill the reservoir. For item i > k, drawing j uniformly from 1 through i and replacing only when j <= k gives that new item probability k / i of entering. The replacement rule also reduces the survival probability of older items by the exact amount needed to keep all items equally likely.

Let n be the number of items processed. Time is O(n) because the algorithm handles each item once and does constant work per processed item in the illustrated model. Auxiliary space is O(k) because the reservoir contains at most k references. It does not store the full stream. For a truly infinite stream, n keeps growing and the exhaustion-based function does not naturally finish. After any finite prefix with n >= k, the current reservoir is still a valid uniform k-sample.

A common mistake is choosing a random reservoir slot for every new item. That makes later items too likely to survive. Another mistake is drawing only from 1 through k instead of 1 through the current position i. Candidates also sometimes forget that j is one-based while a Python list index is zero-based. Another mistake is storing the whole stream, which breaks the O(k) memory goal. It is also incorrect to return fewer than k items silently when a finite stream is shorter than k, or to say an exhaustion-based function returns from a truly infinite iterator without an external stopping condition.

Explain the invariant before writing code: after n >= k items, every seen item must have probability k / n of being in the reservoir. Then show how drawing j from 1..i and replacing only when j <= k preserves that invariant.