13 Meta AI Engineer Interview Questions & Answers

I would require every accepted comment to point to exact changed code, explain what is wrong and why it matters, cite supporting evidence, and give a concrete remediation. I would also tell the reviewer to reject speculative findings, merge duplicate findings about the same issue and location, and return no comment when the evidence is insufficient. The final output should follow a structured contract so application logic can validate required fields and changed code locations.

The goal is to stop a reviewer from producing vague comments that do not show where a problem exists or why it matters. I would provide the pull request diff, file contents, project context, and review guidelines. The reviewer should focus on lines that were added or modified. Every accepted comment must point to a changed location, explain the issue, show supporting evidence, and request a usable correction. Duplicate comments should be merged. Speculative claims should be rejected. If the code and allowed context do not provide enough evidence, the reviewer should produce no comment for that finding.

1. Should comments be limited to lines that were added or modified in the change?
2. What supporting context may the reviewer use, such as file contents, project rules, or documentation?
3. Should the reviewer return no findings when every possible issue lacks enough evidence?

I would organize the prompt around input, analysis, an output contract, and rejection rules.

First, I would provide the pull request diff, relevant file contents, project context, and review guidelines as data. I would instruct the reviewer to focus on added or modified lines. Nearby context can help explain a finding, but the comment itself should point to concrete changed code.

Next, the reviewer gathers evidence. For each possible issue, it should use the code, allowed documentation, or project rules to support the claim. If the evidence is weak or missing, it should reject the finding instead of guessing.

For every accepted finding, I would require exactly one comment with five ideas. It should state what is wrong and why it matters. It should give the exact file and changed line range. It should cite the supporting evidence. It should provide a concrete remediation, with a small code example or steps when useful. It should also state the affected behavior and a confidence level.

I would add explicit rejection rules. Do not comment when there is no supporting evidence. Reject speculative or style only findings that are outside the review guidelines. Merge a duplicate about the same issue and location. Reject an issue that was not introduced or affected by the current change.

The model output is probabilistic, so application code should still treat it as untrusted. Deterministic validation can check the required structure and whether reported locations belong to the supplied change. Semantic evidence still needs careful review because a valid structure does not prove that a finding is correct.

1. Provide the pull request diff, relevant file contents, project context, and review guidelines as clearly separated input data.
2. Instruct the reviewer to focus on lines that were added or modified.
3. Require the reviewer to gather evidence from the changed code, allowed documentation, or project rules before accepting a finding.
4. For every accepted finding, require what is wrong and why, the exact file and changed line range, supporting evidence, a concrete remediation, the affected behavior, and a confidence level.
5. Reject a finding when evidence is missing or too weak to support the claim.
6. Reject speculative or style only findings that fall outside the review guidelines.
7. Merge duplicate findings about the same issue and location so one issue produces one comment there.
8. Reject an issue that was not introduced or affected by the current change.
9. Return the accepted findings in a structured format.
10. Use deterministic application logic to validate required fields and changed code locations before displaying or acting on the comments.

A common mistake is asking the model to review code without defining what evidence it may use. Another is allowing a comment that does not point to exact changed code. Teams may also request a fix but accept vague advice instead of a concrete remediation. Other mistakes include keeping duplicate comments about the same issue and location, accepting speculative style comments outside the review rules, reporting problems that were not introduced or affected by the change, and forcing the model to produce a finding even when the evidence is insufficient.

Start with the contract. Say that every accepted comment needs a changed location, an explanation, evidence, and a concrete remediation. Then explain the rejection rules for weak evidence, unsupported claims, duplicates, and issues unrelated to the change. Finish by separating probabilistic model output from deterministic application validation.

I would use several independent controls. First, I would keep private repositories out of training data unless their use is explicitly approved. During inference, I would isolate each tenant and repository, give the agent only the files it needs, remove secrets, restrict tools and network access, and use short lived credentials. I would scan model outputs for secrets and private code before anything leaves the system. I would also keep controlled audit logs, support immediate access revocation, contain suspected leaks quickly, and continuously run tests that try to extract private code.

The main goal is simple: private source code should only be seen when it is truly needed, and it should not escape to another user or outside system. I would protect it before the model is built, while the agent is working, and after the agent creates a response. I would also limit who can reach the code, record important access, quickly remove access when needed, and regularly test whether someone can trick the system into revealing information that should stay private.

1. Can customer repositories ever be used for model training or improvement?
2. Does each customer need separate storage, credentials, and execution environments?
3. Which logs may contain prompts, code, model responses, or tool results?

I would start with training data. Private repositories should be excluded unless there is explicit permission and a valid data policy. The ingest path should check where data came from and whether its license allows use. It should remove secrets and personal data, filter duplicate and very similar code, keep provenance, meaning a record of each item's origin, and audit what was used. Tests should then try prompt extraction, membership inference, which checks whether the model reveals that particular private code was in its training data, very similar code queries, and code completion probes. A detected leak should lead to data correction, stronger filters, or model adjustment.

At inference time, every request should be authenticated and authorized against the correct tenant and repository. The context builder should fetch only required files, truncate large inputs, mask secrets and identifiers, and summarize where possible. Tenant isolation, repository access control, network segmentation, and separate tenant keys reduce the chance that one request can reach another tenant's code.

The prompt guard should treat repository text and user text as untrusted data. It should detect prompt injection and extraction attempts while keeping system instructions authoritative. The model should run in an isolated session with no internet by default, no write access to private storage, and short lived secrets.

Every response should pass an output guard before release. It should scan for secrets and personal data, detect private code or suspiciously similar content, and apply policy based redaction or refusal. Provider controls should prevent unapproved use of customer data for training, minimize retained logs, protect data with encryption, and keep retained data only for the shortest approved period.

Finally, use least privilege roles, which give each identity only the access it needs. Use just in time access, which grants temporary access only when required, short lived tokens, regular permission reviews, and immediate revocation. Monitor unusual access, suspicious prompt patterns, policy blocks, and possible extraction attempts. If leakage occurs, confirm scope, revoke keys, block sessions, isolate affected systems, remediate the cause, notify affected parties when required, and feed lessons back into continuous tests and policies.

1. Training contract: allow only approved training sources. Remove secrets and personal data. Track provenance, meaning where each training item came from, and keep access history.
2. Request contract: authenticate the user, authorize the tenant and repository, and rate limit requests before retrieval.
3. Context contract: fetch only needed files, truncate large inputs, mask sensitive values, and summarize when possible.
4. Prompt contract: keep system instructions separate from untrusted repository and user content. Detect prompt injection and private code extraction attempts. Refuse requests when policy requires it.
5. Runtime contract: isolate the session, restrict tools and network access, prevent writes to private storage, and use short lived credentials.
6. Output contract: treat model output as untrusted. Scan it for secrets, personal data, private code, and suspiciously similar content before release.
7. Provider and logging contract: prevent unapproved training use, minimize retained data, protect stored and transmitted data with encryption, and retain logs only as long as policy requires.
8. Access contract: apply least privilege, temporary access when needed, short lived tokens, regular reviews, strong authentication, and immediate revocation.
9. Failure contract: detect abnormal access or extraction behavior, contain the session, revoke credentials, investigate scope, remediate the cause, and update tests and controls.
10. Evaluation contract: continuously run extraction tests such as prompt extraction, membership inference, very similar code queries, and code completion probes.

A common mistake is relying on a system prompt that says not to reveal code. A model can still fail, so deterministic controls must enforce repository boundaries and output policy. Another mistake is giving the agent an entire repository when only a few files are needed. Teams also sometimes forget that prompts, tool results, model responses, and logs can contain private code. Other mistakes include reusing credentials across tenants, keeping tokens alive too long, allowing unrestricted network access, using customer data for training without explicit approval, scanning inputs but not outputs, keeping sensitive logs indefinitely, and testing only normal requests instead of deliberate extraction attacks.

Explain the answer as layers. Start with clean training data, then tenant and repository boundaries, then minimal inference context, prompt and tool safeguards, output scanning, access revocation, monitoring, incident containment, and extraction tests. Make it clear that no single prompt can guarantee privacy. The strongest design combines prompt rules with deterministic access and data controls.

I would use section-aware chunks, embeddings for semantic matches, and BM25 for exact terms. I would retrieve a modest set from both, fuse and rerank the candidates, then build cited context from the best few chunks. If evidence is weak, I would broaden retrieval, clarify, or say there is not enough support.

This question asks how I would help a local search tool find the best parts of Markdown and PDF files before it writes an answer. I need to explain how I break files into useful pieces, how I match a question to those pieces, how many possible matches I keep, and how I keep the search responsive. I also need to explain what happens when the files do not contain enough useful information, so the tool does not pretend it has strong support when it does not.

1. Should the tool search only the local Markdown and PDF files, or can it use any external source?
2. Do the files have access rules that must be applied before retrieved text reaches the model?
3. Is low response time more important than getting the best possible ranking from a more expensive reranker?

I would separate the design into offline ingestion and online retrieval.

Offline, I would parse each Markdown and PDF file and keep useful structure such as headings, page numbers, file paths, and chunk identifiers. I would create medium, section-aware chunks. A chunk should be large enough to hold one complete idea but small enough to retrieve precisely. I would use a small overlap only when a section boundary could split useful context.

For representation, I would create an embedding for each chunk. An embedding is a numeric representation of meaning. I would also build a BM25 lexical index. BM25 is a keyword-ranking method that works well for exact terms and rare words. The two signals solve different problems. Vector search helps when the user and document use different words with similar meaning. BM25 helps when exact names, codes, or uncommon terms matter.

At query time, I would first do light query processing, such as fixing obvious typos, expanding a useful synonym, or extracting a filter when needed. Then I would retrieve a modest candidate set from vector search and another modest set from BM25.

I would combine the two ranked lists with reciprocal rank fusion, or RRF. RRF gives credit to chunks that rank well in either list without requiring vector and BM25 scores to use the same scale. I would then apply a cross-encoder reranker only to this small merged set. A cross-encoder reads the query and each candidate together and gives a stronger relevance score. Reranking only a small set avoids paying that higher cost across the whole corpus.

Next, I would keep only the best few chunks that fit the model's context budget. I would carry source metadata such as file path, chunk id, heading path, page number, token information, and access tags when those fields exist. This supports inline citations and lets the system filter restricted content before it reaches the model.

For latency, I would keep both retrieval indexes local, retrieve only a modest number of candidates, and use the more expensive reranker only on that limited set. This controls online work without claiming a fixed response-time number.

The weak-evidence path is important. If retrieval scores are weak or no relevant chunks are found, I would not fabricate a grounded answer. I would broaden or rephrase retrieval, ask a clarifying question when useful, or tell the user that the local documents do not provide enough evidence.

1. Offline ingestion: parse Markdown and PDF files and keep useful structure such as headings, page numbers, file paths, and chunk identifiers.
2. Chunking: create medium, section-aware chunks with small overlap only when boundary context needs it.
3. Representation: create embeddings for semantic retrieval and a BM25 lexical index for exact-word retrieval. Keep source metadata with each chunk.
4. Online query processing: fix obvious typos, expand useful synonyms, or extract filters only when needed.
5. Hybrid retrieval: retrieve a modest candidate set from vector search and another modest set from BM25.
6. Fusion: combine the two ranked lists with reciprocal rank fusion.
7. Reranking: run a cross-encoder reranker only on the small merged candidate set, then keep the strongest few chunks.
8. Context assembly: fit the strongest evidence within the model context budget. Keep source metadata for citations and apply access filters before restricted text reaches the model.
9. Generation: generate from the retrieved evidence and cite sources inline.
10. Weak or no evidence: broaden or rephrase retrieval, ask for clarification when useful, or state that the local documents do not provide enough evidence.

Common mistakes are using only vector search and missing exact terms, using only keyword search and missing semantic matches, making chunks so large that retrieval becomes vague, making them so small that useful context is fragmented, retrieving too many candidates and increasing latency, reranking the whole corpus, dropping source metadata needed for citations, failing to apply access filters before restricted chunks reach the model, and forcing an answer when retrieval has weak or no evidence.

Explain the flow in order: section-aware chunks, embeddings plus BM25, modest candidate sets, RRF fusion, cross-encoder reranking, cited context, and weak-evidence behavior. For each choice, briefly say what problem it solves and what cost it adds.

At a high level, I would run deterministic checks before letting the LLM review a pull request. The main challenge is giving the model useful facts without giving it raw tool authority. I would split the flow into analysis, normalization, and model review. The checks cover changed-code structure, bugs, security, policy, and missing tests. Their findings are normalized and sent with confidence and evidence. The trade-off is that the model can reason only over the bounded information it receives.

The goal is to inspect a pull request with predictable checks before an LLM reviews it. This matters because the model should reason from clear findings instead of controlling analysis tools directly. The pull request provides changed files, the diff, and metadata. Five deterministic checks inspect different kinds of problems in parallel. Their results are then combined into one consistent format. The LLM receives those bounded findings with confidence and evidence, plus the diff and metadata. This keeps the model focused on review comments and suggestions while deterministic tools own the analysis work.

1. Should every repository run the same five checks, or can repositories enable different rules?
2. Should every finding include the exact rule or check version used to produce it?
3. What per-file and total finding limits should we apply before sending results to the LLM?

I would start with the Pull Request as the input. It provides changed files, the diff, and metadata for the checks.

The important boundary is that deterministic tools run before the model. The LLM receives their results later. It does not get raw authority to run those tools itself.

Next, I would run the five checks shown in the diagram. Changed-Code Structure parses the diff and builds an AST or structure map. Its output includes files, symbols, functions, a call graph, and a change summary.

Static Bug Analysis uses linters, type checkers, nullability checks, and similar rules. It returns issues with severity, location, and rule ID. Security Analysis looks for vulnerabilities and misconfigurations. It returns findings with a CWE or OWASP identifier when the checker provides one, plus severity and location.

Style & Policy Checks find formatting, naming, and repository-policy violations. Their output includes the rule, location, and an auto-fix when safe. Missing Tests Analysis looks for changed code without enough tests using coverage information and heuristics. The first four checks have rule-derived confidence. Missing-test findings use check-derived confidence because their signals may include heuristics.

Each check sends its output to Aggregate & Normalize Findings. This step removes duplicate findings, ranks them by severity, and converts them into one consistent schema.

This layer keeps deterministic facts rather than adding opinions. That gives the Code-Review Agent a predictable input contract even though the checks produce different kinds of results.

Before the LLM receives the findings, each result follows the Findings Contract. Structured fields include the tool, rule ID, file, location, message, and severity.

Confidence is derived from the originating rule or check, and the rule version is included. Evidence includes the rule or check ID plus an exact bounded snippet or trace. This can contain line numbers, rule output, or a link. Per-file and total issue caps keep the input bounded. Sensitive paths can be redacted.

Finally, the Code-Review Agent receives the normalized findings with confidence and evidence, plus the diff and metadata. It combines those findings with repository context to write review comments and suggestions.

The LLM cannot fetch new data or rerun the deterministic tools. That is the main authority boundary. The benefit is a clear and traceable review input. The downside is that the model can reason only from the bounded evidence supplied by the pre-LLM pipeline.

The benefit is that the LLM receives clear findings instead of raw tool access. Each result has a known source, location, severity, confidence source, and supporting evidence. Normalizing the outputs also gives the model one consistent format. The downside is that this boundary limits what the model can investigate by itself. It cannot rerun a checker or fetch extra data. Missing-test analysis may also use heuristics, so its confidence can differ from a strict rule result. We accept this because bounded inputs make reviews easier to trace, control, and explain.

At a high level, I would build this as a bounded code-review workflow that turns pull-request changes into useful, evidence-based comments. The main challenge is letting the model reason about code without letting it act freely. I would explain three flows: inspect and analyze the changed code, generate and validate grounded feedback, then let the developer review and approve the result. Deterministic checks, least-privilege permissions, budgets, and observability keep the workflow safe. The trade-off is more control and reliability at the cost of extra validation work.

The goal is to help a developer review a pull request before it is merged. The system should find real bugs, security problems, weak tests, and code-quality issues. It should explain each finding and point to useful evidence. The hard part is that AI reasoning can be uncertain, while review comments must still be safe and useful. The diagram solves this with one bounded workflow. It inspects changed code and repository context, analyzes the change, creates grounded comments, runs deterministic checks, and keeps the developer in control of approval.

1. Should the agent only post comments, or may it ever change code?
2. Which repository files and security data may the agent read?
3. Which deterministic checks must pass before comments are posted?
4. How strict should token and time budgets be for each pull request?

I would start when a pull request is opened or updated. A webhook carries its metadata, changed files, and diff into the workflow. Repository Context adds the file tree, recent commits, coding standards, project docs, and dependency graph. External Knowledge adds security and language references.

Next, Inspect Changed Code parses the diff and builds an AST, which is a structured view of the code. It identifies changed functions, APIs, and data flow. The AI Code Review Agent plans risky work first, tracks progress, calls approved tools, and stops when work is done or budget is reached.

The Analyze stage checks four areas. Bug Detection looks for logic errors, null-safety problems, and edge cases. Security Analysis looks for injection, authentication issues, cryptography mistakes, exposed secrets, and unsafe APIs. Missing Tests finds changed behavior without tests or with weak coverage. Code Quality checks complexity, style, duplication, and naming.

Grounded Comment Generation turns findings into specific comments. Each comment explains why the issue matters, points to a file and line, suggests a fix, and may show a before-and-after example. Evidence comes from the diff, repository context, docs, rules, and analyzer results. Deterministic Validation reruns static analyzers, lint or type checks, build or unit tests, policy checks, secret checks, and comment-format or link checks. A failed check follows the No path back for correction or re-analysis. A pass follows Yes to posting.

Post Review & Feedback writes inline comments and a summary status. Findings can be Blockers, Must Fix, Consider, or Looks Good. The developer reads the feedback, updates the pull request, and can request another review. At Approved?, No returns to Developer Action. Yes allows Merge.

Permissions & Safety uses least privilege, meaning only needed access. The agent can read repository code, config, and docs, and write PR comments. It cannot merge directly or access secrets or production data. All actions are audited and logged. Quality & Cost Controls use budgets, model routing, cached context and embeddings, deduplication, and early stopping. Observability & Continuous Improvement tracks latency, cost, pass rate, feedback, errors, traces, and periodic evaluations. The trade-off is extra checking in exchange for safer reviews.

The benefit is that the agent does not rely on the model alone. Static checks, tests, policy checks, and evidence links make the review more trustworthy. Least-privilege access also limits what the agent can do. Token and time budgets stop one pull request from using too much work. Cached repository context and embeddings can reduce repeated analysis. Deduplication avoids posting the same finding many times. The downside is extra complexity and slower reviews. More validation steps take time, and re-analysis can use more budget. We accept this because safer comments and clear evidence are more important than the fastest possible review.

At a high level, this system turns a support ticket into a bounded investigation that can gather evidence, test hypotheses, and recommend or take safe actions. The main challenge is giving the agent useful tools without giving it unlimited control. I would explain three flows: ticket understanding and evidence collection, the Observe-Decide-Act investigation loop, and controlled remediation with verification. Human approval, checkpoints, audit records, and production evaluation keep the system safe. The trade-off is stronger safety at the cost of slower high-risk actions.

The system must take a support or engineering ticket and investigate it much like an experienced engineer would. It first understands the problem, gathers useful evidence, tests possible explanations, and decides what to do next. The hard part is allowing useful automation without letting the agent make unsafe changes. The diagram handles this with ticket ingestion, evidence collection, an Observe-Decide-Act loop, controlled remediation, verification, and always-on safety controls. Humans can take over whenever needed.

1. Which actions may run automatically, and which always need human approval?
2. Which systems may the agent read from or change?
3. What should happen when evidence stays weak or conflicting?
4. Which production outcomes matter most for this system?

I would start by turning the incoming ticket into a clear problem statement. A ticket may come from email, a portal, chat, or an API. The system uses NLP to understand and classify it, then produces a normalized ticket.

Next, the system gathers relevant data across the stack. The diagram includes logs, metrics, traces, deployments, incidents, documentation, and topology or configuration data. This matters because the agent should base decisions on evidence instead of guessing.

The collected context enters the Agentic Investigator. State & Memory keeps the ticket context, evidence, hypotheses, actions already taken, and their results.

During OBSERVE, the agent selects what to inspect next from the current state and available knowledge. During DECIDE, it builds and ranks hypotheses, plans the next action, and chooses which tool to use. The Hypothesis Tracker keeps several possible causes and records evidence for or against each one.

During ACT, approved tools gather more evidence or perform allowed actions. The visible tool choices include Tool Use through structured APIs, Search & RAG for documents and runbooks, Code / Script analysis tools, Workflow orchestration, and Notification updates. Planning & Control sets goals, stop conditions, tool budgets, timeouts, and escalation rules.

When enough evidence exists, Diagnose & Recommend produces a Root Cause Analysis with an evidence summary, contributing factors, and confidence. It also produces Recommended Actions with a step-by-step plan, expected impact, risks, and trade-offs.

Remediate uses bounded authority. Low-risk actions may auto-execute within policy. Medium-risk actions request approval. High-risk actions require a human to execute them. Actions are logged, and they should be idempotent, meaning repeating the same action should not create extra unwanted effects. They are reversible when possible.

After remediation, Verify & Close checks the result with tests, metrics, or canaries. If successful, the ticket closes with a summary. If not, the system adapts the plan, continues, or escalates.

Governance, Safety, and Operations stay active across the whole design. Access & AuthZ applies least privilege per tool and action. Guardrails enforce policies and safe action boundaries. Audit & Trace records decisions, tool calls, inputs, and results. Checkpoints & Recovery save state so work can roll back or resume safely after failure. Observability tracks agent metrics, quality, latency, cost, and errors. Evaluation & Learning uses outcomes and feedback to improve prompts and playbooks. Production evaluation tracks resolution rate, time to resolution, action success rate, escalation rate, user satisfaction, and cost per ticket.

The benefit is that the agent can investigate tickets with a repeatable process and gather evidence quickly. Bounded authority limits what it can change without approval. The downside is that approvals, checkpoints, and verification add time. High-risk problems may still need a human. State & Memory also adds work because evidence, actions, and results must stay consistent. Tool budgets and stop conditions protect cost, but they may stop an investigation before every question is answered. We accept these limits because safe automation matters more than maximum automation. More evidence may improve a diagnosis, but extra searches also increase latency and cost.

At a high level, I would predict the chance that a user gives negative feedback on each candidate post, then use that probability as a penalty in feed ranking. I would build time-correct examples from impressions and later feedback, separate strong explicit negatives from weaker implicit signals, and use user, post, and context features. I would calibrate the model before serving. Online, a feature service feeds the trained model, and ranking subtracts a weighted dislike penalty. I would validate offline first, then run an A/B test because better offline metrics may not improve the real feed.

The goal is to make the feed show fewer posts that users are likely to dislike. We first watch what happens after a post is shown. A clear action such as hiding or reporting a post is strong evidence. A quick scroll or long ignore is weaker evidence. We use past information to learn a score for each new post. That score lowers unwanted posts in ranking. The main challenge is learning from feedback without using future information, while still checking whether the change actually improves the user experience.

• Should a strong negative action within seven days define the main target?
• Should weak signals affect training with smaller weights than explicit negatives?
• Which online metric should decide whether the experiment ships?

I would start by defining an observable negative-feedback target. After a post impression, I watch for strong negative actions during the next seven days. Examples are Not interested, Hide post, Report post, or hiding all posts from an author. These strong actions make the target label y = 1. If no strong negative appears in that window, the main target label is y = 0.

I would also keep weaker negative signals. Quick scrolling, collapsing, or ignoring a post can suggest dissatisfaction. They are noisier, so I would give them less training weight than explicit actions instead of treating them as equally strong evidence.

Every user-post interaction is logged with a timestamp. For an impression at time t, the features must come from time t or earlier. The negative label can be observed later, inside the window from t to t plus seven days. This prevents future information from leaking into the model features.

I would also deduplicate repeated impressions before building the training examples. The logged events include impressions, clicks, hides, and reports. The resulting training data connects each user, post, context, timestamp, features, and later negative outcome.

The feature set follows the diagram. User features include age, locale, interests, session signals, and past likes, hides, or reports. Post features include author, topic, content embeddings, and past engagement statistics. Context features include device, network, time of day, feed position, and a surface such as Home or Video.

A feature store keeps low-latency, reusable features. The same feature definitions should support both model training and online scoring. This reduces differences between what the model learns from and what it sees when serving predictions.

The model predicts P(dislike within seven days). This is a binary classification problem because the main target is either a strong negative outcome or not.

I would use weighted binary cross-entropy, with explicit negatives receiving more importance than weaker implicit signals. This also helps when strong negative events are less common. I would use a time-based train and validation split so later behavior does not leak into earlier training examples.

Offline evaluation includes AUC-ROC, PR-AUC, calibrated log-loss, Lift@K or Capture@K, and a calibration curve. After model fitting, I would calibrate probabilities on validation data. Calibration means making the predicted probabilities better match observed outcome rates. The calibrated score is then used online.

Candidate generation first retrieves posts the user might be interested in. The feature service then produces real-time user, post, and context features for each candidate. Those features go into the trained dislike model, which returns P(dislike).

The ranking layer combines that prediction with the base relevance score. The diagram uses Final Score = Base Relevance - lambda times P(dislike). Lambda controls how strongly predicted dislike reduces a post's rank. A larger value makes the system more averse to negative feedback. The highest final scores become the Top N posts shown to the user.

Offline metrics are useful, but they do not prove the feed experience improved. I would therefore run an A/B test against the existing ranking system. Primary metrics include a lower hide or report rate and higher meaningful engagement. Guardrail metrics include time well spent, diversity, and user satisfaction.

The monitoring loop analyzes the experiment, collects new signals, and updates the data, features, and model when needed. This connects real user behavior back to future training. I would ship only when the online experiment shows a useful improvement without unacceptable guardrail regressions.

The main trade-off is how strongly we trust negative signals. Explicit actions such as Hide or Report are clear, but they happen less often. Weak signals such as quick scrolling are common, but they can be noisy. Giving weak signals smaller weights uses more information without pretending every signal means the same thing. Time-based splitting is safer because it matches real deployment, but behavior may change over time. Calibration adds another model step, but it makes the dislike probability more useful for ranking. A larger ranking penalty can reduce unwanted posts, but it may also lower relevant content too much. Offline metrics are faster to measure. Online experiments take longer, but they show whether real users actually benefit.

At a high level, I would treat notifications as candidates from product, social, advertiser, and system sources. Each source creates candidates with metadata. I would first apply hard eligibility rules and deduplicate similar items. Then I would apply user fatigue and frequency controls before ranking the allowed set with real-time context and multiple objectives. Business and suppression rules choose priority and the final items to serve. Delivery outcomes become training labels and experiment feedback. The main trade-off is balancing user value against fatigue, safety, and added system complexity.

This problem is about choosing which notifications a person should receive, when they should receive them, and which ones should be skipped. Many parts of the product can create possible messages. The system must avoid duplicates, respect user choices, reduce overload, and still show useful items. It must also learn from what users open, dismiss, or report. I would follow the diagram from candidate creation, through filtering and ranking, to final delivery, tracking, training, and experiments.

• Which notification sources are most important: product, social, advertiser, or system?
• Can critical notifications, such as security alerts, override normal rules?
• Should ranking optimize mainly for engagement, user value, safety, or a mix?
• Which delivery channels are available for each notification type?

I would start by letting each source create its own raw candidates. Product can create tips and reminders. Social can create likes, comments, mentions, and follows. Advertiser sources can create promotions or recommendations. System or transactional sources can create security alerts or order updates. Each source has a Candidate Generator per Source. It creates raw candidates with metadata. A shared candidate format lets later stages compare candidates from different sources.

The next step is deterministic filtering. Hard Eligibility Filters must pass before ranking. They check whether the user is active and reachable. They also check global or channel opt-in, policy and safety, content validity and ownership, cooldown rules, and rate limits by type or source. Deduplication then groups notifications that mean the same thing across sources. For example, the same promotion may arrive from two sources. The system keeps the best version instead of showing both.

Eligible candidates then pass through User Fatigue and Frequency Controls. User Fatigue Scoring looks at recent notification volume, dismissal or snooze rate, engagement trend, and time context. Frequency Capping limits messages per user, type, and channel. The diagram also has Budget Allocation. It decides how many notifications a user can receive now and which types can use that budget. These controls reduce overload before expensive ranking and final selection.

The allowed set goes to Multi-Objective Ranking. Feature Inputs come from Data Stores and the Real-time Context Service. Stored data includes user profiles, notification history, preferences, content, policies, and experiment configuration. Real-time context includes device, operating system, app, network, location, current activity, session, and time features. The Ranking Model predicts several outcomes, including engagement, dismissal, conversion or value, and report risk. Scoring and Re-ranking combine these objectives to maximize useful value while limiting fatigue and negative actions.

The ranked list then reaches Priority and Suppression Decisions. Business Rules can assign critical, high, normal, or low priority. They can also apply override rules and channel constraints. Suppression Rules remove items that are too similar to something shown recently, were snoozed, have low predicted value, or have exhausted their quota. Final Selection chooses the top notifications to serve per channel. This keeps deterministic product rules after ranking, where they can still block an otherwise high-scoring candidate.

Serving delivers selected notifications through push, in-app, email, or SMS. Tracking records impressions, opens, clicks, dismissals, and conversions. Training Data and Labels use implicit signals such as open, click, dwell, conversion, and share. They also use explicit signals such as dismiss, snooze, report, and block. Model Training and Evaluation trains ranking and prediction models, validates them offline, checks calibration and bias or safety, and compares champion and challenger models. Online Experiments use A/B tests or multi-armed bandits to test ranking weights, rules, frequency caps, and templates. Monitoring and Observability watches delivery rate, engagement, dismissal and snooze rates, complaint and block rates, latency, errors, fatigue scores, experiment health, and resource usage.

The benefit of this design is that hard rules and learned ranking are separated. Safety, opt-in, validity, cooldowns, and rate limits do not depend on model scores. The model only ranks candidates that are already allowed. This is safer and easier to explain. The downside is that several stages add latency and operational complexity. Fatigue controls reduce annoyance, but they can also hide useful notifications. Multi-objective ranking can balance user value and negative feedback, but its weights need careful tuning. Deduplication improves the experience, but incorrect grouping can remove a useful message. Experiments help tune these choices, while monitoring provides guardrails for complaints, delivery quality, latency, fatigue, and cost.

At a high level, I would build the coding assistant around repository-aware context, safe model generation, and fast streaming back to the developer. A developer sends a prompt, selected code, and IDE context through the API Gateway. The Orchestration Service manages the request, retrieval finds relevant repository information, and the LLM Gateway routes generation to the right model with fallbacks. Tools and safety checks validate risky actions and outputs. Caching, tenant isolation, monitoring, cost controls, evaluation, canary releases, and rollback keep the system fast, private, reliable, and practical to operate.

The goal is to help developers write, understand, change, and test code using information from their own repository. The assistant should not rely only on the current prompt. It should use useful files, symbols, documentation, recent work, and repository rules. It must also protect private code and reduce unsafe suggestions. The design therefore combines repository search, model generation, controlled tools, validation, feedback, and operational controls. I would explain it by following the attached flow from the developer's IDE, through context and model serving, and back to the streamed response.

• Which experiences matter most: completion, chat, edits, refactoring, or tests?
• How strict are repository privacy and tenant-isolation requirements?
• Should we favor lower latency, stronger models, or a balance?
• Which generated actions may use execution or testing tools?

The developer works through an IDE integration such as VS Code, JetBrains, Neovim, web or CLI, or mobile. The request contains the prompt, selected file or code, and IDE context. It first reaches the API Gateway. The gateway handles authentication, rate limiting, and tenant isolation. Tenant isolation means one organization cannot access another organization's repository data. The accepted request then moves to the Orchestration Service, which manages the session, routing, context assembly, and caching.

Repository data enters a separate indexing path. Git repositories, file contents, code metadata such as syntax structure and symbols, issues, pull requests, documentation, and README files are processed by the Indexing Pipeline. It chunks and parses content, creates embeddings, removes duplicates, and applies access controls. An embedding is a numeric representation used for semantic search. These embeddings are stored in the Vector DB. This path keeps searchable repository knowledge ready before interactive requests arrive.

For an interactive request, the Retriever finds relevant material using semantic search, keyword search, or a hybrid of both. Retrieved candidates can be ranked again by the Reranker so the most useful passages appear first. The Context Assembler combines selected code, documentation, the file-system view, and conversation history. The goal is to use a small amount of high-value context instead of sending an entire repository. The cache stores reusable responses or embeddings to reduce repeated work and improve latency.

The LLM Gateway handles model routing, fallbacks, and load balancing. The Model Layer can contain hosted frontier models, fine-tuned or adapted models, and smaller fast models used locally or at the edge. A simple request can use a faster model, while a harder task can use a stronger model. This routing also helps control cost. If the preferred model is unavailable, the gateway can use a configured fallback. The model path remains separate from client logic, so model choices can change without changing every IDE integration.

The design gives the assistant controlled access to tools such as sandboxed code execution, unit tests, linting, formatting, static analysis, documentation search, and dependency resolution. A sandbox is an isolated environment that limits the risk of running generated code. The Safety & Policy layer checks prompt injection, private information or secrets, allow-or-deny code rules, license and compliance concerns, and output validation such as schemas or tests. These deterministic checks are important because model output is probabilistic and may still be wrong or unsafe.

The Response Generator formats the generation result together with the selected context. It can return completions, explanations, edits, tests, refactors, or commands and show sources when available. The Response to Client component sends that result back to the IDE. Streaming improves perceived latency because the developer can see useful output before the complete response is finished. This keeps the interactive path responsive while the deeper repository, model, and validation work stays behind the service boundary.

The feedback loop collects user feedback, accepted or edited completions, telemetry with consent, and sampled human review. The encrypted Data Lake stores raw data, anonymized data, and features used by the Training / Adaptation Pipeline. Approved code and instruction examples can be curated, filtered, trained on, evaluated, and registered before rollout. Evaluation covers test correctness, lint and static-analysis quality, helpfulness, task success, latency, and adoption. Operations track metrics, traces, logs, cost, availability, privacy controls, and service-level objectives. Feature flags and canary releases limit rollout risk. If quality or reliability drops, the system can roll back the affected model or release.