engineering
Posted 2 weeks agoStaff Software Engineer, Trust & Safety
at Replit
Foster City, United StatesRemote
Responsibilities
- IN THIS ROLE YOU WILL… - Design and implement LLM guardrails that detect abuse scenarios in AI-generated code and agent interactions - Build AI-powered detection systems that use LLMs to identify malicious patterns, classify threats, and automate response decisions - Build and operate abuse detection systems that identify phishing, cryptomining, account takeover, and financial fraud across millions of daily user actions - Design automated response mechanisms that enforce platform policies without manual
Requirements
- What makes this role unique is the AI-native nature of Replit's platform.
- You'll work on problems that barely exist elsewhere: building guardrails for AI-generated code, detecting prompt injection attacks at scale, and using LLMs as a defensive tool against abuse.
- experience applying AI to security problems, this is one of the few places you can do it in production with real attackers.
- experience in security engineering, anti-abuse, trust & safety, or fraud detection - Strong programming skills in Python and/or TypeScript for building detection systems and automation -
- Experience with SQL and data analysis at scale (BigQuery, Snowflake, or similar) -
- Experience building or fine-tuning ML/LLM-based classifiers for security or abuse detection - Familiarity with prompt injection, jailbreaking, and other LLM-specific attack vectors - Ability to investigate complex abuse patterns and translate findings into automated defenses - Familiarity with common attack patterns: phishing infrastructure, account takeover, credential stuffing, resource abuse - Clear communication skills for working across Security, Support, Legal, and Engineering teams. NICE TO HAVE: -
- Experience at a platform company dealing with user-generated content or compute abuse (hosting providers, cloud platforms, developer tools) - Background in fraud detection, payment abuse, or financial crime - Familiarity with device fingerprinting, IP reputation, and email validation services -
- Experience with CI/CD security tooling (SAST, SCA, Dependabot, Snyk) - Knowledge of container security, Linux internals, or cloud infrastructure (GCP preferred) - Prior work with abuse reporting pipelines, trust & safety tooling, or content moderation systems TOOLS + TECH STACK FOR THIS ROLE - Languages: Python, TypeScript, Go, SQL - Data: BigQuery, Hex - Detection tools: Slurper, Netwatch, Stytch (device fingerprint); ClearOut (email reputation) - CI/CD Security: Dependabot, Snyk, SAST/SCA scanners -