other
Posted 3 hours agoQuality Analyst II
at HighLevel
United StatesRemote
Responsibilities
- Review pre-call briefs for accuracy, completeness and signal quality — customer context, adoption signals, retention flags Evaluate in-call guidance — next-best-action prompts, suggestions and recommendation flows — for relevance and accuracy in live conversation Score post-call outputs: quality grading, AI-generated summaries, CRM updates, sentiment signals Audit transcription-driven follow-ups — customer updates, recap notes, follow-up tasks — for accuracy, tone and customer impact Classify issues by
- Refine and iterate on prompts within existing skills and sub-agents Update knowledge base entries to close information gaps surfaced in reviews Tweak existing skills and orchestrations to improve output quality Test changes, validate improvements and roll out updates inside ORA Document what you changed and why — every iteration is a data point 3.
Requirements
- About us HighLevel is an AI-powered business operating system that gives agencies, entrepreneurs and SMBs the infrastructure to build, automate and scale.
- Learn more about us on our YouTube Channel or Blog Posts Role Overview We are looking for a Quality Analyst II to join Customer Success — Programs & AI Innovation as an evaluator-first specialist focused on the quality of AI-powered workflows that support our advisors before, during and after every customer call.
- You will spend most of your time evaluating AI output across ORA, our internal AI platform — reviewing pre-call briefs, in-call guidance, post-call grading and transcription-driven customer follow-ups.
- Evaluate AI Quality Across the Advisor Stack You are the quality owner for advisor-facing AI inside ORA.
- You review AI output across the full call lifecycle and surface what is working, what is broken and where the bar needs to be raised.
- Tag conversations by product area, severity and issue type with consistent classification Maintain a clear, trusted log of issue types, trends and conversation behaviors Surface recurring patterns and systemic gaps to leadership and the Lead Analyst Recommend prioritized improvement opportunities — what to fix, what to build, what to redesign Build reporting that gives Success leadership visibility into AI quality and the improvement backlog 4.
- Stay Sharp: Learn, Experiment, Share The AI space moves weekly.
- Stay current on LLM behavior, prompt patterns and AI evaluation techniques Run small experiments with new prompts, tools or evaluation approaches inside ORA Share what you learn with the broader Success team and the Lead Analyst
- The AI space changes weekly and we hire people who level up on their own without waiting for permission or perfect instructions. 2+ years of
- experience in Quality Assurance, Customer Support, AI evaluation or a related role where you reviewed and improved customer-facing or advisor-facing work Working understanding of how LLMs behave — prompt design, common failure modes, conversational AI patterns General understanding of databases and how data flows between systems Comfort reading code and making minor edits — enough to follow integrations, debug prompt behavior, iterate inside ORA and hold informed technical conversations with engineering.
- Strong written communication and attention to detail — you can document a failure mode so clearly that an engineer can act on it without follow-up Ability to follow structured guidelines and scoring rubrics consistently while suggesting improvements to them Comfort working with evolving tools and AI-assisted workflows Self-starter mentality — you can manage review volume, prioritize what matters and move work forward with minimal direction Bachelor’s degree in a related field, or equivalent practical
- experience with AI development platforms such as Claude, Cursor or similar tools Familiarity with retrieval-augmented generation (RAG), vector databases or knowledge-base architecture Familiarity with SQL or similar database querying
- Experience with APIs, webhooks or data integration between systems Background in customer success, support, sales enablement or call center environments
- Experience with the HighLevel platform or comparable SaaS products Prior
- experience evaluating AI-generated conversations, chatbots or automated support output
Additional details
- Today, HighLevel supports SMBs across 150+ countries, fueling community-driven growth rooted in real customer outcomes.
- To date, businesses operating on HighLevel have generated over $7 billion in ecosystem value, demonstrating the impact of shared infrastructure at scale.
- By centralizing conversations, automation and intelligence into one system, we help businesses move faster, reduce complexity and execute efficiently.
- Behind the platform, HighLevel powers more than 4 billion API hits and 2.5 billion message events daily.
- With 250 terabytes of distributed data, 250+ microservices and over 1 million domain names supported, our architecture is built for performance, resilience and long-term scalability.
- Our people With over 2,000 team members across 10+ countries, HighLevel operates as a global, remote-first organization built for speed and ownership.
- We value initiative, clarity and execution, creating space for ambitious people to build systems that support millions of businesses worldwide.
- Here, innovation thrives, ideas are celebrated and people come first, no matter where they call home.
- Our impact Every month, HighLevel enables more than 1.5 billion messages, 200 million leads and 20 million conversations for the more than 1 million businesses we support.
- Behind those numbers are real people building independence, expanding opportunity and creating measurable impact.