Braintrust — Customer Solutions Architect
Application Overview
| Field | Value |
|---|---|
| Company | Braintrust |
| Role | Customer Solutions Architect |
| Applied | 2025-01-20 |
| Status | Applied |
| Source | |
| Link | View Posting |
| Resume | Customer Solutions Architect variant |
Company Overview
Braintrust is the AI observability platform. By connecting evals and observability in one workflow, Braintrust gives builders the visibility to understand how AI behaves in production and the tools to improve it.
- Customers: Notion, Stripe, Zapier, Vercel, Ramp
- Size: 11-50 employees
- Industry: AI/ML Developer Tools
- Stage: Startup (Series A/B likely based on customer base)
Job Description
Customer Solutions Architect - Field Engineering Team
About the Role
Deeply technical, customer-facing role focused on helping teams deploy and operate Braintrust in production environments. Work with platform, DevOps, and ML infrastructure teams at leading AI-forward organizations.
Key Responsibilities
Customer Enablement & Support:
- Act as trusted advisor for customers deploying and scaling Braintrust in production
- Lead onboarding and implementation engagements (customer-hosted infrastructure, observability pipelines, evaluation workflows, third-party integrations)
- Debug customer issues across layers (SDKs, cloud infrastructure, model outputs)
- Own proactive health checks, upgrade support, and reliability best practices for self-hosted deployments
Solution Design & Implementation:
- Design and document end-to-end customer evaluation pipelines using Braintrust SDKs and APIs
- Build technical artifacts (scripts, templates, notebooks) to accelerate onboarding
- Consult on architectural decisions: LLM evaluation, prompt versioning, eval reliability
Internal Collaboration:
- Translate customer friction into actionable product feedback
- Partner with sales, support, and engineering on handoffs
- Contribute to internal runbooks and documentation
Requirements
Must Have:
- Customer-facing technical roles (Solutions Architect, Forward Deployed Engineer, etc.)
- Fluency in Python or TypeScript; comfort debugging code, APIs, infrastructure
- Strong DevOps/cloud architecture knowledge (Docker, Terraform, Kubernetes, AWS/GCP/Azure)
- Ability to explain technical concepts to engineers and non-technical stakeholders
- Comfortable working autonomously in fast-paced startup environment
Nice to Have:
- Self-hosted or hybrid SaaS deployment experience
- Familiarity with LLM/AI development workflows
- CI/CD pipelines and networking/security best practices
Why This Role
Company Interest
- Mission alignment: Solving the exact problem I encountered building AI workflows — “without visibility into how AI behaves in production, you’re flying blind”
- Customer base: Working with elite engineering teams (Notion, Stripe, Zapier, Vercel)
- Market timing: AI observability is critical infrastructure as LLMs go to production
- Startup stage: Small team (11-50), broad ownership, high impact
Role Fit
- Direct experience match: 100+ deployments across managed cloud, self-hosted, and hybrid
- AI expertise: Co-founded AI Transformation Initiative; built LLM evaluation pipelines
- Technical depth: Python, SDKs, APIs, cloud infrastructure debugging
- Customer success: 120+ enablement sessions, 4.8/5 satisfaction, escalation resolution
Concerns/Questions
- Salary not disclosed — need to understand comp range
- What’s the on-call/support rotation structure?
- What percentage of customers are self-hosted vs managed cloud?
- What’s the current team size for Field Engineering?
Match Analysis
Requirements vs. My Experience
| Requirement | Match | Evidence |
|---|---|---|
| Customer-facing technical role | Strong | 5+ years Solutions Architect at Tyler |
| Python fluency | Strong | Python scripts, Jupyter notebooks, automation |
| API/infrastructure debugging | Strong | ”Debugged issues across SDK, API, and infrastructure layers” |
| DevOps/cloud (Docker, AWS, Azure) | Good | Docker, AWS (accredited), Azure experience |
| Kubernetes, Terraform | Gap | Not explicitly listed — may need to address |
| Self-hosted/hybrid deployments | Strong | ”100+ environments: managed cloud, self-hosted, hybrid” |
| LLM/AI workflows | Strong | AI Transformation Initiative, evaluation pipelines |
| Technical communication | Strong | 120+ sessions, 4.8/5 rating |
| Startup autonomy | Strong | Led initiatives, cross-functional coordination |
| Health checks/reliability for self-hosted | Strong | ”Owned proactive health checks and reliability guidance” — verbatim match |
Keyword Alignment
Job posting language directly matched in resume:
- “trusted advisor” ✓
- “proactive health checks” ✓
- “self-hosted deployments” ✓
- “debug customer issues across layers” ✓
- “technical artifacts (scripts, templates, notebooks)” ✓
- “evaluation pipelines” ✓
- “translate customer friction into product feedback” ✓
Resume Used
Variant: Customer Solutions Architect
Key Points Emphasized
- AI Transformation Initiative (LLM evaluation pipelines, prompt testing)
- 100+ customer deployments (managed cloud, self-hosted, hybrid)
- Technical artifact creation (Python scripts, Jupyter notebooks, runbooks)
- Cross-layer debugging (SDKs, APIs, infrastructure)
- Proactive health checks and reliability guidance
- Customer enablement (120+ sessions, 4.8/5 satisfaction)
Cover Letter
Opening Hook
Connected personal experience (building AI evaluation pipelines at Tyler, realizing “you’re flying blind” without production visibility) to Braintrust’s core mission.
Key Messages
- Deployment expertise: 5 years deploying software across managed cloud, self-hosted, and hybrid
- AI hands-on: Built LLM evaluation pipelines to test prompt variations
- Customer empathy: 120+ enablement sessions, 13/14 escalations resolved
- Technical + communication: Can write Python scripts AND explain trade-offs to CTOs
- Startup ready: Comfortable with autonomy, ambiguity, fast pace
Closing
Expressed genuine interest in Braintrust’s mission (AI reliability) and readiness to contribute immediately.
Application Materials
| Document | Filename | Download |
|---|---|---|
| Resume | Jake_Lawrence_Resume_Braintrust_CSA_01202026.pdf | View Resume |
| Cover Letter | Jake_Lawrence_CoverLetter_Braintrust_01202026.pdf | View Cover Letter |
Application Package Ready — Resume and cover letter specifically tailored for Braintrust’s Customer Solutions Architect role with strong AI/LLM and deployment experience positioning.
Interview Tracking
Schedule
| Round | Date | With | Format | Duration | Notes |
|---|---|---|---|---|---|
| Phone Screen | TBD | Recruiter | Phone | 30min | Initial screen |
Preparation
Key Themes to Hit
- AI Transformation Initiative — built evaluation pipelines, tested prompts, automated workflows
- 100+ deployments across all environment types (managed, self-hosted, hybrid)
- Technical depth: Python, SDKs, APIs, infrastructure debugging
- Customer outcomes: 4.8/5 satisfaction, 13/14 escalations resolved, 87% playbook adoption
- Startup mindset: autonomy, ownership, fast iteration
Questions They Might Ask
- “Tell me about your experience with LLM evaluation workflows”
- “Describe a complex deployment issue you debugged across multiple layers”
- “How do you handle a customer struggling with self-hosted deployment?”
- “What’s your approach to building technical documentation?”
- “Tell me about a time you translated customer feedback into product improvements”
Questions to Ask Them
- “What’s the split between managed cloud and self-hosted customers?”
- “What does the typical customer deployment journey look like?”
- “How does the Field Engineering team work with Product/Engineering?”
- “What are the biggest technical challenges customers face today?”
- “What does success look like in this role at 30/60/90 days?”
Interview Notes
(To be updated after interviews)
Networking
LinkedIn Connection
- Johnny Carlin — 3rd connection, NIU alum
- Role: “Helping teams measure, evaluate, and improve AI in production”
- Action: Consider reaching out after applying
Status History
| Date | Status | Notes |
|---|---|---|
| 2025-01-20 | Applied | Initial application submitted with tailored resume and cover letter |
Outcome
Final Status: Applied
Date Closed: —
Notes:
- Strong match for AI/LLM experience and deployment background
- Startup environment aligns with desired career direction
- High-quality customer base (Notion, Stripe, Zapier)
Learnings: (To be updated after outcome)
Created: 2025-01-20 Last Updated: 2025-01-20