Janvi Kalra's Journey: From Software Engineer to AI Engineer at OpenAI & Strategic Advice for AI Career Growth

The Pragmatic Engineer

Early Career & Big Tech Internships [02:31]

• Securing Internships at Google and Microsoft [02:31]
  • Janvi applied through company portals without personal connections.
  • Essays detailing personal projects outside of classes and specific reasons for wanting to work at each company were crucial.
  • Interview preparation involved rigorous LeetCode (then "Cracking the Coding Interview") practice.

    

• Lessons from Big Tech Internships [07:11]
  • Intellectual Privilege: Gaining access to and understanding large codebases (e.g., Google's search algorithm) and internal tools (e.g., Google's Momma) was intellectually stimulating.
  • Technical Skill Development: Learned crucial professional practices like operating in large codebases and the importance of writing unit tests, skills not typically taught in college.
  • Mentorship and Career Guidance: Valued access to experienced engineers for career advice, which influenced her post-graduation decisions.
  • Optimizing Internship Experiences: In hindsight, she suggests combining big tech and startup internships for a more robust overview of different work environments.
• Pursuing Specific Interests [09:04]
  • Expressed a strong interest in operating systems for her Microsoft internship after a compelling computer architecture class.
  • Highlighting specific interests can lead to placements on desired teams, demonstrating the value of "if you don't ask, you don't get."

Strategic Startup Selection [11:35]

• Decision to Join a Startup (Coda) over Big Tech [11:35]
  • Mentors advised considering startups for diverse experiences.
  • Upsides of Big Tech: Learning to build reliable software at scale, working on "moonshot" projects without immediate revenue pressure, faster green card processing, higher average pay, and a stronger resume signal.
  • Upsides of Startups: High volume of code shipped due to more problems than people, exposure to "0 to 1" greenfield problems, broader skill development (technical like front-end, backend, infrastructure; non-technical like business insight, product management), and greater agency over work.
• Criteria for Evaluating Startups (Janvi's 4-Point Rubric) [16:58]
  • High Revenue and Steep Revenue Growth Rate: Crucial for career growth at a startup.
  • Large Market: Ensures room for expansion and long-term viability.
  • Loyal, Obsessed Customers: Indicates strong product-market fit and customer retention.
  • Competitive Advantage: Understanding why the company will succeed in its specific market niche.
• Importance of Due Diligence [18:20]
  • Emphasizes personal responsibility for vetting companies, especially for equity compensation.
  • Methods for assessing customer obsession: researching online (Reddit, YouTube), talking to actual users of B2B products.
  • Warns against accepting vague answers; engineers, as investors of their time and equity, have a right to detailed financial and strategic information.

Transitioning to AI Engineering [21:38]

• Initiating the Shift at Coda [21:38]
  • Inspired by ChatGPT's release in late 2022, Janvi sought to join Coda's new AI team but was initially declined.
  • Decided to self-learn and build AI projects in her free time to demonstrate passion and capability.
• Self-Learning Curriculum and Tools [22:16]
  • Foundational Knowledge: Started with the basics of deep learning (tokens, weights, embeddings) and the evolution of LLM architectures (RNNs to Transformers).
  • Learning by Doing: Prioritized hands-on building through hackathons (internal and external) and personal projects.
  • Public Building: Shared learnings and progress on her blog, keeping her team informed.
  • Open Source Resources: Utilized documentation from projects like LangChain for RAG (Retrieval Augmented Generation).
• Impactful Project: Coda's Workspace Q&A and Coda Brain [30:34]
  • Identified a customer pain point: difficulty finding internal knowledge within Coda documents.
  • Leveraged existing Coda assets (redone search index, LLM integration infrastructure, existing chatbot) to rapidly prototype a "Workspace Q&A" chatbot using RAG in a few days.
  • The prototype impressed Coda's CEO, Shishir Mehrotra, who envisioned a broader "enterprise search" tool, leading to the creation of "Coda Brain."
  • Part of a small, high-stakes team that developed Coda Brain, which was later showcased at Snowflake Dev Day.
  • This experience demonstrated the power of proactive learning and initiating projects without waiting for formal opportunities.

The Role of an AI Engineer [25:48]

• Core Responsibilities and Skills [25:48]
  • Primarily involves building products on top of AI models.
  • Entails significant experimentation with new tools, prototyping solutions to customer problems, and then building for production.
  • Requires domain-specific skills: fine-tuning models, prompt engineering, hosting open-source models, and especially evaluation (writing "evals").
  • The core foundation remains traditional software engineering principles.
• Challenges with Non-Deterministic Models [23:37]
  • Building reliable systems with stochastic models presents a unique challenge, as traditional deterministic testing methods are less effective.
  • Evaluation often requires using LLMs themselves to assess the accuracy of other LLMs.
• Learning by Doing: Hackathons [27:30]
  • Hackathons are invaluable for hands-on learning with rapidly evolving AI tools.
  • Examples include exploring OpenAI's function calling feature and fine-tuning models like Llama for specific use cases (e.g., converting natural language to Coda formulas).

Navigating the AI Job Market [37:40]

• Interviewing Across 46 Companies [37:40]
  • Janvi's job search spanned six months, with initial struggles in the first half while ramping up preparation.
  • The AI space is "noisy," requiring thorough due diligence to assess growth potential.
  • Advice from a mentor: only join if it's a "heck yes" and if you have savings, otherwise, don't.
• Categorizing AI Companies [38:56]
  • Product Companies: Build applications on top of existing models (e.g., Cursor, Codium, Hebia).
  • Infrastructure Companies: Create tools to help product companies effectively use LLMs (e.g., inference providers like Modal, vector databases like Pinecone, observability tools like Braintrust). Key questions for evaluation include profit margins (aim for ~70% gross margins like good software businesses) and the ease of building equivalent solutions in-house.
  • Model Companies: Form the base of the AI ecosystem by building intelligence (e.g., Google, Meta, OpenAI, Anthropic). Key questions for evaluation include ability to fund expensive model training and staying ahead of open-source alternatives.

    

• Interview Process in AI [45:39]
  • Varied, often including traditional coding (data structures and algorithms, front-end/back-end) and system design (e.g., using resources like Alex Xu's books).
  • Project interviews (building something in a day) are preferred for demonstrating passion and capability.
  • Advocates for the industry to shift more towards project-based, code-reading, and debugging interviews over pure LeetCode.

AI Engineering at OpenAI [49:08]

• Janvi's Role in AI Safety [49:08]
  • Works on ensuring AGI (Artificial General Intelligence) benefits all humanity.
  • Responsibilities include developing low-latency classifiers to detect harmful model/user behavior and measuring unknown harms of increasingly capable models in the wild.
  • Manages safety mitigation services and integrates them with product launches.
  • The team addresses "edge cases" of AI behavior and potential misuse, similar to fraud teams in payments.
• OpenAI's Unique Culture: Speed and Scale [51:01]
  • Combines the rapid iteration speed of a startup with the massive user scale of a large tech company.
  • Services handle extremely high request volumes (e.g., 60,000 requests/second).
  • Fosters an open culture where engineers are encouraged to ask "why" and "how."
  • High passion for the mission and continuous shipping characterize the work environment.
• Shipping Process and Trust in Engineers [52:30]
  • Engineers are highly trusted and empowered to drive ideas from conception to production.
  • Minimal "red tape" and streamlined deployment processes (e.g., one reviewer needed for deploys even on high-traffic services).
  • This trust comes with the expectation of high competence and product-mindedness from engineers.

Evolution of Software Engineering in the AI Era [57:50]

• Impact on Junior Engineers [57:50]
  • AI tools are not disproportionately worse for junior engineers; instead, they enable everyone to move higher up the stack and be more creative.
  • Empowers younger engineers to propose and ship more ideas.
  • Distinguishes between using AI "to learn" versus "to avoid learning." Both approaches can coexist, but deep understanding is crucial for robust system ownership and debugging.
• What Remains Constant in Software Engineering [1:02:16]
  • Code remains the primary manifestation of innovation.
  • The need to understand and design high-level systems, debug effectively, and read code proficiently persists.
• The Blurring of Roles: Product, Data, Full-Stack [1:02:40]
  • AI blurs traditional boundaries between PMs, designers, data scientists, and engineers.
  • Engineers are increasingly expected to handle product work, design considerations, and data pipelines (becoming more "full-stack").
  • The most efficient engineers are those who can "zoom out" to articulate architectures and write effective prompts for models, and "zoom in" to meticulously review generated code for bugs and edge cases.

    

    

• Embracing Change and Continuous Learning [55:41]
  • A surprising reality of AI engineering is the constant need to "scrap" and rebuild solutions as models rapidly improve or new capabilities emerge.
  • Engineers must be comfortable with their work becoming obsolete quickly, viewing it as continuous learning rather than waste.

    

  • The ease and lower cost of code production with AI leads to frequent "expansion and collapse" phases in feature development.
• AI Stack for Coding: Cursor for hackathons, deep research tools, ChatGPT (for search and tutoring), internal RAG tools for documentation.
• Book Recommendations:
  • "The Almanac of Naval Ravikant" for pragmatic career and life advice.

    

  • "The Mythical Man-Month" for insights into software engineering project management.

    

  • "Software Architecture: Perspectives on an Emerging Discipline" for foundational software architecture.

    

• Key Professional Advice: "Don't wait for someone to give you the opportunity to do something; go work on it yourself."