The Evolving Role of Software Testers in the Age of AI and Automation: Challenges, Value Communication, and Future Skills

Richard Seidl

Challenges in Modern Software Testing [00:00]

• Historical Struggles with Acceptance [00:03:57]
  • Software testing has historically struggled with industry acceptance, often being seen as expendable.
  • Past technological hypes, like the rise of Selenium for test automation around 2010-2011, led to the belief that human testers would become obsolete.
  • The current AI revolution presents a similar challenge, with renewed discussions about the necessity of testers.
• The Communication Gap [00:04:28]
  • Testers excel at identifying and communicating problems and issues within products.
  • However, they often fall short in articulating the overall business value and importance of their craft and quality measures.
• Quality as a First Target for Cuts [00:05:39]
  • During economic contractions or when budgets are tight, managers and companies frequently choose to cut quality assurance and testing efforts first.
  • This tendency overlooks the long-term repercussions of neglected quality.

Developer Responsibilities and Future Demand [00:06:05]

• Increasing Developer Burden [00:06:12]
  • Many companies are placing a growing number of responsibilities on developers, encompassing not just coding and basic testing, but also pipeline maintenance and production support.
• Long-Term Risks of Rapid AI/Low-Code Adoption [00:06:42]
  • While AI and low-code tools offer quick app generation, leading to fast product launches, this speed often comes at the cost of neglecting fundamental system architecture.
  • In the next 5-10 years, there is an anticipated huge demand for professionals who possess a deep understanding of system architectures to manage and fix issues arising from quickly generated, poorly architected products.
  • This highlights a potential future need for quality advocates, testers, and test architects to ensure robust systems.

Neglected Non-Functional Requirements [00:09:09]

• Architectural Impact on NFRs [00:08:09]
  • When applications are generated rapidly by AI without a solid architectural foundation, non-functional requirements (NFRs) like performance and security are often compromised.
  • Remedying performance or security flaws rooted in architecture post-release is exceptionally costly and time-consuming.
• Lack of Product Manager Awareness [00:09:07]
  • Product managers frequently overlook the importance of NFRs during the planning and development phases.
  • While accessibility is gaining some attention, vital NFRs such as load and performance testing are still commonly neglected.
  • Security testing often relies on external experts, indicating an internal gap in proactive consideration.

Tech Discussions: Architecture & KPIs [00:11:40]

• Tailoring Communication Strategy [00:10:21]
  • The key to effectively communicating testing's value is to adapt the message to the audience's background (e.g., business, technical, marketing).
  • For business stakeholders, focus on connecting testing activities directly to business outcomes like increased revenue, improved user satisfaction, or achieving specific KPIs.
  • For technical audiences, delve into architecture maps, identify weak spots, and propose technical testing activities such as API testing, contract testing, or specific non-functional tests.
• Focusing on Business Metrics [00:12:11]
  • Avoid technical metrics like "code coverage percentages" when speaking to non-technical management, as these often lack clear business meaning for them.
  • Instead, frame testing contributions in terms of their impact on tangible business results like customer satisfaction or revenue targets.

Evolving Role of Tech Skills [00:15:02]

• Domain Knowledge Over Deep Technical Skills [00:15:02]
  • While deep technical skills and programming knowledge were previously crucial, the advent of AI and low-code generating significant portions of code shifts the focus.
  • Testers will increasingly need strong domain knowledge to critically evaluate the output of AI and ensure it aligns with business needs and requirements.
• Importance of Foundational Technical Understanding [00:16:22]
  • A basic understanding of system mechanics, such as request-response cycles, client-server communication, and modern technology stacks, remains beneficial.
  • This foundational knowledge helps testers formulate insightful questions, generate creative testing ideas, and distinguish between human-written and AI-generated code components.
  • Emphasizing "out-of-the-box" thinking and edge-case testing will be particularly important as AI handles more standard coding.

Prompt Review in AI Development [00:17:31]

• A New Shift-Left Testing Practice [00:17:31]
  • A novel approach to quality assurance in AI development is to conduct "prompt reviews."
  • This involves performing quality checks on the prompts that are used to generate AI code.
  • Slight alterations in prompt wording can lead to significantly different AI outputs, making prompt quality critical.
  • Prompt reviews can serve as a new form of "shift-left" testing, addressing potential issues at the earliest stage of AI-driven development.

Explore and Test New Tools [00:21:50]

• Criteria for Effective Tools [00:19:40]
  • Testing tools, especially AI-driven ones, should be user-friendly, easy to install, and integrate seamlessly into existing workflows.
  • Avoid tools that demand excessive configuration, complex dependency management, or large downloads, as these impede efficiency.
• Benefits of AI-Driven Tools [00:20:20]
  • Promising AI tools offer features like:
    • Automated test case generation from requirements and designs, directly integrating with ticketing systems.
    • Background tracking of customer behavior to identify common user journeys and suggest test automation scenarios.
    • Prompt-based interfaces to generate test automation scripts or configurations.
• Hands-On Evaluation is Crucial [00:21:50]
  • Testers should actively experiment with and try out as many new tools as possible.
  • This practical experience helps in understanding what truly works, discerning real value from marketing hype, and selecting the best tool for specific company tech stacks and situations.

Embrace AI Revolution Now! [00:24:35]

• Continuous Learning and Adaptation [00:24:36]
  • It is imperative for testers to engage deeply with AI by learning about prompting techniques, the internal workings of Large Language Models (LLMs), vector databases, and various AI principles.
  • The current AI revolution is accelerating at a much faster pace than previous technological shifts, such as the mobile revolution.
• Reinforcing Fundamental Testing Skills [00:25:55]
  • Despite the rapid advancements in AI, fundamental testing skills remain valid and essential.
  • Testers should confidently adapt and apply their core knowledge, rather than believing their skills are becoming irrelevant.

Guest Collaboration & Future Topics [00:26:52]

• Both the host, Richard Seidl, and guest, Daniel Knott, expressed enthusiasm for continued collaboration and future discussions on various topics pertaining to software quality and the evolving tech landscape.