For the past year and a half, the number one thing on anyone’s mind in the technical space has been AI. The functionality of tools like Chat GPT are unquestionable, and productivity has never been higher. However, as we begin to incorporate AI into almost every aspect of our work, it is important to examine the effects of utilizing it, not just on our productivity, but on the quality and quantity of our work.
A recent real world experiment led by Kian Gohar, CEO of GeoLab, and Jeremy Utley from Stanford University put this question into real life scenarios. The purpose of their study? To measure the impact of AI assistance on the quality and quantity of ideas generated in a structured creative task.
Their insights provided a framework to follow as brands work towards integrating AI into their problem solving tool kit. At Transparent Partners, we are eager to help your business harness AI’s capabilities to foster innovation and creativity.
Experiment Overview
The study brought together employees from four distinct companies—two based in Europe and two in the United States. These employees, up to 60 from each company, were divided into smaller teams. Then, they were asked to develop innovative solutions to pressing business problems unique to their respective organizations. The issues ranged in complexity, from creating internal training resources to scaling up B2B sales for specific products. The experiment was then divided into two groups. The first was a control group that tackled the problems unaided. The second group was an experimental group that received assistance from an open-source version of ChatGPT.
Before beginning the task, all participants were briefed. This included a short presentation about their specific challenge and additional detailed information sheets to ensure a common understanding across teams. From there, they had 90 minutes to ideate and brainstorm a set of potential solutions. These solutions were then anonymously evaluated by the problem “owners” within each company, ensuring an unbiased assessment of the ideas.
Findings
The experiment yielded some unexpected results that challenge common assumptions about the efficacy of AI in enhancing creativity and problem-solving. Contrary to what many might expect, the introduction of AI assistance did not lead to a dramatic increase in the quantity or quality of ideas generated by the teams.
The teams equipped with AI assistance produced, on average, 8% more ideas than those who worked without. More notably, the quality distribution of the ideas were varied and unexpected. The AI-assisted teams coasted somewhere in the middle, meaning they didn’t have the lowest grade, or the highest. In fact they saw a reduction in the number of top-grade ‘A’ ideas by 2%. This suggests that while generative AI can help avoid the worst ideas, it might also lead teams towards more mediocre and ‘safe’ ideas, limiting the emergence of truly innovative solutions.
These results make the disparity in confidence levels between teams even more intriguing. Those using ChatGPT reported a significant boost of 21% in confidence in their problem-solving abilities. This heightened sense of assurance, however, was not mirrored in the actual grades assigned to their ideas, indicating a potential discrepancy between perceived and actual effectiveness when using AI in creative endeavors.
These findings upend the expected narrative that AI’s vast informational and computational capabilities would automatically translate into superior creative output. Instead, it appears that while AI can assist in navigating away from unfeasible ideas, it might also inadvertently steer teams towards more conventional solutions, reducing the likelihood of groundbreaking innovation.
AI Integration
The experiment sheds light on crucial aspects of how to integrate AI within businesses’ workflows to ensure it enhances innovation, and does not stifle creativity. The Harvard Business Review recently featured an article focusing on the experiment, and it outlined significant insights on integrating AI. Here at Transparent, we are incorporating these approaches to leverage AI in a way that fosters innovation without disrupting our creative processes.
Precise problem statements – The study recommends to craft highly specific problem statements for AI tools to generate beyond-average solutions and avoid lackluster results from general inquiries. In the realm of generative AI, the clarity and specificity of the question directly influence the quality of the solution. A vague question like “How can we improve our marketing budget allocation” is likely to yield generic answers. Instead, formulating highly detailed queries such as “What changes to our reporting will improve marketing spend by 10%?” helps generate more specific answers.
Transparent has instituted an ongoing training program focused on enhancing our team’s skills in AI prompt writing. These sessions ensure the team can effectively craft prompts, while keeping pace with the latest advancements and best practices in AI technology. We encourage team members to share tips on what works when using a chatbot, and what prompts they have found most success with. Examples include, telling the chatbot to ask questions if it needs further clarification in order to provide the most optimal solution.
Relevant Data Training – Input the contextual data to ensure the AI tool is working from the most updated information. Generative AI systems lack the deep contextual understanding that human team members acquire over time. To bridge this gap, we recommend inputting relevant data into the AI tool before leveraging it so the AI is able to generate more targeted solutions.
Transparent has developed and regularly utilizes custom GPT models tailored to meet diverse use cases. This allows us to input all contextual data into the GPT’s knowledge base so we can get the most applicable solution to the issue we are trying to solve. This also ensures that the output is viable and relevant for our use, without as much additional prompting.
Interactive Dialogue – Engage ChatGPT as a dynamic conversation partner rather than a one-time advisor to lead to more innovative solutions in creative tasks. It’s vital to engage in an iterative process with the AI, refining queries based on initial responses to uncover more nuanced solutions to your issues.
In Transparent’s AI training sessions, we emphasize treating chatbots like ChatGPT, Claude, Perplexity, and Gemini as conversational partners rather than mere search tools. By engaging these AIs in ongoing dialogues, they gather contextual insights that enhance their ability to deliver tailored solutions. Our team adopts a dynamic test-and-learn approach, akin to refining a campaign strategy. This involves exchanging prompts and refining queries through iterative discussions to achieve the most effective outcomes.
Individual Ideation – Allocate 15-30 minutes for individual brainstorming prior to AI interaction to avoid groupthink and enhance the diversity of ideas in team discussions. This step is instrumental in capturing unique insights each team member brings to the table, which are then enriched through AI-assisted refinement.
As we continue to grow in our AI knowledge and expertise, this step becomes the most important. At Transparent, we prioritize leveraging our own knowledge and expertise before incorporating AI. Recognized as a valuable tool for boosting ideation and efficiency, AI is seen as a complement, not a replacement, for our intellectual efforts. By emphasizing personal expertise, we ensure that AI is used as an aid to enhance our work, rather than a substitute.
Leverage External Facilitation – Consider bringing in an AI expert external facilitator to oversee the final decision-making process. This individual can help synthesize the AI-generated ideas with team suggestions, ensuring that the final marketing strategies are both innovative and aligned with business goals.
Conclusion
At Transparent, we regularly meet with AI experts and thought leaders to enhance our understanding of how to optimally use AI and fully recognize its benefits. We strive to guide brands through their exploration and utilization of generative AI because of our commitment to remaining informed. This helps us act as external facilitators within our own organization and ensure that our outputs are in line with our business outlook and that we are using AI thoughtfully and responsibly.
Incorporating AI into knowledge workers processes is an exciting step in idea innovation, however this should not be taken lightly. AI can give workers a false sense of confidence in their outputs, and stifle truly groundbreaking ideas. By adopting guidelines and steps to correctly utilize AI, businesses can harness the full potential of their workers, and get the results that they can be confident about.
As AI grows more complex and its use becomes more prevalent in the digital era, the importance of improved understanding, strategic planning, effective implementation, and careful data governance becomes crucial. At Transparent, we are expertly equipped to guide your brand through these critical phases of AI integration.
Contact us today to unlock the transformative power of AI for your organization and lead the charge in technological innovation.
