Implications for Human Analysts from Generative AI

Jul 10, 2023 | PMA

By Graham Goodwiler

Summary

There’s been a lot of talk about generative AI with the likes of OpenAI’s ChatGPT and Google’s Bard being the frontrunner platforms. With this new technology, it can take a simple prompt or complex question and treat it with nuance to give you a detailed response using multiple sources across the internet. At its simplest, it’s like an auto-generated Wikipedia page for anything you’d want to know. But it can do much more than that from planning out your vacation itinerary to creating websites to giving you nutrition advice based on your dietary preferences. The only limit really is our creativity. That being said, when it can research, design presentations, and give you the key takeaways all within a couple of minutes, it’s understandable that some analysts might be fearful of generative AI taking their job. In this blog post, I’ll talk about how generative AI will impact the role of the analyst and what it might mean for human analysts in the future. 

In this article we’ll cover:

  • The Rise of Asymmetric Insights
  • Use Cases for Analysts
  • Advanced Methods and Tips for Success 
  • Recommended Resources

 

The Rise of Asymmetric Insights

As an analyst, if all you do is ad hoc secondary research where you compile some sources to give a summary of what’s happening, then you are much more likely to be replaced by generative AI. However, there are some things you can focus on to increase your value.

  1. Primary research – By including primary research into your analytic process, you can glean information that isn’t available online, accessible to generative AI, or likely your competitors either. These insights can provide extraordinary value due to their richness and precision to the key questions you have. You can get your information directly from the original source or other experts to get insights no one else has. 
  2. Forecasting – The ability to incorporate defensible forecasts through analytic rigor can also separate your work from generative AI and build your credibility with your intelligence consumers. If you ask generative AI for future forecasts, it will either repeat what others have already said, or at best, extrapolate prior data points. Estimative analysis should involve a mix of historic patterns, current trends and future potentialities. An excellent analyst can take all of this into account to forecast the likelihood of different scenarios, their implications, and what it means for their decision makers. 
  3. Relationships & Tailoring – Most importantly, no matter how well you understand the generative AI algorithm, it will still be a bit of a black box and you have to trust the intentions and coding that the developers had/have when building it. You, on the other hand, can be a trusted source of information based on your human interaction with your stakeholders, your credibility of years of strong analysis, and your willingness to work with them to achieve their intended outcomes. Your ability to build trust, answer follow-up questions, and tailor your deliverables to their needs will be even more valued in an age of generative AI. 

 

Use Cases for Analysts

The above capabilities will continue to be strong differentiators for human analysts in the world of generative AI. As I said earlier, if all you do is answer simple questions using online sources, then it is likely to take your job. However, if you’re like the rest of us with complex asks and large deliverables, you should treat it as a force multiplier. Generative AI will change the role of human analysis, but in a positive way, I think. When you think about the intelligence cycle, the most time consuming aspect is the collection piece but the most valuable aspect is the analysis. Bard and ChatGPT enable us more time to do high-quality analysis and better tailor our deliverables, while reducing the upfront time of collection. Here are some examples of how it can do that: 

  1. Information retrieval – Ask it to explain complex topics or provide overviews of subjects related to your research. It can provide summaries or detailed information on a wide range of topics, based on the data it was trained on. Using a model with access to current information, it can provide you with recent news, including sources and summaries. 
  2. Brainstorming ideas – If you’re seeking fresh perspectives or ideas, you can ask it to brainstorm with you. This could be useful for developing new research questions, hypotheses, or methods. These models are much more flexible with search than a simple Google search so it can take in the context of what you’re trying to say instead of needing to be super explicit in your wording. 
  3. Identifying trends – For qualitative data, you could ask it to suggest possible themes or patterns that might be worth exploring. You can also ask it about emerging trends in different fields or provide an overview of recent developments pertaining to a trend. Again, with this fuzzy search, you don’t need every key word available to track something, you just need to list out the concept you want to know about and it find the associated terms that go with it. 
  4. Writing assistance – It can help with writing tasks, such as drafting emails, reports, or presentations. You could also use it to generate outlines for your research papers or to suggest ways to phrase or structure your arguments. As an example, I like to use it to get a head start on something I want to write about, so it will write up a structure similar to what I want then I can edit and add-in further information but that initial draft saves me a lot of time. 
  5. Competitor overviews – While ChatGPT is trained up to September 2021 (if it doesn’t have access to the internet), it can still help outline strategies for analyzing competitors or suggest potential areas to explore based on typical business practices. A quick summary of a business or what its products are is a great first step to doing a more in-depth competitor analysis, particularly if you can also incorporate primary research, forecasting, and tailoring it to your interests. 
  6. Learning and development – ChatGPT and Bard can be useful tools for learning new topics related to your research or work. You can ask it to explain concepts, provide examples, or generate quiz questions for you to test your knowledge. Instead of going to YouTube to learn about a new topic, maybe try ChatGPT first? 

 

Advanced Methods and Tips for Success 

After going through these use cases, you may try one of these generative AI models and not have much success or not be sure what to say. You likely know how to use Google effectively but this has a different search algorithm so it won’t work the same. But just like there are advanced Googling tips and tricks, there are also advanced tips to using generative AI. Here are a few tricks I’ve learned so far: 

  1. Give it a persona – While asking questions, give it a perspective or lens to view the information. Something like “as a market intelligence analyst for a small CPG company, what would I need to know about…” is a much better prompt than saying “tell me about …” because it doesn’t know who you are or what you care about. 
  2. Be explicit – If you need specific information or a certain type of response, it’s often helpful to be explicit about that in your prompt. For example, you might say, “Can you explain the inflation reduction act in simple, non-technical language?” or “Can you write an introductory paragraph for a blog about generative AI and human analysts?”
  3. Add context – If you’re asking a follow-up question or want the model to continue a certain line of thought, it can be helpful to include some of the context from your prior questions and answers in your prompt to get it to connect the dots.
  4. Try prompt engineering –  If the output isn’t what you expect, try rephrasing the prompt or asking the question in a different way. You can also experiment with the tone, formality, and length of your prompts.
  5. Use system messages – System messages are a more advanced technique where you communicate your instructions as a system message rather than a user prompt. For example, instead of saying “Translate the following English text to French:…”, you can say “System: Your task is to translate English text to French. User:…” This can help when you have multiple follow-up messages or tasks to do and don’t want to repeat the same prompt each time. 
  6. Point out mistakes – If the model makes a mistake, you can explicitly point out the error and ask the model to correct it. For example, “You mentioned that Paris is the capital of Italy, which is incorrect. Please provide the correct information.” The system is still learning and just sees what’s available on the internet so there’s a lot of misinformation out there that it could pick up. If you tell it what’s wrong, then it will adjust and improve the subsequent responses. 
  7. Check the settings – Depending on the application you are using to interact with the large language model, you might have various settings available to adjust the behavior of the model, like the temperature (which controls randomness) and the max tokens (which controls the length of the response).
  8. Refine the output – Just like how we usually aren’t perfect the first time, the output likely won’t be either. So continue to give it feedback so it will take its own output and refine it over several iterations. This can be used to generate more thoughtful and polished output.
  9. Be curious – As these models continue to improve over time and more plug-ins, features, and applications become available, try them out and judge for yourself what works and what doesn’t at its current state. By playing around with it, even just to see how it would approach something you’ve already done, you’ll learn more and more tips, tricks, and use cases. 

 

Recommended Resources

 

Conclusion

Many analysts may be, and perhaps some should be, concerned about their role’s existence in the age of generative AI. It will change how analysts do their work. There will likely be fewer simple information gathering exercises but there will be a greater opportunity to be more efficient and have more insightful analyses. Generative AI may get you 80% of the way there but it’s always that last 20% that provides 80% of the value. Models like ChatGPT and Bard will weed out the poor analysts and make the best analysts even better. There will always be a need for human analysts but we need to grow along with it.