How to use GPT-4 to prepare for a PM Interview

+ how Product Managers need to prepare for the AI revolution

The AI bullet train seems to be showing no signs of stopping.

Just a week after GPT-4 dropped, major tech brands went on an announcement spree. The AI wars have truly begun. And we’re still in the opening scene.

5 notable announcements:

• Google Bard: Google’s direct response to ChatGPT. (sub-par in terms of results)

• ChatGPT Plugins: ChatGPT’s launches its App Store + it can now search the web.

• Github Copilot: Think AI-guided code & pull requests that’ll slash dev time.

• Canva Create: Create presentations, generate designs & edit images with a click.

• Microsoft Loop: An AI-juiced app for notes & productivity that’ll give the folks at Notion some sleepless nights.

In this edition of Behind Product Lines, I’d like to delve into how Product Managers need to think about the AI revolution.

The agenda today:

• Will AI replace product managers?

• Why Product Managers need to prepare an AI strategy

• How to use GPT-4 to prepare for PM interviews at Big Tech

---

Will AI replace product managers?

As exciting as AI is, it has sent a shiver down the spine of many professionals.

Skills that were acquired, honed and polished over decades (like written communication, critical thinking & cost-benefit analysis) are in danger of being casually commoditized with the AI revolution.

On one hand, AI seems like an empowerment tool.

But on the other, it also sounds like an existential threat.

Image credit: Serene Hour

But is it?

Let’s break it down. Every job — not just product management — constitutes of three broad components:

• Human skills e.g. emotional intelligence, communication, negotiation etc.

• Domain knowledge e.g. business history, industry insights, intricacies, trends etc.

• Technical faculty

  • digital skills e.g. wireframing, coding, SEO, animation etc. OR

  • physical skills e.g. mechanical repair, plumbing, surgeries.

Tools like ChatGPT are able to create ripples in all three but in some more than others.

• They do a decent job in written communication - from emails to slides to memos. They can’t drive human relationships though…yet.

• They are well equipped to handle technical queries when they pertain to the digital domain e.g. ChatGPT is a beast when it comes to coding. Similarly, tools like Galileo will simplify interface design. However, when physical intervention is required, AI tools won’t be able to help without an appropriate interface.

• They cover a large surface area of public domain knowledge. However, AI will need training data to get up to speed on undocumented history of a business, niche topics & domain-specific acumen accumulated via experience.

Image credit: Tidio

Now, you must be thinking.

“Doesn’t every product role require all three?”

Reality: Product jobs live on a spectrum.

Every company defines the boundaries of PMs differently, so the depth of the three levers will vary. PMs will have to honestly introspect where they lie on the spectrum.

Based on that, there will be a few potential outcomes as AI becomes mainstream:

• the product job might get completely replaced by AI

• it might not be replaced but the number of PM resources required will drop

• the product role will evolve allowing it to focus on other aspects of the job or new opportunities

• the role will not be affected much at all.

I have to point out something here.

Many people forget that even with the inclusion of AI, someone will need to drive the AI product. Tasks won’t get done by themselves. Someone has to decide what needs to be done, feed in a prompt, refine it with additional parameters and then review that output. Then, that output would need to be discussed, shared, approved & executed on. In the initial days, AI won’t be self-sufficient.

So, what product jobs will fall in each category?

1. Job Replacement by AI

   AI has the potential to replace basic PM roles that are heavily focused on transactional skills (and less on human interaction) like data analysis, JIRA ticket creation or similar repetitive tasks.
   

   Ex: Advanced AI tools will be able outperform PMs on tasks like analyzing large data sets, conducting sentiment analysis on customer feedback, culling out anomalies & even suggesting remedial actions.
   

   Thus, if a Product owner focuses purely on such technical endeavors, they will have to diversify quickly.
   

2. Reduced PM Resources

   Product jobs that require technical faculty but also require business history & domain knowledge will evolve. Such PMs will learn to push off tasks like backlog management, documentation & perhaps even certain tactical executions (e.g. framing an A/B test) to an AI tool.
   

   Instead, they will focus on making larger decisions based on domain-specific insights and the history of the product & business — training data for which likely be sparse. Ex: a PM on an agri-tech product working on emerging markets where data hasn’t been digitized.
   

   The by-product of this might be a reduction of product folks needed to pull off the same amount of work.
   

3. The Safe(r) Zone
   Job roles that require a strong balance of all three levers: human, technical and domain knowledge will remain largely protected.
   
   They will still push off grunt work to AI products but that will allow their role to lean more towards focusing on customer insights, cross-functional alignment & strategic innovation.
   

   Such roles are typically heavy on stakeholder management and demand communication & negotiation skills that are a cut above what AI can generate today.

Here’s what Shreyas Doshi, a thought leader in product, has to say about this:

One thing’s for sure. Regardless of where you lie on the job spectrum, Product Managers have to start embracing AI in their daily operations and quickly figure out how it can empower them to do more.

This means studying prompt design, uncovering viable use cases & keeping a look out on AI tools that are likely to cut down time-to-market.

Marty Cagan, a veteran in product management, alludes to this in his blog post:

I hope you are spending time with your product teams learning about these new enabling technologies, creating some early prototypes, exploring the implications and the opportunities.

Every disruptive technology has certain common traits, but one thing about this current wave is that it will likely not just impact our products, but it will also likely impact each of the roles on a product team.  

The most obvious impact is on engineers (GitHub Copilot is an early example), but I expect impacts on design and product management as well.

While I hesitate to make predictions on something so disruptive so soon, I do believe that the coming years will shine a very strong light on product management.

Remember that product managers are responsible for the value and viability of what we build, and those risks are right at the center of this new wave of disruptive technologies.  The ethical implications alone can keep a strong PM busy.

The tricky part to all of this? Preventing our critical thinking to get rusty due to over-reliance on AI.

In summary, if your product job involves customer relationships, creative thinking & persuasive communication across a stakeholder matrix, AI will not be replacing you. However, you need to learn to wield it’s power as the benchmarks of productivity & success will soon be revised.

---

How Product Managers need to prepare for AI

ChatGPT just launched plugins. The gold rush is about to begin. And Product Managers need to pay attention.

Plugins is being coined as ChatGPT's App Store moment.

It'll enable ChatGPT to overcome the biggest complaint naysayers have been painting it with: inability to access up-to-date information.

Plugins will allow ChatGPT to tap into the internet for data beyond the Sept 2021 cutoff. It'll also help it access third-party services like OpenTable, Expedia, Instacart & many more apps.

Imagine using a single interface to:
- Shop for products.
- Book your plane tickets.
- Order groceries.
- Tabulate competitor insights.
- Craft an email & send it .

While I doubt that all apps will converge to ChatGPT, the coverage will eventually be significant.

What does this mean for Product Managers?

Fact: ChatGPT has over 100 Million DAUs+. As internet real estate, that's massive visibility.

Of course, it'd be naïve to say it'll topple Google overnight. But AI is placing a real dent on "10 blue links". And PMs & growth teams need to plan for it.

That mileage is the driving factor behind a multitude of apps actively building a ChatGPT plugin.

If not already done, it's time to frame your product's AI strategy.

Not to dial up FOMO here, but it's likely that your competitors are already on the plugin API waitlist.

So, what should Product Managers do?

Start by learning a bit more about large language models. Educate yourself on what all of this means.

• Video: a whiteboard reel on ChatGPT behind the scenes

• Article: what large language models are & how they are trained

• Video: how ChatGPT is trained

• Article: learn the plugin flow for ChatGPT

Then, work on some of these areas:

💬 Talk to customers about AI & ChatGPT. 
Start openly talking to them about their relationship with ChatGPT. Are they actively using it? What kind of problems are they solving with it? What's frustrating them? Are they getting approached by providers?

🔬 Research your target audience sentiment.
Use your LinkedIn/Facebook channels & your email list to run polls/surveys on how prospects are using ChatGPT in their daily lives. Is your audience bought on? Are they skeptical? If so, why? What use cases are exciting them the most?

🔝 Bring it up with leadership.
Sync up with the higher-ups to get a temperature check. What value do they see? What areas do they deem more important to enhance with AI than others? What's their reaction to results from 1 & 2?

💡 Brainstorm with the product & growth team.
While ideating on AI is essential at the moment, don’t also fall into the bandwagon effect and build because “you have to”. Focus on customer problems you can solve. What use cases do ChatGPT plugins enable that intersect with our product & customer needs? What experiments can we run to test our hypothesis?

👨‍🔬 Allocate some RnD time with dev.
If point 1 to point 4 deem AI as in integral area, work with devs to explore technical feasibility. First, get on the plugin API waitlist to get access. Study other implementations. Pin 1-2 use cases & collaborate to build a cohesive experience.

⚠ Plan for limitations.
- OpenAI's APIs aren't known for scale yet. Speed is flaky.
- Security is a concern. Sam Altman recently admitted to a data privacy bug in the open source library.
- With fast execution comes a bundle of security loopholes. Bounty hunters might have some field days.

👀 Keep an eye out for competition.
First-movers will make a lot of noise about their achievement. It already happened in the industry I operate in (events). A segment of your customers will notice & start discussing your roadmap. Keep tabs on their movement.

📢 Engage product marketing early.
Ideate on how this will potentially play in positioning & messaging. Study how other products have injecting AI in their literature.

ChatGPT's plugins warrants a conversation internally.

Even if you decide to place bets elsewhere, the team needs to be clear on why.

Is your leadership not convinced that they need to respond? This post from Sarah Guo provides a great breakdown of why every company needs an AI strategy.

---

How to use GPT-4 to prepare for PM Interviews

GPT-4 is clearly a step above it’s predecessors.

When I learned that it passed a number of standardized tests & exams, I figured it had superior simulation superpowers.

Thus, it made me wonder: can it also provide better interview guidance than GPT-3?

But before I proceed into some of the interesting stuff, a word of caution.

Is leveraging GPT-4 to prepare for PM interviews even a good idea?

Here’s why it’s probably a BAD idea:

• Every company has a different definition of it’s product role. The scope of work, expectations & required skills vary on a spectrum. GPT-4 will likely not be privy to every company’s flavor of PM and thus, it’s guidance needs to be taken with a grain of salt.

• GPT-4 does a good job in giving blueprints and a list of ideas/avenues to explore when answering an interview question. However, an astute interviewer will easily read through templated answers. They may change the parameters to push you to see how you think on your feet. Thus, combining GPT-4’s magical powers with your own experience & learning is essential to make a power combo.

• Do NOT attempt to answer a take-home assessment by copy/pasting GPT-4 responses. When it comes to specific domains, it’s easy to figure out what responses look generated by an AI. You may use GPT-4 as an aid (ONLY if it’s allowed).

Therefore, while GPT-4 can assist in preparation, don’t let it be the only source to get you ready for your interview.

Now that the disclaimer is out of the way, let’s get straight into it.

First of all, the wrong way of approaching GPT-4 for anything is to directly feed it a simple prompt like “How would you design an alarm clock for the blind?”

No. You have to create context for it to be more useful.

What context can you give? Well, you should have 2 artefacts at your disposal by the time you get an interview: the job description & your resume.

Prompt:

I'm applying for a Product Manager job at [company]. I'm going to set the context for you by sharing the job description and company profile first:
[Company Profile + JD]

Here’s how ChatGPT responds (I’m using a job at Stripe as an example):

Prompt:

Now, I want you to read my resume for further context.
[RESUME TEXT]

Response:

Great. Now, it knows what you’re up against.

Next, warm up your game by understanding a bit more about the company. GPT-4 can read between the lines on the JD and share a summary. It works better when the company is well known, of course.

Prompt:

Summarize the problem [company] solves, the main product goals and
their closest competitors.

Response:

Alright. I skimmed through the JD and I spotted some technical jargon that I’m not too familiar with. That makes me nervous. I need to cover that base.

Prompt:

Briefly explain what I need to know about [topic] in
very simple terms in the context of this job.

Response:

Alternatively, I can also ask: “What is some technical jargon that I need to be comfortable with based on the industry of the company and the job description?”

Alright then.

Let’s start ideating on questions & answer strategies. You can further qualify the prompt by asking for “include specific examples of a response”.

Prompt:

Share 7 questions I can expect in the Product Manager interview at [company name] based on the job and give me pointers on how to answer each.

Response:

Alternatively, you can head to Glassdoor.com or Exponent to fish out past questions asked at the company.

Or get an idea from the HR manager or network contacts in LinkedIn who have taken the interview in the past.

Ex: here’s a question I found on Glassdoor on Stripe: ”One of our merchants is noticing an increase in fraud. How would you solve the problem?"

Next, let’s get some help on the questions related to soft-skills and behavioral aspects. Many experts recommend reading Jackie Bavaro’s book called “Cracking the PM Interview” for answering such questions.

How about we summon the book’s insights directly into GPT-4?

Prompt :

Based on Jackie Bavaro’s book “Cracking the PM Interview”, share 5 questions and her recommended answer strategies for each.

Response:

If you find different set of behavioral questions on Glassdoor, simply add that in and ask GPT-4 to answer it in light of Jackie’s lessons.

Now, let’s take it up a notch.

Interviews, sometimes, require you to come to a whiteboard to ideate. If you have no clue on how to approach those, here’s a prompt that can help:

Assume I am asked to come to the whiteboard to answer a design question like this: [TASK]. What should I do on the whiteboard?

Response:

OK. Play time is over.

Let’s turn GPT-4 into a mock interviewer.

Woah. How?

Behold Prompt:

I want you to stage a mockup interview for this role with me. It will work like us: Ask me a question. I will type in an answer. Then, based on my answer, ask me 1 follow up question. After I answer that, give me feedback and then ask me the next question. Limit to 5 questions. Mix your questions up and relate them to one of these categories [product design, behavioral, analytical, estimation, product sense, situational questions, roadmapping, prioritization, strategy questions, metrics, stakeholder management].

Here’s GPT-4 with the first question along with my response:

Next, here’s GPT-4’s follow up question to my answer:

Here’s ChatGPT’s feedback and question 2:

This is a really powerful unlock.

This helps you get a sense of how your answers can be put under the microscope. If you want GPT-4 to get tough on you, adapt the prompt to add qualifiers like “give critical feedback” or “highlight weak areas in my response”.

Of course, typing out a response to each question can be time consuming. After all, we speak at interviews rather than type. You can use Descript or Podcastle to record a vocal answer, transcribe it and paste it back into ChatGPT.

Finally, need to look smart when the interviewer opens for questions?

Prompt:

Share a list of questions I should ask the interviewer with regards to this job. Come up with non-generic, insightful questions specific to the position at hand.

Response:

Hope these tips help you in polishing up your prep for your next PM interview.

Good luck!