6 Honest Lessons From 3+ Years of AI

My first recorded conversation with ChatGPT was March 31st, 2023, with good old GPT-3.5. It was four months after ChatGPT launched, and at the time it felt like magic. Every release since has made the one before it feel somewhat primitive. Three years and several existential crises later, here's what I've actually learnt from using AI on a daily basis.

My first recorded AI chat was "how do I perform a bit scan forward in C", but I'm pretty sure this wasn't the first. I seemed to think it was pretty important to delete my old chats to keep everything organised back when I first started. That didn't last long.

1. AI is a Massive Skill Leveller

AI lacks the nuance and wisdom of a human expert. I lack the domain knowledge to know where to start. Together, we can do a pretty decent job of it. It might not be perfect, but it is enough to get you started, and you can learn the rest as you go.

AI can turn you into a programmer, mathematician, marketer, scientist, engineer, copywriter, designer, analyst, or really anything you set your mind to. Personally, I've used AI to help me with SEO and marketing, web design and development, to improve my legal and financial literacy when starting my business, and I've learnt to use a number of different programs and tools in the process.

Don't get me wrong, I'm far from being an expert in any of these. But more and more, I'm starting to believe that I don't really need to be. I can be far more effective with a broader skillset than a narrower but deeper one.

Either way, there isn't much that you can't learn with it, and as a bonus, it won't judge you for asking stupid questions.

2. Use it for Prototyping and Mock-ups

I wouldn't trust AI to write high-stakes user-facing code, implement security features, or handle sensitive user data securely. Not because I don't think it can, but because I don't have the knowledge or experience to validate what it's doing and to know that it's not missing anything important. In fact, the more I use AI, the less I am inclined to trust it to do something that I couldn't do myself.

But what I do find it useful for is putting together simple tools where the stakes are much lower. I've built some tools to help manage my website, scripts to scrape data from the internet, and some visualisation tools to help me learn. These are all simple programs and utilities. It doesn't matter if they don't work perfectly, as long as they're good enough.

The ability to throw something like this together in minutes rather than hours (or potentially days) as it would have years ago, has made AI such a valuable tool.

It's not just limited to coding, either, (although code often plays a part behind the scenes). I've used AI to quickly mock up designs for websites and landing pages to get a sense of how they should look before I build them. For one article I started drafting recently (and ultimately scrapped), I had AI rewrite it from a different angle to see if it would flow better. I wouldn't use the result, but it quickly showed me whether the idea had any merit.

These are all things that don't have to be perfect. A lot of the time, if it was something I was actually going to build and publish to the world, I'd start again from scratch and architect the whole thing properly. But most of the time, you just need a quick test to see if it's worth trying out, and sometimes the quick fix turns into the perfect tool for the job.

3. Be Careful with Decision Making, Planning and Especially Personal Guidance

You have an expert in your pocket who can give a tailored answer to any question you can dream up. For factual queries like "how do I use this software?" or "how can I train for a 10k run?", it's perfect. But once you're into emotional or non-objective territory, I'd be more cautious.

Sycophancy is still a big problem with AI. Under the pretence of being "helpful", AI overly affirms users and is biased towards agreeing with whatever they say. Don't get me wrong, we've seen remarkable improvements on this front in the past couple of years (I'm looking at you, Claude), but it still causes me problems on a daily basis.

The best approach I've found to mitigate this is deliberately asking AI to disagree with you. This changes the helpful answer from "blind confirmation" to completely tearing your argument apart.

AI will never not come up with an answer. What I mean is, whether you have a completely bulletproof plan or the worst idea ever conceived, AI can always generate a list of pros and cons. Ultimately, it's up to you to weigh up the options based on its feedback.

I'd definitely caution you against using AI to talk through personal or emotive topics. There's been research to show that, in personal guidance conversations, AI sycophancy makes people more convinced they are right, and less open to changing their mind. Sometimes, as much as you might not like to hear it, you were in the wrong and we can't reliably count on AI to break the news to us. Of course, if you want to be blindly validated in your beliefs, go right ahead, I wish you luck. But for everyone else, you should probably go to therapy instead, or at least speak to a responsible and (relatively) unbiased human.

4. It's Fantastic for Research

Most AI use cases boil down to synthesising information from different sources. Whether it's writing an email based on your prompt and the doc you uploaded; a customer support bot answering questions, or summarising some information from Google, it's all really just different flavours of the same thing. And AI happens to be particularly effective at it.

For research, I usually use Gemini to gather sources. I can't fully justify why I opt for Gemini for this. Maybe it's because I've been brainwashed into using Google products to browse the internet, or maybe it does actually surface better results. Either way, they have a large free daily quota, so I use it for more menial tasks and save my paid quota for other things. (No offense Google).

When I have some resources to work from, I usually plug them into something like NotebookLM and ask for a summary. From there, you can query the original texts and ask questions until it starts making sense.

Personally, I like to use this workflow for finding and digesting research papers. I read a lot about AI research and some of it can be quite dense, so having an AI tool there to help translate things from "academic jargon" into "normal person" (again no offense, I feel like I'm offending everyone today), proves really helpful.

Whether it's research or some other text synthesis task, there are 2 main problems you might run into that you should be aware of. Both of them, however, are largely solvable.

• Yes, AI can hallucinate, but when you directly supply it with the reference material the hallucination rate drops substantially. This is known as grounding. Tools like NotebookLM do this especially well and even supply direct links to the part of the text where the original information came from so you can always go back and check it yourself. This is great when accuracy really matters, but most of the time, the answer you get from any old AI tool will be sufficient.

• The trickier problem to spot is when an AI slightly twists or overstates the meaning of the text in a sort of silicon-powered Chinese Whispers. The best way to check is to ask a separate session or an entirely different model to review the output against the original. Ask how accurate the summary is compared with the source, and for places where the overall point is correct but there is a slight inconsistency.

5. Automating Processes

'AI automation' sounds scary and technical but it doesn't have to be. You'll hear people talk about "n8n pipelines", "Zapier integrations", and "custom agents" but you don't really need any of that.If you wanted to dip your toes in without drowning in technical jargon, the place I'd suggest starting with is Agent Skills.

It's basically just a set of instructions written in plain English that an AI agent like Claude can follow whenever you need. The best part is, for simple workflows, you don't even need to manually build it yourself; you can ask AI to do it.

For example, I have a skill for producing titles and descriptions for the blog posts on my website. I give it the blog post I'm writing, then it goes away and reads the article, identifies a keyword, writes a few title and description candidates, checks that they're a suitable length, and makes sure that everything is written in my brand voice.

This is a workflow that you could ask Claude to set up for you and have it working in less than 10 minutes. It's a really helpful way to package the things you're doing with AI into simple, reusable tools so you don't have to re-explain it every time.

The best part about it is that it really can be as simple as a set of written instructions. However, if you are looking for something more advanced, the ceiling is there with custom scripts, connectors and MCP servers. But that is entirely optional and you can build a powerful workflow without them.

6. It's only as good as the context you give it

AI gives you better answers when it understands the full scope of your prompt. A clearly defined task with specific instructions, input data, and the constraints that it needs to act within gives you something far more useful than a vague prompt would. Beyond that, supplying it with your reasoning, motivations, goals and the purpose of what you're trying to do will give an infinitely more informed and considered answer.

There are lots of ways to set persistent context for AI so you don't have to keep re-explaining yourself. Start using "Projects", Claude.md files, "memories", or if you want to go really over the top, build a custom knowledge-context system that feeds into your favourite agent.

Really, anything will do. Do some research, pick an approach that works with your chosen AI tool, and start using one today. I promise it will save you so much time and the results will be worth it.

What makes something AI-viable?

What do all of these use-cases have in common? They're all tasks where the efficacy of the output matters but the subjective quality doesn't.

AI doesn't have taste. It doesn't possess the intuition and tacit knowledge to truly master something to the extent that a person can. The best use cases are where AI extends our agency rather than asking it to exercise its own.

• It doesn't matter if the research report AI wrote happens to sound like AI. So what? Did it effectively convey all the information it was supposed to? Great!

• It doesn't matter if AI can't decide your life choices for you. It almost certainly shouldn't. It's great for helping you consider different options but use your own judgement.

• And it doesn't matter if the quick bit of code you wrote has a few bugs in it and wouldn't pass a security audit. Nobody else is ever going to see it or use it anyway but it was enough to prove to yourself that the idea was worth trying out.

Go experiment with it. Build something. The more you use it the more you'll understand it, and the more you understand it, the better the results will be.