Ask HN: Go deep into AI/LLMs or just use them as tools?

It probably helps a little to understand some of the internals and math. Just to get a feel for what the limitations are.

But your job as a software engineer is probably to stick things together and bang on them until they work. I sometimes describe what I do as being a glorified plumber. It requires skills but surprisingly few skills related to math and algorithms. That stuff comes in library form mostly.

So, get good at using LLMs and integrating what they do into agentic systems. Figure out APIs, limitations, and learn about different use cases. Because we'll all be doing a lot of work related to that in the next few years.

Even the basic chat UI is a structure built around a foundational model; the model itself has no capability to maintain a chat thread. The model takes context and outputs a response, every time.

For more complex processes, you need to carefully curate what context to give the model and when. There are many applications where you can say "oh, chatgpt can analyze your business data and tell you how to optimize different processes", but good luck actually doing that. That requires complex prompts and sequences of LLM calls (or other ML models), mixed with well-defined tools that enable the AI to return a useful result.

This forms the basis of AI engineering - which is different from developing AI models - and this is what most software engineers will be doing in the next 5-10 years. This isn't some kind of hype that will die down as soon as the money gets spent, a la crypto. People will create agents that automate many processes, even within software development itself. This kind of utility is a no-brainer for anyone running a business, and hits deeply in consumer markets as well. Much of what OpenAI is currently working on is building agents around their own models to break into consumer markets.

I recommend anyone interested in this to read this book: https://www.amazon.com/AI-Engineering-Building-Applications-...

I see this a lot, but I think it's irrelevant. Even if this is a bubble, and even if (when?) it bursts, the underlying tech is not going anywhere. Just like the last dotcom bubble gave us FAANG+, so will this give us the next letters. Sure, agentsdotcom or flowsdotcom or ragdotcom might fail (likely IMO), but the stack is here to stay, and it's only gonna get better, cheaper, more integrated.

What is becoming increasingly clear, IMO, is that you have to spend some time with this. Prompting an LLM is like the old google-fu. You need to gain experience with it, to make the most out of it. Same with coding stacks. There are plenty of ways to use what's available now, as "tools". Play around, see what they can do for you now, see where it might lead. You don't need to buy into the hype, and some skepticism is warranted, but you shouldn't ignore the entire field either.

So in that case I don’t see why not?

But as for my 2 cents, knowing machine learning has been valuable to me, but not anywhere near as valuable as knowing software dev. Machine learning problems are much more rare and often don’t have a high return on investment.

1) Established companies (meta/google/uber) with lots of data and who want MLEs to make 0.1% improvements because each of those is worth millions.

2) Startups mostly proxying OpenAI calls.

The first group is definitely not hype. Their core business relies on ML and they don’t need hype for that to be true.

For the second group, it depends on the business model. The fact that you can make an API call doesn’t mean anything. What matters is solving a customer problem.

I also (selfishly) believe a lot of the second group will hire folks to train faster and more personalized models once their business models are proven.

If you are considering whether the future will boost the demand to build AIs (i.e. for clients), we could say: probably so, given regained awareness. It may not be about LLMs - and it should not, at this stage (it can hit reputation - they can hardly be made reliable).

Follow the Classical Artificial Intelligence course, MIT 6.034, from Prof. Patrick Winston - as a first step.

If you're good at what you're doing right now and you enjoy it — why change? Some might argue that AI will eventually take your job, but I strongly doubt that.

If you're looking for something new because you are bored, go for it. I tried to wrap my head around the basics of LLMs and how they work under the hood. It’s not that complicated — I managed to understand it, wrote about it, shared it with others, and felt ready to go further in that direction. But the field moves fast. While I grasped the fundamentals, keeping up took a lot of effort. And as a self-taught “expert,” I’d never quite match an experienced data scientist.

So here I am — extensively using AI. It helps me work faster and has broadened my field of operation.

But I believe that the value will come after the bubble is burst, and the companies which truly create value will survive, same as with webpages after the dot com bubble.

Most of my LLMs made lots of mistakes, but Codex with $200 subscription changed my workflow totally, and now I'm having 40 pull requests/day merged.

Treat LLMs as interns, increase your test coverage with them to the point that they can't ruin your codebase and get really good at reviewing code and splitting tasks up to smaller digestible ones, and promote yourself as team leader.

I gave it an honest chance, but couldn’t get a single PR out of it. It would just continue to make mistakes. And even when it got close I asked it a minor tweak and it made things worse. I iterated 7 times on the same small problem.

If you want to switch fields and work on LLM internals/fundamentals in a meaningful way, you'd probably want to become a research scientist at one of the big companies. This is pretty tough because that's almost always gated by a PhD requirement.

For example, if we wanted to conduct an analysis with a new piece of software, it wasn't enough to run the software: we needed to be able to explain the theory behind it (basically, to be able to rewrite the tool).

From that standpoint, I think that even if you keep with #2, you might benefit from taking steps to gain the understanding from #1. It will help you understand the models' real advantages and disadvantages to help you decide how to incorporate them in #2.

Very wise advice! And the more complex systems are, the more this is truly needed.

1/ There aren't many jobs in this space. There are still far more companies (and roles) that need 'full-stack development' than those focused on 'AI/LLM internals.' With low demand for AI internals and a high supply of talent—many people have earned data science certificates in AI hoping to land lucrative jobs at OpenAI, Anthropic, etc.—the bar for accessing these few roles is very high.

2/ The risk here is AI makes everyone good at full-stack. This means more competition for roles, less demand for roles (now 1 in-experienced engineer with AI, can output 1.5x the code an experience Senior engineer could do in 2020).

In the short/medium term, 2/ has the best risk/reward function. But 1/ is more future proof.

Another important question is where are you in your career? If you're 45 years old, I'd encourage you to switch into leadership roles for 2/. This wont be replaced by AI. If you're early in your career, it could make more sense to switch.

[1]https://news.ycombinator.com/item?id=44079296

Do you need to understand how the circular saw and drill are made?

That doesn't mean knowing every single bit there is to know about it, but a basic understanding will go a long way in correctly using it.

You don't need to deep dive into the maths. You'll need to understand the limitations, the performance bottlenecks, etc. RAGs, Vector DBs, etc

Work for companies (as a consultant?) to help them implement LLMs/AI into their traditional processes?

Learning to work with the outputs of them (which is what I do) can be much more rewarding. Building apps based around generative outputs, working with latency and token costs and rate limits as constraints, writing evals as much as you write tests, RAG systems and embeddings etc.