I made something useful with AI
This is the second post reviewing Tim's personal journey with AI.
Tim is the original founder of PrimaTrade and, as some of you will know, founder of some other companies as well. Today I am the Chief Product Officer at PrimaTrade. I am also not a "technical" person when it comes to software development.
We don't normally include personal posts on our website - but this one nicely links to our business.
The first post is here: https://prima.trade/post/are-current-ai-tools-useful.
As foreshadowed in that post, I said I would come back and explain what I managed to do.
As you will read, there is a beginning (= toy), a middle (= still not there) and a end (wow!).
Here it is - "I made a something useful with AI!"
I set out to investigate four areas, two of which were to test out the ability of AI to advise and create documentary artifacts that form a natural part of our working process.
This post focusses on the final area - actually building and delivering a working product to the outside world.
I set out to build an intelligent "web chat bot".
We all know chat bots. I hate them, my colleagues hate them. As one of my colleagues wrote to me (strong language edited out):
Me...Where is my delivery, it's late! Chatbot...it was delivered yesterday. Me ..No it wasn't.. where is it? Chatbot.. do you have any further questions? No..where is my order, it wasn't delivered yesterday! Chatbot... Your order was delivered yesterday. Me... put me through to a person who I can converse with. Chatbot .. disconnects ,
But an AI-powered chat bot should be able to do much better than that.
And here it is live on https://www.es3g.com
ES3G is another company in our stable providing a technology that supports human rights due diligence in supply chains.
Click the link to go to the page and look for the "widget" in the bottom right hand corner that says "ask me anything".
Try it out. You actually can ask it anything (even whether Leeds United will win the FA cup or who might win the Superbowl next year).
We are now in the process of deploying the chat bot on https://prima.trade as well. Check back shortly - it will be there.
The chat bot brief
Given our collective negative experience, we wanted a chat bot that would be functional and easy to use.
We decided I had to do without any technical support from colleagues, noting that I am a non-technical person.
Functional requirements
Be useful, responsive and actually provide useful information about the business.
Not hallucinate, especially on key questions about our technology - like pricing, legal matters, processing model and implementation.
Offer a two-stage response:
Here is an answer based on our website
Would you like to expand that answer to get more context from the wider web?
Prompt users with a follow on question after each response to get a discussion going.
Enable us to test different AIs (eg: both Claude from Anthropic and ChatGPT from OpenAI)
Useability requirements
Work nicely on mobile as well as desktop, adjusting to screen sizes and expanding (appropriately) when a user calls it up
"Pop" on the screen to attract attention but then simply be present.
Include calls to action (eg: a button to book a demo and make links clickable)
We think it the chat bot now meets the brief. You can let us know easily (reach me here).
Making something useful with AI: the process
The actual process is not the point of this exercise - we did this to explore what's possible and to understand better what AI can do.
The beginning
But the process is what we all start with - so here is how we went forward. At each point, I asked the AI how to do things.
With some clear instructions (see the brief above) I quickly got back a basic dumb chat bot. Not what we wanted - so next steps were needed. It looked like a chat bot and behaved like one.
It was also running in the AI environment - and I wanted to be able to run it locally and then deploy it. So with AI guidance, I got a local environment set up on my PC (Windows) that could run the code generated by the AI. Install "Microsoft Visual Studio", which is free.
I needed to understand how to deploy it to the web.
Had to set up two online environments for that (under AI instruction).
There are, it seems lots of options - I was directed to two environments: Railway and Netlify. Followed my nose and they were quite easy to work out at a very basic level.
Then I wanted to connect the questions coming through the bot to the AI. I asked it how to do that - and it got me to put down US$5 onto their API subscription platform which gave me a key that I could put into Railway.
It is all a bit like playing with meccano or lego. If you are that kind of person (I still am), this is all quite fun.
This allowed me then to generate an app that basically enabled a user to type in a question and have it sent via the backend to the AI, to receive an answer from the AI and then route it forwards to display the answer to the user.
At this point we are about 4 hours in. Whilst there were some frustrating moments, the actual process was reasonably enjoyable - a great use of time outside normal working hours - and I am like any founder and have a rather elastic definition of what is a regular working day (or week for that matter).
But the AI was hallucinating. Ask it what our platform costs, and it makes up the answer. That's not acceptable from a business point of view - in other words, it's not useful.
The middle
The middle is the useful process (from a learning and business point of view).
Here's where we converted the "toy" output into something we could direct and control, given the limitations of the AI systems themselves.
Making simple things with an AI is not that hard. But simple things are rarely useful in business.
At this point, I have validated the principle that I can make a toy - and one that gets us from A to B but there are a lot of issues before it can be useful.
I quickly found that both ChatGPT and Claude get indigestion with a code file of more than 1000 lines. When I ask them to improve something or add something, inevitably something else breaks.
So I took the step to break up the application into components. This step alone took around 2 hours to sort out - iterating the provided code through the AI platforms until, eventually, a component approach worked. The app has become 16 files instead of the previous 2 files (the two previous files were a backend and a frontend - now the whole app is in pieces).
Splitting the code into components enabled iterative progress to resume.
The next step was to stop the hallucinations. Without this, we would not have a useful result.
What I learned was that the two AI services I am using (ChatGPT and Claude) do not allow me to create some kind of "private training" process. The AIs do seem to maintain a common thread through a single chat - but then forget all that when the next chat starts.
So any form of control over the responses has to be generated via the prompt - what you ask it to do. And then we have several problems:
We need a source of guidance - something we can provide when a chat starts that can provide the guardrails to prevent unwelcome hallucinations.
We need to upgrade the prompting to pick up that source of guidance so the request from the user is then wrapped into a more comprehensive prompt to the AI.
The more we send, the slower the app will respond and the more expensive the process will be - the AI system charges us based on "tokens" which are related to what is going back and forth. If the prompts get longer, more tokens are consumed.
So I asked for help from the AIs and got some ground rules about how much guidance we can sensibly provide and how we might structure that. I then made a bunch of decisions (ie: judgements) - and tried it out.
Several hours later and about 50 iterations of the code - I got to a balancing point where the hallucinations stopped, the answers were good - and I sat within the rules that the AIs had suggested.
There is no "right answer" here. Iteration and judgement are important.
So now we have a chatbot that is functionally useful.
We have something - but how to make it work properly from a usability point of view. At this stage, the chat bot is not very easy to use, especially on the mobile, even though it works.
The last lap
This was the longest part of the whole task. At this point, I have spent maybe 10 working hours iterating through the various processes to get a web chatbot that is functionally useful:
Users can ask any question they like
They do get sensible answers back
Where the question and answer relates to our business, the answers are reliable and seem to be accurate
But the user experience remains problematic, especially on mobile. I want the app to be really easy to use, engaging, responsive, and I want it to "pop" on the screen so that attention is drawn to it.
So I googled what best practice looks like for webchat apps or chatbots and came up with a series of features and behaviours that I wanted to see.
For example:
Provide a pop up message over the widget, and correctly identify which AI we are using (as we can switch between Claude and ChatGPT on the backend).
Float up a series of potential questions as semi-transparent prompts above the app that are clickable and which then go away - and make the number of prompts adjust to the screen resolution
On the mobile, don't load the keyboard when the chatbot is first clicked - as it takes up most of the screen.
When the user exits the chatbot, don't exit the whole website ...
And others ....
I wish I could tell you that this was quick and easy. I spent probably another 10 hours on the task of making the webchatbot behave acceptably from a useability point of view.
I had to become a lot more familiar with the actual code lines that I would have liked, because I quickly realised that allowing the AIs to simply update the whole file generally meant they fiddled with things I didn't want touched - so it had to be done as a "patching" process - meaning the changes were provided as instructions to edit the files rather than by generating completely new versions.
What have we learned - what is useful
The most useful part of the process is the middle.
The beginning and end were necessary steps to deliver a finished product. But the useful learning came from the middle step. After all, it is never going to be my job actually to write code and build products that go into our production environments.
The middle step has taught us:
How to use the publicly available tools (eg: ChatGPT, Claude) and make them operate within guardrails and to principles that we provide. So we get a "tailored" result without having to build our own LLMs (ie: large language models).
That is important because
it means we can create value for our business very quickly - a day or two.
we do not have to hire expensive AI engineers and specialists, we have got the principles now
the cost of using AI this way to support our production environments is probably 1/1000th of the cost of building our own LLM
As mentioned above, this is the middle step. It is the process of "prompt engineering" - and then working out how to do that dynamically so that inputs from our world can generate controllable outputs from the AI world that we can re-use with confidence.
So where next?
Outside helping us to write clearer and better instructions to our teams (the "advisory" part of what AI does) - have we now learned enough to create re-usable AI tools that can be put into our production environments?
Yes.
Our CTO and I can now clearly see where and how certain operational processes on our platform can be enhanced by adding in AI capabilities.
We can also that this can be done in a production environment, safely and with guardrails, quite quickly and without significant costs.
As a business user, and without going through the long pain of "the last lap", I can create prototypes that we can use to create and validate tools using sample data, configuring the tools so that they behave the way we want (the middle step in the process above). Then I can hand these over to our professional development team for implementation.
The use cases are different for the two platforms here that we operate:
the PrimaTrade platform
the Ask the workers platform (run by ES3G)
Areas of focus will be:
Fewer of our own resources required to monitor and support user activities (transactions, messages) on our platforms.
New forms of enquiry that can be bolted easily into our dashboards (eg: automatically harvesting, analysing, categorising and reporting social media data on businesses that are on the platform - managing language differences automatically).
Using natural language processing to triage text inputs in any language from users (eg: via grievance channels that are part of our Ask the workers platform).
What we have left to check and think through now relates to confidentiality of our data - and to ensure that what we send outside our platform is sanitised and anonymised before being processed by a third party AI environment. We can also see how to do that now.
Exciting times
Now that September is underway, back to the regular work.
Would I recommend that you have a go [as a non-technical person]?
Definitely yes, especially if your inner child still enjoys building things.
Maybe do it outside regular working hours and let your family know you will be lost for a while.
It is a remarkably satisfying process.
Do contact me if you would like to discuss any part of this post. You can reach me easily by:
Going to www.es3g.com and firing up the chatbot and clicking the "demo" button. That's a link directly to my calendar!
Or email me via info@prima.trade.
Or book a call via this link.