HUGE ChatGPT Upgrade: 16K Context Window – Lower Prices + Testing
We just got some exciting news from OpenAI. They have now updated their API and function calls, so we are announcing updates, including more steerable API models, functioning calling capabilities, longer context, yes, and of course, lowered prices, yes! I’m very excited for this, so let’s take a quick look here before we do some examples. You can see here today we are following up with some exciting updates, so updated and more stable versions of GPT-4 and GPT-3.5 Turbo. Yes, I like that. This is a big one, new 16k context window of GPT-3.5 Turbo. That is very interesting; that’s a four times increase in the context window, and I have an example lined up for you that we’re going to take a look at. 75% cost reduction of the embeddings. I guess that’s other… 0.0025 cost reductions for GPT-5 Turbo. Okay, that’s cool. And there are some interpretation timelines for GPT-3.5 and GPT-4. Let’s take a look at the models.
So if we go down here, you can see we have a new GPT-4 32k version. I don’t have access to that yet, but I am trying to get that as soon as possible because I really want to play around with that. The most exciting thing today is actually the GPT-3.5 Turbo 16k that offers a four times contact length, like I just said, at twice the price. You can see now it used to be 0.002 in the GPT-4 Turbo API. If you take a look at the price here, you can see the GPT-3.5 Turbo’s input token has been reduced by 25, so you can use this model now for 0.00015 per 1k token inputs and 0.002 per output, so that is roughly 700 pages per dollar. That’s quite cheap, right? And the 16k will be priced at the…uh, 0.03 per 1k input and 0.04 per 1k token. So that’s double the price, but we get four times the context window, and yeah, very excited for this. I think we’re gonna take a look at an example of this. Just wanted to quickly show you in case you didn’t know what this exactly means if you are going to use this API.
So we have the 4K HR GPT API here, we have the 16k here, and you can see this is outside the context window. Now if we put in my YouTube channel, it’s all about AI, and we put in around 3,000 words here, and if we ask ChatGPT, “What is my YouTube channel name?” It can’t find any information about the YouTube channel. This is because we can only fit 3,000 words or 4K tokens above this, and if we try to ask, then it can only read or remember these words, so this is gonna end up outside the context window, right? But if we have 16k here, so you can see I put my name, “YouTube channel name is all about AI.” I put, let’s say, 11,000 words here, and then I can ask, “What is my YouTube channel name?” I said GPT-4 here, but ChatGPT can really answer this and say, “Your YouTube channel name is all about AI,” because it can read up to, let’s say, 12,000 words, circular-ish, and yeah, that is basically how this works, and this is a big difference because now we can feed it a lot more window.
I want to take you over to another example of this. Here you can see I just created a very simple chatbot. So it’s just a user and a chatbot, so I can talk to this now. So, “How are you?” Right, so this is basically a simple chatbot, basically like ChatGPT, but you can see now I have used a different model here. I have used the GPT-3.5 Turbo 16k, and we are going to test this limit now. So what we are going to do is, I have this document here; I will quickly show you. So this document is about… You can see it’s 26 pages, around that. So if you go out to words here, it’s 9,800 words. So what I’ve done here is, if I scroll down here, you can see I copied something from my website, “What is the Tree of Thought prompt method?” Right, and I just copy this, paste it into the middle of those 20 pages, I copy this whole thing. I went back to the chatbot and I gave it, like, a system prompt, and I fed all of this in through the chatbot.
So here is your knowledge base. So what is good about this is that now we can put in, like, 10,000 words here, almost, in the knowledge base, and we can question the chatbot about this, right? So we’re gonna give that a test now. So remember, I put in that Tree of Thought thing, so we are gonna ask it that question. So let’s just copy this, “What is the Tree of Thought prompt?” And let’s go back to the chatbot, ask that question. So, hopefully, I can find this now in the knowledge base. Yeah, so you can see here, the Tree of Thought is not AI problem-solving method used in prompt engineering in multi-gody models, such as ChatGPT, to avoid invalid input and spinner. So there was no place for this to take this other than looking at the knowledge base that we fed in here. So this opens up a whole lot of opportunities that we can use this for. So when we are, like, putting in 10,000 words here, that’s quite a lot we can work with, right? So very exciting news. I can’t wait to play around more with this. Hopefully, I can get a 32k version from GPT-4 soon. Yeah, this was very good news, very exciting, so I can’t wait to play around more with this and try to build some more exciting stuff when it comes to this context window. So I’m going to be exploring that in the weeks to come. So hopefully, I will see you then. Have a good day!
