GPT 4 Got Upgraded – Code Interpreter (ft. Image Editing, MP4s, 3D Plots, Data Analytics and more!)
I just got access to the code
interpreter plugin about 48 hours ago
and I’ve been running experiments on it
non-stop since then I’ve come up with
about 18 examples to show you guys its
power most of them I reckon haven’t been
seen before I predict many Industries
will have to update overnight when it’s
released more widely and at the end of
the video please let me know what you
think and what other experiments that we
can try first though what about this one
a 3D surface plot just quickly the way
it works is you click this little button
3D Surface Map
to the left of the text box and then you
can upload many different file types
like CSV files Word files images and
even short videos then it will
automatically analyze the file type
without you pressing anything and then
of course you give it a prompt and as
with all of chatbt it becomes a
conversation so the first 3D surface
plot was decent but it was too small so
I simply said in natural language can
you make it four times bigger thank you
and of course you have seen the amazing
end result even with the lighting look
at the shadow is there I believe this is
based on a real contour map of a volcano
in New Zealand and I could do a whole
video just on this but I have 17 other
examples to get to but this one was
truly amazing did you know for example
it can generate QR codes I said create a
QR Codes
QR code that I can scan with my phone to
reach the following URL and lo and
behold it creates it and yes it does
work maybe I’m easily impressed but I
think that’s pretty amazing and what
about a 3D scatter plot this is truly
3D Scatter Plot
remarkable I uploaded the data from
gapminder and it created this chart
based on the median age of over a
hundred countries from 1950 I think
projected to 2100 and I asked highlight
the UK this is indeed the UK’s median
age through those years in red but I
know what you might be thinking that is
amazing that it’s 3D and interactive but
the blue kind of merges and it’s hard to
see what’s going on I engage in a
conversation and look what it created it
picked out the 30 most populous
countries and separated them off with
separate colors look at that that is
gorgeous
now you might have the critique that the
median age is in descending order in the
y-axis going from 20 down to 60 so in a
sense the median age is actually Rising
not falling but nevertheless that’s
easily amendable and that is truly an
incredible diagram and look just for fun
I’m going to go into the data look at
this I’m traveling into the data this is
so wild I don’t know how helpful it is
but I think that’s just beautiful and
crazy there are so many Industries data
analytics accounting consultancy that
this will affect by the way it got all
of this done in about a minute I see a
lot of people online talking about five
seconds later it is no way done in five
seconds you have to wait 30 seconds a
minute sometimes much longer before I
move on I want to give you a killer tip
that it took me quite a while to work
out so when you get access try to
remember this say output the
visualization as a downloadable file if
you don’t add that phrase as a
downloadable file what will happen is it
often gets stuck at this stage of the
code it’ll either say fig.show or
plot.show and then just stop I found
that I encountered this problem far less
often if I said output a downloadable
file next did you know that code
Optical Character Recognition
interpreter can do optical character
recognition I screenshotted this text
from a New York Times article I think it
was and I asked OCR the text in this
image and write a poem in Danish about
it now I don’t want to exaggerate it
often gets OCR wrong I don’t want to get
your hopes up it fails more often than
it succeeds but when it works it can do
it understood the text and then did a
poem in Danish about the text now I’m
going to need a Danish speaker to tell
me if that was a good poem but either
way it could do it how about this one it
can do interactive time series with
Time Series Range Sliders & Selectors
range sliders and selectors I uploaded a
CSV file on life expectancy data from
the entire world and I just said can you
pick out the U.S UK and India and create
a Time series with range slider and
selectors again that killer phrase
output a downloadable file and here is
what it came up with notice how the life
expectancy for all three countries Rises
during the 20th century and look how I
can select down here interactively a
range of the data and even by clicking
up here a 10-year interval or 50-year
interval but here’s the crazy thing I
did nothing I just uploaded the file
there were hundreds of countries in
there you can see here all the steps
that it did and if you click on the
Arrow you get to see the actual code
then it goes through shows its
explanation and eventually gives you a
link that you can simply click and get
the file downloaded and if you weren’t
that impressed already here’s where it
gets fairly game changing you can get it
to do the data analytics not just the
Data Analysis
visualizations for example I said find
five unexpected non-obvious insights
from this data and offer plausible
explanations for them this was bad to
the median age data for the most
interesting observation provide a
compelling and clear visualization now
ignore the first diagram which wasn’t
that good because of the x-axis but look
at the insights this is data analytics
you can see here that the original file
was called median age years and it was
just a table of data no analysis
whatsoever but look what gpt4 picked out
in site one the global median age has
been steadily increasing over time it
calculated the global median age that
wasn’t included in the data it was just
country data and it says it’s gone from
around 22 years to over 38 years in 2023
and it’s projected to continue rising to
approximately 44 years by 2100 and then
it offers a cogent explanation this
trend is likely due to a combination of
increasing life expectancy and
decreasing fertility rates worldwide as
Medical Technology improves more people
are living longer birth rates are
declining particularly in developed
regions is pick this all out and then it
moves on to the the next Insight the
countries that have seen the most
significant increases in median age are
these ones and again it gives an
explanation as to why their median age
might have risen more than any other for
example Albania has seen significant
emigration of younger people which could
also lead to an older median age is it
me or is that kind of crazy that it
crunched all the data visualized it but
then also gave really interesting
analyzes of the data now you can read
the other analyzes but each of them are
really interesting and the final
visualization which I asked for is
brilliant I think notice how the graph
goes from green to red when you get to
the Future projection I didn’t ask it to
do that now obviously in this video I’m
going to focus on the flashy visuals and
the cool little tricks it can do but in
terms of data analytics that is what is
going to change jobs change Industries
and remember this is code interpreter
Alpha version one look at the difference
between mid-journey version one and now
mid Journey version 5 a year later how
about basic video editing now there is a
7. Video Editing
limit to what it can do but it can do
some basic video editing if you ask it
for example I uploaded a short file and
asked it to rotate the file 180 degrees
and it was able to do it now I’m not
saying that is massively useful but it
was able to do it here is a similar
example I uploaded an image file and
then said can you zoom out from the
center of the image now initially it did
zoom in but then I clarified that I
wanted it to zoom out from the center
just to be cheeky I also asked can you
make it black and white oh and I also
asked to add music but it couldn’t add
music anyway here is the end result by
the way it gave it to me as an mp4 file
and look it zooms out from the center
and it’s made the image black and white
now because I got access so recently I
honestly haven’t explored the limits of
what kind of video editing I can do with
chat GPT code interpreter but I will let
you know when I can now back to
visualizations I gave it a hypothetical
scenario that sounds kind of realistic I
sent 231 CVS got 32 responses 12 phone
interviews three follow-up face-to-face
interviews and one job offer which I
rejected I’ll put a downloadable Sankey
diagram of this data I did then get it
to change the coloring slightly but I
think that’s a pretty cool Sankey
diagram look sent CVS 231 and then
receive responses and you can go down 32
phone interviews 12 face-to-face
interviews and three job offers and one
rejected offer obviously I could have
tweaked that for hours make it more
visual make it more interactive maybe
make a gif of it but for two minutes
work I think that’s a pretty interesting
and incredible output next and here is
one that you might say is a little bit
Steganography
concerning and it’s about steganography
now I will admit I am not at all an
expert in fact I know virtually nothing
about it essentially what it involves
though is hiding a message inside an
image or in inside some code and gpt4
was more than willing to play along and
it encoded a secret message into an
image there was the image by the way and
if you looked at that you’d think that’s
totally normal that’s just a silly
little image right well apparently
here’s what it can do to a casual
Observer it looks like a simple image
with some shapes but it actually
contains the hidden message hello world
then it provided a python function which
can be used to decode the message from
the image now obviously this is just a
silly example that is totally harmless
but am I being crazy in thinking this is
a somewhat concerning ability for future
language models to possess especially
when they reach the level of an AGI
often openai talk about future versions
of GPT doing scientific research and
finding things that humans wouldn’t have
discovered but let me pose the scenario
that it gets better than any human
expert at steganography anyway enough
from me I’ll let the experts weigh in on
that one next did you know that gpt4
with code interpreter can do to text to
speech just before anyone comments
though why did I write proceed without
further question because GPT 4 with code
interpreter has a tendency to always ask
clarifying questions and if you have
access to only 25 messages every three
hours you don’t want to use up half or
more of them on clarifying what it wants
to do or saying yes please do that but I
found writing proceed without further
question means it gets straight to it
and essentially you get double the
number of prompts for your money anyway
as you can see I asked turn this entire
prompt starting from the beginning into
a text speech file now quite a few times
it denied it had the ability to do this
but eventually I got it to work it was
actually when I finally gave it this
prompt and it worked I say it worked but
it didn’t quite work as intended check
it out here is the text-to-speech that
it came up with a large language model
trained by open AI when you send a
message containing python code to python
it will be executed in a stateful device
a notebook environment python will
respond with the output of the execution
or timeout after 120.0 seconds internet
access for this session is disabled do
not make external web requests or API
calls as they will fail now thank you
Stephen Hawking for that message the
only thing is it had nothing to do with
my original prompt now anyway when you
get access to code interpreter play
about with text-to-speech because it is
able to do it even if it denies it time
for a fun one I asked create a tree map
Treemaps
of the letters in the following quote
and I’m not going to read it out because
I am not good at tongue twisters anyway
I said give each part of the tree map a
different color and output a
downloadable file proceed without
further question and here is the output
and I checked it for the letter P and it
was correct that there were 36 instances
of the letter P in the output and look
how it’s proportional with the number of
instances of the letter and the size of
each rectangle I think that is pretty
insane okay back to something more
serious I uploaded this file which is an
image of a math problem quite a hard one
as well and you guessed it I said solve
the math problem in this image it then
extracted the text from the image
presumably using OCR and then proceeded
to solve it and I’m going to get onto
this in a second it is better at math
than Wolfram Alpha I know that’s a big
claim but it’s far less buggy I found
Wolfram Alpha crashing very frequently
anyway here are the two solutions and
isn’t that incredible from a photo
essentially it then extracts out the
math problem including the two square
roots and then solves it this is all
within the same window of chapter no
need for any other apps or extensions
next it can do radial bar plots which I
think are really quite beautiful I’m not
saying this is the best one ever and I’m
Radial Bar Plots
sure you could tweak it to make it more
clear and beautiful look at that the
life expectancy in the US climbing from
1800 and then it goes clockwise reaching
a projected almost 90 by 2100 again I’m
sure you could do a far better job than
me in extracting out a more beautiful
diagram but aren’t radial bar plots just
beautiful to look at speaking of cool
diagrams how about this I didn’t even
specify which visualization to do I
uploaded this same life expectancy data
and I just said what are the most
advanced and Technical visualizations
you can do with this data proceed to do
them now honestly it picks some
visualizations that I don’t think are
the most advanced but nevertheless it
was creative here is what it did it does
frequently make the mistake of
cluttering the axes and having far too
many labels so that you can’t see
anything so scrub that one out not great
but what about the next few remember it
just did this on its own this is a heat
map and you can see some really
interesting things from this data like
India starting with a much lower life
expectancy than anyone else but
gradually Rising but still falling
behind the others even in 2100 and look
at China look how the life expectancy
drops in the 60s and 70s I think we all
know what happened there compare that to
the US which is a gradual continual
Ascent actually aside from 22 20. look
how the shade gets a little darker in
2020. obviously you guys can probably
work out what happened around then but
then the projections are for it to go up
toward 90 by 2100 that’s a beautiful and
clear heat map that I didn’t even ask
for it to do let’s look at the next one
box plot do you remember those from
school you get the upper end of the data
the highest one the lowest one the
median the first quartile and third
quartile and it’s a great way of
statistically representing a set of data
and it’s done it for every 50th year
starting in 1900. obviously a slightly
less beautiful diagram than some of the
ones you’ve seen today but for the
statisticians in the audience you will
know that this is a very useful metric
for a lot of data the individual points
above and below are typically when there
are outliers in the data I would
estimate that all of these
visualizations only took around two two
and a half minutes so definitely not the
10 seconds as I said that you often see
on Twitter I mean have you ever seeing
gpt4 give an answer in less than 10
seconds speaking of useful I think many
professionals will find the next thing
that I’m about to showcase the most
useful of all any insights that Gypsy 4
finds Trends medians analyzes whatever
you can ask it to add to the original
file and then download it do you
remember that the original file was
called median age years well notice this
file name median age years with insights
it has created a downloadable new file
with the insights included and look at
some of the insights that I mean you
have the change from 1950 to 2100 and
here is the average median age
throughout the period and the change
from 2023 to 2100 notice that the
original file didn’t have those columns
they were added by gpc4 with code
interpreter and now how about data
progression video files I was honestly
shocked when I saw that it could do this
but I asked can you make a 256 by 256
MP4 that gradually reveals the lines as
they progress on the x-axis this was
about the median age over time here is
what it did and look at how the data and
the chart progresses as time moves along
I was really shocked to see this and the
line in red which is going to be labeled
at the end is the global median age and
remember it calculated that that wasn’t
in the original file now I’m not sure
why it picked out these four countries
maybe because they represent extremes
either way I think the result is
phenomenal and I’m genuinely impressed
that it did this even though I know the
final result could be improved
dramatically for example far higher
resolution and maybe the global median
age labeled from the start and actually
now that it’s got to the end I can see
why it did pick out these countries
because Niger did have the lowest median
age in 2100 and it looks like Puerto
Rico had the highest and the fastest
aging one was Albania next and this this
is going to shock quite a few people
Image Editing
what about image editing I created this
image in mid-journey version 5 and then
here’s what I asked I said use opencv to
select the foreground of this image and
look what it did it picked out the
foreground no Blue Sky now I know it’s
not perfect but it’s nevertheless
impressive all within the window of
chapter BT this does actually make me
wonder if open Ai and chat to BT is
eventually not now but in a few years
gonna swallow all other apps or maybe
Google’s Gemini but either way one
interface one website one app doing the
job of all others and by the way of
course chapter BT is now available on
iOS but imagine you have one app and it
can do image editing text-to-speech
video editing everything data analysis
not add gpt4 levels but GPT 6 or gbt 7
levels if you can get every piece of
information service and application in
one interface a bit like now people
being addicted to their smartphones
won’t people be a addicted to this one
interface again that’s not going to
happen now but I’m just posing it as a
question to think over for the moment
though before anyone gets too carried
away it does still hallucinate quite a
lot so I uploaded this image and I asked
it questions about it and it answered
and I was like wow it can do image
recognition it said this image appears
to be a digital painting of a humanoid
figure at a desk with a rather complex
background I was initially amazed until
I realized that it probably got that
from the file name because when I asked
it questions it got it wrong so I said
what is on the desk now look back
there’s this weird kind of microphone
and a bit of paper and not much else a
keyboard and look what it said there are
multiple floating holographic displays
okay a mouse not really a desk lamp I
can’t see that and then tools and
devices now correct me if I’m wrong but
I think most of those are incorrect now
obviously I need to do far more
experiments to see if it actually can
recognize any particular images and
maybe I’m putting it down too harshly
but at the moment it does seem to
hallucinate if you ask it about too much
of the detail of an image next remember
how one of the key weaknesses of GT4 is
that it can’t really count things
especially not characters words Etc and
even more so it can’t do division and
some of you might be thinking well with
Wolfram Alpha it can do those things not
quite here is an example of the code
interpreter plugin essentially eating
Wolfram Alpha obviating it making it not
obvious what the utility of it is if
you’ve got code interpreter I asked
divide the number of the letter e’s in
this prompt by the number of the letter
T’s now you might think code interpreter
can improve things by doing the
character counting but it can also do
the division notice how it counted the
characters correctly compared to Wolfram
Alpha and of course got the division
correct as well so if it can do Advanced
quadratics and do division and character
counting Etc it does beg the question
what would we use Wolfram Alpha for that
we can’t use code interpreter for I
honestly might not know something that
you guys know so do let me know in the
comments it also also got this math
question correct and notice you get
these beautiful map visuals that you
don’t get with the base version of gpd4
you get something more like this where
the visuals aren’t as clear and notice
the base version of GT4 gets the
question wrong it can’t do division but
with code interpreter it gets the
question right next one is a quick one
pie charts nothing too special but I
think it is a fairly beautiful
visualization it doesn’t seem to matter
how big the CSV file is that you upload
this next example was really quite
fascinating it was a word puzzle I have
tried this particular word puzzle on
gpt4 dozens of times the reason I picked
this puzzle is called a Word ladder is
because it really struggles with the
puzzle if the number of steps required
is more than a certain number usually
about five or six steps it gave me a
really interesting border of the limits
of gt4’s planning abilities with
language anyway it always gets it wrong
here is a demonstration with the base
model of gypsy 4. you might say why is
this wrong but look at how it’s changed
change from Seas to sags which is more
than one letter change and that’s
typical of the kind of Errors it makes
what about with code interpreter well
you can probably guess the ending given
that I featured it in the video but it
gets it right I believe it draws Upon A
hard-coded word set and this does Point
towards the kind of puzzles that I think
gpc4 with code interpreter will be able
to solve things like crosswords and
sudokus okay not exactly world changing
but nevertheless I think quite
fascinating and how about Venn diagrams
the reason I picked this example is that
I had to go through about 10 steps to
get it to create this rather basic
Bonus: Venn Diagrams
three-way Venn diagram this represents
the overlap between dogs Ai and desks
and apparently all of them are loyal
companions well we will see about that
but anyway it took quite a few steps to
get it right which was pretty annoying
but here’s the really interesting thing
once I got it set up in the way that I
like all I had to do was say use the
format above to create a new three-way
Venn diagram this time for Mango’s Movie
heroes and marmosets try to make each
entry funny and use different colors
proceed without further questions so it
may have been a struggle to set up
initially but once done it was so easy
to iterate a new three-way Venn diagram
and actually it was better than the
original apparently all three are adored
by fans worldwide apparently only
marmosets and Movie heroes can climb up
trees really fast and mangoes and
marmosets can hang upside down that’s
crazy one or two prompts iterating on a
design already agreed upon this is
honestly what is likely to happen in the
future with people spending hours to
find the perfect data visualization or
piece of data analysis and then just
hitting copy paste for all their other
files perfect it once and then it does
the rest for you a quick couple of bonus
ones before I finish you can just ask it
to come up with a visualization giving
it no direction at all it came up with a
distribution of prime numbers up to ten
thousand thing is I believe there’s a
slight mistake at the beginning because
I think there’s only 25 in the first 100
and 21 in the next 100. so you probably
do want to still check the outputs that
code interpreter gives you and that’s
another reason it’s not going to
instantly replace all data analysis and
data visualization it’s not perfect and
it’s not fully reliable but you’ve got
to look ahead to where things are going
I’m going to end where I started with
this insane 3D surface map of a volcano
if this is what gpd4 can do now with the
Alpha version of code interpreter what
will GPC 5 or 6 do with version 7 or 20
of code interpreter I was about to
speculate about that but then I got
distracted with trying to get inside
this volcano it is kind of fun look I’m
going above and into the volcano let me
know what you will try when you get
access I know they’re rolling out
steadily and I know that some people
will have had access to it for about
three weeks so hopefully if you want to
experiment with it you will be able to
soon in the meantime do let me know if
you have any ideas that you want me to
experiment with and thank you so much
for watching all the way to the end have
a wonderful day
