Cristian Magherusan-Stanciu from AutoSpotting, a company helping to cost-optimize their AWS EC2 spent by automatically supplying matching workloads with spot instances.
For questions, you can reach Cristian at:
- LinkedIn: https://www.linkedin.com/in/cristimagherusan/
- Twitter: https://twitter.com/magheru_san
You can learn more about AutoSpotting at:
- Website: https://autospotting.org/
The Cloud Commute Podcast is presented by simplyblock (https://www.simplyblock.io)
01:00:00
On a daily basis, I use ChatGPT for
01:00:03
software development.
01:00:05
So all my code over the last year
01:00:07
pretty much is
01:00:09
generated by the AI.
01:00:13
And yeah, I'm using it, not
01:00:16
offering AI tooling.
01:00:19
Like, you know, every application
01:00:21
now has an AI feature.
01:00:23
I don't have an AI feature in
01:00:25
AutoSpotting, but the way I build
01:00:27
AutoSpotting
01:00:28
evolved using the AI.
01:00:35
You're listening to simplyblock's
01:00:36
Cloud Commute Podcast, your weekly
01:00:38
20 minute podcast about
01:00:39
cloud technologies, Kubernetes,
01:00:41
security, sustainability and more.
01:00:43
Hello everyone. Welcome back to
01:00:45
the next episode of
01:00:46
Simplyblock Cloud Commute.
01:00:48
I'm really happy to have you all
01:00:50
back and I'm really happy to have
01:00:52
my guest for today.
01:00:54
Cris, I'm really sorry I don't
01:00:56
try to pronounce your last name.
01:00:57
I can only fail.
01:00:59
I mean, I guess, location-wise,
01:01:02
we're not too far
01:01:02
off from each other,
01:01:03
but I will not try that.
01:01:06
Maybe just introduce yourself.
01:01:08
Who are you? Where
01:01:09
you're from? What you do?
01:01:12
Yeah. My name is Cristian.
01:01:13
I'm based in Berlin,
01:01:14
but I'm originally from Romania.
01:01:17
I came to Berlin
01:01:18
more than 12 years ago.
01:01:20
Enough to be local, but not quite,
01:01:25
especially with my name.
01:01:28
It's hard for everybody to
01:01:30
pronounce it, but don't worry.
01:01:32
What I do is I have a lot of
01:01:36
background in this IT space.
01:01:39
I've been working
01:01:41
with AWS for many years.
01:01:43
Eventually, I entered this area of
01:01:46
cost optimization and had
01:01:50
some new stuff in that space built
01:01:53
over the last eight years now.
01:01:55
It's called AutoSpotting.
01:01:58
It's a tool for cost
01:01:59
optimization in AWS.
01:02:03
For a while, I actually worked at
01:02:06
AWS itself and then quit
01:02:08
AWS one and a half years ago to
01:02:12
work full-time on this.
01:02:14
I'm also helping customers with
01:02:17
cost optimization as a service.
01:02:20
It's just myself, so
01:02:22
I'm like a solopreneur.
01:02:25
I have an offering which comes
01:02:29
around just optimizing costs for
01:02:33
startups and
01:02:34
medium-sized companies.
01:02:35
I try to focus on
01:02:37
people who are more,
01:02:38
let's say, more agile,
01:02:40
more my state of mind as well.
01:02:45
With building tools as I do stuff
01:02:47
and offering these,
01:02:50
the entire thing as a service.
01:02:53
Right. You
01:02:54
mentioned AutoSpotting,
01:02:56
and I think that is your company
01:02:57
that's also the name of the tool.
01:02:59
Maybe you can
01:03:00
elaborate a little bit.
01:03:02
You mentioned it's in the cost
01:03:04
optimization space,
01:03:05
so how does that tool work?
01:03:08
Yeah. It's meant to orchestrate
01:03:11
the spot instances in AWS.
01:03:15
I'm not sure if your audience is
01:03:17
familiar with this topic,
01:03:18
but spot is just unused
01:03:20
capacity in the cloud.
01:03:24
It's not very far from the idea of
01:03:27
when you go on a
01:03:28
last-minute trip somewhere.
01:03:32
Imagine that the
01:03:34
hotel has fixed capacity.
01:03:38
Basically, they can
01:03:40
take a number of people,
01:03:41
but not always
01:03:42
it's at full capacity.
01:03:44
What they do is they give this
01:03:46
free capacity to other
01:03:48
people at a lower price.
01:03:50
They get it at the discount.
01:03:55
That's also what
01:03:56
I do in the cloud.
01:03:57
With spot instances, you actually
01:03:59
get this unused
01:04:02
capacity at the discount.
01:04:07
But basically, unlike the hotel,
01:04:11
when, for example, let's say your
01:04:15
hotel would get full,
01:04:18
the capacity, you would just be
01:04:20
told there's no
01:04:21
more free capacity,
01:04:22
but the cloud
01:04:23
providers, the way it works,
01:04:25
they try to have this illusion of
01:04:27
unlimited capacity.
01:04:29
From the cloud
01:04:30
providers' perspective,
01:04:34
it's a big problem if a customer
01:04:35
fails to provision some of the
01:04:37
capacity they need.
01:04:39
What they do is they always try to
01:04:41
have some spare capacity,
01:04:43
keeping it in a way that allows
01:04:46
anybody who wants
01:04:47
to spin up something
01:04:49
to be able to do it.
01:04:51
That gets them in a situation
01:04:54
where they need to
01:04:55
have a spare at all times.
01:04:59
And it's also like what happens is
01:05:04
you have a lot of
01:05:06
options to choose from.
01:05:08
So if you get back to the analogy
01:05:10
of the hotel space,
01:05:14
it's like you would imagine having
01:05:16
all the hotels in a big
01:05:18
city in a single chain.
01:05:21
And let's say there's an event and
01:05:24
you want to go there.
01:05:26
If one of them is full, you can
01:05:28
always be
01:05:29
redirected to another one.
01:05:31
So that's kind of how it works
01:05:33
with the spot instance,
01:05:36
when a certain
01:05:38
capacity pool gets full,
01:05:43
you can go and get
01:05:45
capacity from a different one,
01:05:46
as if you would go to a different
01:05:48
hotel in case the
01:05:50
first one is full.
01:05:52
So that's kind of you can get this
01:05:54
still illusion of
01:05:57
unlimited capacity,
01:05:58
but by distributed over
01:06:01
multiple capacity pools.
01:06:04
And the tool that I have is
01:06:07
actually meant to orchestrate.
01:06:11
It was back in the days where this
01:06:13
spot service was
01:06:15
still at the beginnings.
01:06:17
But basically it was orchestrating
01:06:19
this diversification across
01:06:20
multiple capacity pools
01:06:24
in a way that's reliable so that
01:06:26
you always get that capacity.
01:06:29
Now, what happens in the cloud and
01:06:31
it's not happening
01:06:31
in the real world
01:06:34
is the capacity can be claimed if
01:06:39
somebody else needs it.
01:06:41
So imagine you would go to this
01:06:42
trip and you would
01:06:45
be told by the hotel
01:06:46
that somebody wants to pay the
01:06:47
full price for this.
01:06:50
And then next day you would have
01:06:51
to vacate your room
01:06:52
and go somewhere else.
01:06:55
Right, right.
01:06:58
Yeah, it's not something that can
01:06:59
happen in the real
01:07:00
world, but in the cloud,
01:07:01
they do this. So they give that
01:07:03
capacity to somebody else
01:07:05
who pays the full price.
01:07:06
And then you're outside
01:07:10
like with luggage.
01:07:13
But my tooling
01:07:14
would actually
01:07:17
find a replacement
01:07:18
spare capacity for you.
01:07:22
And if nothing is available, like
01:07:25
across all the options,
01:07:29
then it would find a non-reduced
01:07:32
fee capacity pool.
01:07:35
So then in the worst case, you
01:07:37
still get an expensive room.
01:07:39
But yeah, you get something
01:07:43
where to spend your day.
01:07:45
So that means when I
01:07:48
understood you correctly,
01:07:50
in your tool, I say I need a
01:07:52
capacity of I don't
01:07:53
know, like 50 CPUs
01:07:56
and that amount of RAM
01:07:57
over that many machines.
01:07:59
And you would try really hard to
01:08:01
find spot instances.
01:08:04
And and if for some reason,
01:08:08
AWS says I need like 10 of those
01:08:10
spot instances right now,
01:08:11
because somebody is paying the
01:08:13
full price or something,
01:08:14
you would just spin up other
01:08:16
instances somewhere
01:08:17
else and like, OK,
01:08:18
so we'll get the spot instances.
01:08:21
Yeah, that's kind of how it works.
01:08:23
It's just that the way it works is
01:08:25
it connects to your existing
01:08:27
capacity groups.
01:08:29
You know, in AWS is
01:08:30
called auto scaling.
01:08:32
So you have these groups of
01:08:34
typically identical instances
01:08:37
and AutoSpotting, you install
01:08:40
it in the account
01:08:42
and it can look at all your groups
01:08:45
as long as the
01:08:46
group has a certain tag
01:08:49
attached to it, the group will be
01:08:52
considered for this action.
01:08:55
So it happens entirely without you
01:08:57
having to configure anything
01:08:59
except for the tag that tells it
01:09:01
to look at the group.
01:09:02
So it will go to the group and
01:09:04
replace the instances one by one
01:09:08
with this spot clone instance.
01:09:11
So basically within the groups,
01:09:16
if it just runs and over a few hours,
01:09:20
one by one, the instances will be
01:09:22
replaced with the
01:09:23
cheaper instances.
01:09:25
So how do I have
01:09:27
to think about that?
01:09:29
If AWS says I need those
01:09:32
instances, do they
01:09:33
disappear in like seconds?
01:09:37
Yeah. So when they, when the
01:09:39
capacity is claimed back,
01:09:42
there is an event coming.
01:09:45
And basically when that event
01:09:46
comes, you have two minutes to
01:09:49
vacate your room.
01:09:51
Oh, wow.
01:09:52
And what I do in those two
01:09:55
minutes, I
01:09:56
find you a different one
01:09:58
and start a new
01:10:01
instance with a new capacity.
01:10:04
And then that will be terminated
01:10:06
after those two minutes pass.
01:10:09
You will be evicted from it.
01:10:13
Right. Right.
01:10:21
So your application needs to be
01:10:24
able to sustain
01:10:26
these interruptions.
01:10:28
Like if you think of it,
01:10:31
that the application is running on
01:10:33
multiple identical
01:10:34
instances in a group.
01:10:36
So those groups may scale
01:10:39
up and down based on the load.
01:10:43
And then when they scale up and
01:10:44
down, it means that
01:10:46
it has to be somewhat
01:10:49
flexible with the capacity.
01:10:51
So it's like stateless, basically,
01:10:54
it should be stateless.
01:10:56
And if the group is stateless and
01:10:57
many of them actually are,
01:11:00
then this entire process is
01:11:02
seamless for the user.
01:11:04
So like there is a load balancer
01:11:07
in front of the group
01:11:08
that sends your traffic.
01:11:11
And then the like the users will
01:11:13
not notice anything,
01:11:16
because you still get some
01:11:18
some replacement.
01:11:19
Right. And that means if I
01:11:22
see that correctly, as a developer
01:11:24
building an application, I don't
01:11:26
have to do anything special.
01:11:27
As long as my application is like
01:11:28
cloud-native and
01:11:30
basically scalable,
01:11:33
I'm good. Right?
01:11:36
Yeah, that's the way it works.
01:11:37
And I mean, that's that's the
01:11:39
whole idea how how people can
01:11:41
actually use this
01:11:42
reliably. And I have people who
01:11:44
use it in production like
01:11:47
as long as the application is
01:11:48
built in this
01:11:49
way, it's a great way
01:11:51
to save money for companies. Yeah.
01:11:54
And from what I heard, it also
01:11:57
works together with the automatic
01:11:58
scaling that AWS gives you.
01:12:00
It's just that in the in the worst
01:12:02
case, your tool
01:12:03
steps in, and interferes,
01:12:05
and says, I don't want that
01:12:07
instance. I want that.
01:12:08
Is that correct?
01:12:11
Yeah, that's
01:12:12
exactly the way it works.
01:12:13
So it works with these
01:12:16
API calls to attach and detach
01:12:18
capacity to the group.
01:12:20
So what I do, I spin up an
01:12:22
instance, attach it to
01:12:23
the group and terminate
01:12:26
and detach the
01:12:27
existing one.
01:12:28
Right.
01:12:31
Yeah, it allows me to keep the
01:12:33
group configuration untouched,
01:12:34
which is important
01:12:36
for customers where maybe they
01:12:38
have a legacy application.
01:12:39
They don't want to touch or yeah,
01:12:43
could be for whatever reason,
01:12:45
easier to to just work from
01:12:47
outside the configuration.
01:12:49
Rather than maybe reconfiguring
01:12:52
100 groups one by one,
01:12:55
because the entire
01:12:56
configuration is automated.
01:12:59
But does AutoSpotting also do
01:13:00
like the actual
01:13:01
scaling or is that still
01:13:03
the scaling group from AWS and
01:13:05
you're basically just exchanging?
01:13:07
That's that's the group's job, so
01:13:09
I don't interfere with that.
01:13:11
But when the group spins up the
01:13:12
new scale, the new
01:13:14
instance that it scaled out,
01:13:16
then I immediately replace it.
01:13:20
Interesting.
01:13:21
Yeah.
01:13:23
Okay, cool.
01:13:24
That is that is very cool.
01:13:25
So I guess.
01:13:28
Boldly said the biggest the
01:13:29
thing is the cost
01:13:31
efficiency for the customer
01:13:32
because I guess the spot instance
01:13:34
are way cheaper
01:13:38
than regular instances.
01:13:42
Yeah, I mean, you get around,
01:13:43
let's say it depends a
01:13:45
lot by instance type.
01:13:48
The prices go up and down based on
01:13:50
supply and demand.
01:13:52
There are times during the year,
01:13:56
where people tend to need more
01:13:59
capacity and then
01:14:00
the prices go up.
01:14:02
And then other times when they go
01:14:04
back down.
01:14:05
It's like a marketplace.
01:14:08
So yeah, typically the savings are nowadays
01:14:13
around 60-ish percent. 50-60%
01:14:16
depending on the
01:14:18
instance type and region.
01:14:20
And yes.
01:14:21
So that means it basically gives
01:14:23
you the same
01:14:25
cost cut as a three years
01:14:28
reservation without
01:14:29
the actual reservation.
01:14:31
That's actually really cool.
01:14:33
Wow.
01:14:33
And the benefit is that
01:14:35
if you don't need it,
01:14:37
you just stop it and you don't
01:14:38
pay for it.
01:14:39
Whereas the
01:14:39
reservation is flat fee per hour.
01:14:43
So I mean, I have a customer right
01:14:45
now who's working in
01:14:46
the stock market and they
01:14:49
have this daily traffic pattern
01:14:53
like during the stock
01:14:54
exchange open hours.
01:14:56
They have a lot of traffic and
01:15:01
they spin up
01:15:03
capacity and then at the
01:15:04
end of the day, they can shut it down.
01:15:07
Whereas if you're paying a savings
01:15:09
plan, you would provision the
01:15:11
savings plan for
01:15:11
the whole like 24 hours and you
01:15:15
would definitely get savings.
01:15:18
But if you consider that maybe you
01:15:19
only need that capacity
01:15:20
for the eight hours of
01:15:22
the stock market, probably would
01:15:25
be better to just just run it as on demand rather
01:15:29
than being buying a savings plan. If you cannot use
01:15:34
spot for that workload.
01:15:35
So yeah.
01:15:36
So that means from, from a
01:15:38
customer's perspective or from a
01:15:39
customer's group
01:15:40
perspective, I think there is like
01:15:43
the group that is
01:15:44
perfectly fine with
01:15:45
reservations.
01:15:46
They have like a fairly consistent
01:15:47
kind of load on
01:15:49
the system capacity.
01:15:51
And for them, it's great.
01:15:53
And they, they can look into the
01:15:54
future like, yeah, we
01:15:55
need those systems or we
01:15:56
need that capacity in a year
01:15:58
and in three years. It's all good.
01:16:00
We only scale up.
01:16:02
And then there's the kind of
01:16:04
people or the kind of
01:16:05
companies, like you said,
01:16:06
like stock market or any kind
01:16:09
of like a very regional
01:16:11
online shop or whatever
01:16:12
where you have like high hours or
01:16:14
high load hours,
01:16:16
low load hours.
01:16:19
And I think those are perfect
01:16:21
candidates for AutoSpotting.
01:16:24
Is that correct?
01:16:26
Yeah.
01:16:27
I mean,
01:16:29
I have customers from all sorts of
01:16:30
verticals, but
01:16:32
it's very
01:16:33
important if you can
01:16:36
follow the traffic
01:16:38
pattern of the users.
01:16:40
If it's flat capacity, I
01:16:43
mean, you can as well just
01:16:45
purchase the three
01:16:46
year all upfront and you get a bit
01:16:48
better savings than
01:16:49
this. But yeah, this,
01:16:52
this doesn't have any commitment.
01:16:53
So, if you want to do some different
01:16:57
you have to flexibility to
01:17:00
just stop altogether.
01:17:02
And I think flexibility not
01:17:05
having this fixed commitment
01:17:08
that is really interesting
01:17:09
not only for production, but
01:17:12
I could also see that it's also interesting
01:17:15
during a research phase, a
01:17:17
development phase, stuff like that
01:17:19
where you have
01:17:20
a time where you really need more
01:17:22
capacity because now you're having
01:17:24
this new research project or you
01:17:26
have a bunch of developers working
01:17:28
on something else,
01:17:29
but then you have times where you
01:17:30
really don't need that capacity.
01:17:33
That's cool.
01:17:34
That's really cool.
01:17:37
So you said 60% or roughly 60% is
01:17:42
that virtual machines only or does
01:17:44
that also work for other things?
01:17:47
It's just the
01:17:48
virtual machines.
01:17:49
Okay, just the VMs.
01:17:50
That's fair.
01:17:51
Still, I think a
01:17:52
good chunk of money.
01:17:54
For storage and databases, I have
01:17:56
different tools.
01:17:57
For storage, I have something that
01:17:58
converts between the storage
01:18:00
volumes, GP2 to GP3.
01:18:03
And for databases, I have a tool
01:18:05
that converts the instance type to
01:18:09
the ARM Graviton instance types
01:18:11
and also does a right sizing in
01:18:13
the process of conversion.
01:18:15
So if you have an overprovision
01:18:16
capacity, if we look at the
01:18:18
metrics and say, okay, this is
01:18:19
maybe too big, let's
01:18:21
use the smaller one.
01:18:22
Okay, so that certainly sounds
01:18:25
like you have to come back talking
01:18:26
about those kinds of things.
01:18:27
That's really cool.
01:18:30
And I think you install the tool
01:18:34
into your own AWS account.
01:18:35
I think that is
01:18:36
what you said, right?
01:18:37
Do you do this
01:18:37
from the marketplace?
01:18:40
Yeah, it's available
01:18:41
on the marketplace.
01:18:43
Previously, it was open source, so
01:18:45
you could just get it from GitHub.
01:18:47
But after I left AWS, I'm kind of
01:18:49
full time on this and
01:18:51
trying to make a living.
01:18:52
That's fair.
01:18:53
Yeah, I try to get some revenue
01:18:56
out of it, but I'm not charging.
01:18:59
I mean, the current version is
01:19:00
charging 5% of the savings.
01:19:03
I'm going to increase the pricing
01:19:04
a bit in the next version, but
01:19:07
just that I'm trying to
01:19:09
get like some affiliates
01:19:12
to do marketing for me.
01:19:13
And with 5%,
01:19:14
there's not much to share.
01:19:16
That is true.
01:19:19
Yeah, but other than that,
01:19:23
I'm trying to be as
01:19:24
low cost as possible.
01:19:26
Right, right.
01:19:27
So you said new version,
01:19:29
anything you want to share about
01:19:31
that, anything exciting, you're
01:19:34
really looking
01:19:34
forward to have implemented.
01:19:38
I mean, the last version was like
01:19:41
six months ago, and over these
01:19:43
last six months, every time I have
01:19:45
a customer reporting
01:19:48
something, I implement it.
01:19:51
And basically, I added a bunch of
01:19:53
new interesting things, a lot of
01:19:56
work in the
01:19:57
efficiency of the software.
01:20:00
So it uses less memory from the
01:20:04
Lambda because it's
01:20:05
running in Lambda.
01:20:07
And yeah, there was also something
01:20:11
when it comes to deployment.
01:20:12
So when you have a deployment,
01:20:15
sometimes you spin up instances
01:20:17
during the deployment and you
01:20:19
don't want to interfere with the
01:20:21
deployment until it's over.
01:20:24
So there is a way
01:20:24
to logic for that.
01:20:27
So deployment in the sense of the
01:20:30
customer's application.
01:20:32
So if I deploy a whole new application.
01:20:34
Yeah, okay, that makes sense.
01:20:36
Yeah, so if the application is
01:20:37
deployed on newly launched
01:20:39
instances, sometimes they were
01:20:41
like, "Yeah, I don't want this to
01:20:43
be touched until the
01:20:44
deployment is over."
01:20:46
So it's some kind of a cool-off
01:20:47
phase or whatever
01:20:48
you want to call that.
01:20:49
Yeah.
01:20:50
All right.
01:20:50
Yeah.
01:20:50
Or maybe warm up phase, depending
01:20:52
on what the application does.
01:20:53
Okay.
01:20:54
Cool.
01:20:55
Yeah, I can see how this is very
01:20:57
useful for a lot of customers or
01:20:59
potential customers.
01:21:01
We're almost running out of time,
01:21:04
but I have one last question,
01:21:06
something I really, really asked
01:21:08
everyone so far.
01:21:10
Like, what do you
01:21:12
think is the new trend?
01:21:13
What do you see is the big future
01:21:15
we're heading to?
01:21:18
I mean, over the last year
01:21:22
or so, it's pretty clear it's the
01:21:24
AI and everybody's doing
01:21:28
something in that space.
01:21:29
I mean, it could be as well, just
01:21:31
like the dot com bubble.
01:21:35
The situation could also implode
01:21:38
at some point, but I see huge
01:21:40
gain out of using
01:21:44
the AI on my own work.
01:21:47
On a daily basis, I use ChatGPT for
01:21:50
software development.
01:21:52
So all my code over the last year
01:21:54
pretty much is
01:21:56
generated by the AI.
01:22:00
And yeah, I'm using it, not
01:22:03
offering AI tooling.
01:22:06
Like, you know, every application
01:22:08
now has an AI feature.
01:22:10
I don't have an AI feature in
01:22:12
AutoSpotting, but the way I build
01:22:14
AutoSpotting
01:22:15
evolved using the AI.
01:22:18
So yeah, that's kind
01:22:20
of the trend that I see.
01:22:24
And I think that
01:22:25
makes a lot of sense.
01:22:26
Yeah, that makes a lot of sense.
01:22:28
It's interesting that you say you
01:22:29
build with ChatGPT.
01:22:31
I have mixed
01:22:33
experiences with that.
01:22:34
What I found for all of those
01:22:37
tools, no matter if it's like a
01:22:38
ChatGPT or what is it, CoPilot or
01:22:42
any of those tools,
01:22:43
they really work great when you
01:22:45
give them some source code and ask
01:22:48
them, like, hey, do you have
01:22:49
recommendations?
01:22:50
Like having a virtual
01:22:53
pair programming partner.
01:22:56
That is absolutely amazing.
01:22:58
Not everything is correct.
01:23:00
And sometimes you're like, yeah,
01:23:02
okay, it's maybe
01:23:03
not what you want.
01:23:04
But it's much better.
01:23:06
And I got some pretty decent
01:23:08
results with that
01:23:10
kind of use case.
01:23:12
Anyway, as I said, we are
01:23:14
unfortunately out of time.
01:23:15
20 minutes is super short.
01:23:17
But really, really interesting.
01:23:20
As I said, you probably have to
01:23:21
come back for the other tools when
01:23:23
you want to talk
01:23:23
more about those,
01:23:25
because they also
01:23:25
sound super interesting.
01:23:30
And yeah, thank
01:23:31
you for being here.
01:23:32
It was a pleasure.
01:23:34
Thanks for having me.
01:23:35
And it was also a pleasure for me.
01:23:38
Yeah.
01:23:38
And for the audience, I hope you
01:23:41
come back next week and you listen
01:23:43
in again and with some more cool
01:23:46
audience and guests.
01:23:48
Thank you very much.
01:23:49
Thank you so much.
01:23:52
The cloud commute podcast is
01:23:54
sponsored by simplyblock, your
01:23:55
own elastic block storage engine
01:23:57
for the cloud. Get higher IOPS and
01:23:59
low predictable latency while
01:24:00
bringing down your
01:24:01
total cost of ownership.
01:24:03
www.simplyblock.io

