Integration Tests Done Right: Testcontainers | Oleg Šelajev

[00:00:00] You get all of that, well, kind of for free. Even if you have to configure the test containers, you don't have to deploy an actual Postgres somewhere. It all happens. And you also have probably like one Postgres per test or per test suite or whatever. Yeah, exactly. So as I mentioned, as a developer, you have the full control over the environment. You can create your databases. You can also create separate environments for like a slice of tests, right? Like very often you have to test negative use cases, hard to set, hard to test within like a static infrastructure because you're literally breaking things. Hello, everyone. Welcome back to this week's episode of Simplyblock's Cloud Commute podcast. Like many weeks, this week, I also have like a guest that I know for quite a while. It's been many years. But I think we're going to talk about that in a second. Welcome Oleg. Great to have you on. Maybe just say a few [00:01:00] words about yourself. Who are you, where are you from, how did you end up here? Hi, Chris. Thank you for having me. I'm very, very happy to be on the podcast with you. Yeah. Hi everyone. My name is Oleg and I work as a developer advocate at Docker. And my story of coming here is very, very simple. I was working for a long time on the test containers projects, helping the integration tasks be simpler. And I was on the DevRel team there. And then we joined Docker in December last year. So my currently- I'm still doing DevRel. My focus is developer productivity, test containers is a big part of that. Dabbling in AI lately as well as everyone. I think a very large and interesting topic. And I have a personal take on it, but let's not start with it. [00:02:00] All right. And I have- If you're going to ask me technical questions, my preferred language of choice would be Java to respond. I think it's a very good enterprise popular language. And dabbling in some others as well, but not so proficiently. Right, right. And we know each other from like way before that, when you, when it was still ZeroTurnaround, which was later acquired by another company. Yeah, I don't remember. Yes, yes, we, like, my professional path started quite, quite a long time ago. And at ZeroTurnaround, actually I joined the DevRel team. So when I moved from the being a product engineer to the marketing team, that was I think more than 10 years ago, which is like it is weird to say that, but like it's [00:03:00] quite an experience in the field. I know what you're saying. Normally when people ask me, like how long do I do that? I was like 10 plus years. I stopped counting because at some point you feel, start to feel old. I've heard that you should say a decade because it has gravitas. You know, like I have a decade of experience in developer relations. A decade plus. Yeah. Awesome. I probably need to switch to that. So you talked about test containers. Just a brief introduction to test containers for the people that don't know, because it is an amazing tool. Yes. So test containers are open source libraries and they give you the API to configure, run, manage the life cycle and use applications and services in containers. It gives you the programmatic access to containers that you can run in your Docker environment. And that's what the [00:04:00] library is, right? And the most natural use case for that to actually, well, use containers from code is to create environments for your applications to run. Namely, you would like to do that in your application lifecycle early on to create a local development environment, right? Spin up your necessary services, spin up your database, spin up your, I don't know, Elasticsearch cluster, your Kafka broker, your local stack to model the AWS, or any other services that you are just off the shelf services or internal. And you do that for your application or you do that for your application tests, which was the classic use case, hence the name test containers. The containers, the normal containers, but they are usually used for creating test environments. Now that gives you a number of opportunities and benefits. Your setup is with the code, so it will never get stale. It will [00:05:00] always be up to date with the requirements of your application. You can programmatically create more of them. They are ephemeral setups. Test containers handles the cleanup. So you can, as a developer, you have the full power over the environment. So you don't need to, rely on the pre provisioned third party static services. And the most important one is that this setup that your application runs, it will be completely the same in your CI environment and locally. That means that, well, and it will also use the actual technologies used in production, right? So you get this environment parity. And that means that you can run the application environment in environments where you trust that it's similar to production, so we can verify the behaviors, and like high level behaviors pretty easily, and you can reproduce them in CI and locally. So, it gives you tremendous power and it shifts this tremendous power into the hands of [00:06:00] developers. So we call, so if you are into like high level terms, it's shifting left integration tasks. And that's it. It's an open source library and people love it. So pretty much very, very established in the Java ecosystem, where it started. I think the original test container supplementation was Java. And in Java ecosystem, for example, it's also integrated in application frameworks. So things like Spring Boot, things like Quarkus, Micronaut, all have integrations with test containers where they would use test containers for you by default, right? You just want to say, you are saying, I want to use a database and you don't configure the database and they will be, Oh, it seems like you were using Postgres because of the drivers on the Classpaths and you run your application, but you haven't actually configured Postgres for me. I'm going to spin that up for myself, right? So it [00:07:00] gets you like out of the box, seamless integration, and takes this whole infrastructure setup from you as a developer. So you can focus on the actual application needs, your business value, and the rest is just magically happens to work for you. Oh, that is cool. So that means you literally don't even have to say, I want a test container with Postgres. It will literally just spin it up for you because you are using Postgres. Literally you don't need to say anything. It can spin up for you. And also you don't have to rely on the framework just only, right, like in, you are working in the team. So somebody usually will create those setups for you already, right? So you were as a new developer, for example, your time from cloning the project to getting it running is literally zero. You clone it, you run it, the environment is created for you. The application works. You can debug it. You can explore it. It's a really, really good way to [00:08:00] increase developer productivity. And that means from your perspective, it makes writing integration tests easier and probably better because you get all of that for, well, kind of for free, even if you have to configure the test containers, you don't have to deploy an actual Postgres somewhere. It all happens. And you also have probably like one Postgres per test or per test suite or whatever. Yeah, exactly. So as I mentioned, as a developer, you have the full control over the environment, right? So you can create your databases. You can configure the schema in the databases. You can, for example, in a sensible way, test whether your, like, database migrations will work currently, right? You can also, if you wish so, create separate environments for like a slice of tasks, right? Like very often you have to [00:09:00] test negative use cases, which are hard to set, hard to test within like a static infrastructure because you're literally breaking things, right? So with test containers, you can create an ephemeral environment to that break it as much as you want, verify how your application works when the schema is bad, or when the data is corrupt, or when the data is missing, or when your Kafka is running out of topics or whichever environments you want. You can add additional stuff at like, for example, introduce network effects into your tests, right? You can put a ToxiProxy in front of your database, and just during the test, say like, Oh, now my database responds like, with two second latency in every direction and see how the application handles that. So you can create the environments as easy as you want. You can create them per test, or per test method, or per test suite, or like one as a singleton for the whole run. And test containers will [00:10:00] take care of the initialization of containers, managing them, and then cleaning up at the end. So you can run the tests again and again and again in a very repeatable manner without connecting to this stale kind of pre configured services. It gives you this ability to create the environments and it gives you the ability to put it into application code and then it will sit there and you will use it locally, all, everyone on the team and in CI the same way, right? So you can run anything that runs in the Docker container. And you can configure anything the way you want. It could be your internal Docker images or the images of your internal services or your internal way how you would like to run the particular technology. But also on top of that, because in the code, you are just capable of abstractions, right? The code is good for abstractions. So what you can do, you can take and pre configure a way to [00:11:00] run certain technologies. And those are the test containers modules, which we have the whole ecosystem of. There we go. So we talked a little bit about the power of test containers, about how you can get the repeatable environments in CI and locally, and that you can configure and run anything, right? So what's the best way is to get started with test containers, for example, or like normally as a developer, you don't want to figure out how to run a particular thing in the Docker container. Like it's just not where your expertise should lie. Like you don't need to figure out how to run a, like a RabbitMQ broker or Kafka broker, or like how to run an emulator for the Google cloud, like services, right? So [00:12:00] that's why we have the ecosystem of test containers modules, the pre configured abstractions, which have the default weight. So there is a test containers wrapper, and then there is the actual Docker image that will model the service that you want to use. And those are available to you in just a few lines of code. You add a library to your project. You say you get it on your classpaths. You say like, Oh, new Kafka container or a new like elasticsearch container, or just literally in one line. And you configure it the way you want it to be configured. But most normally, so like, most normally default configuration will be sort of sensible, right? So that means as a developer, you can get started and create environments with technologies that you don't actually know how to run locally, or you might not even know how to run them in Docker, because you are using this higher level abstraction that sits in your code, which is super, super [00:13:00] good for developers because you can explore that naturally, right? You can type a dot in your ID. Your ID will suggest what you can do with this. Right. And this abstraction, this test containers module creators could put the API to configure useful things, right? Like you can, for example, configure any sort of timeouts or enable some features, which normally you need to do by providing environment variables or putting certain files in certain places, which is all kind of like, a little awkward. Right? Well, it is. It's very good for production because like there you can have access to all the knobs and all the settings, and you want to maybe create your YAML file in a specified way. And then you want to see the config that is diffable, right? And then all that. But for development, you normally just want, I want the service, right? I want my Kafka to be there to accept and send me the messages instead of being super sure, like, what's the, like, flush timeout [00:14:00] to the disk. Right. Cause at worst, if it all fails and it all crashes, right, at worst, your test fails, and then you need to rerun this. There were no production data on the line. There were no like transactions on the line. There were no users who will be upset. It's just that it's an inconvenience. But it is a minor one compared to what can happen in production. So, but so the test continuous modules are an amazing thing. And we have like a whole bunch of them for almost all imaginable technologies, yeah, that are popular. Right, right. You already mentioned Java. I know there's Go because I use test containers with Go before. But there's other languages, Python. Yes, there are the- Most of the popular languages are covered. There is test containers Golang, test containers.NET, test containers Node.js, Python for sure, which is very, getting very popular lately as well. And there were also like, [00:15:00] so less, less popular in the enterprise languages. There is a test containers Rust. There is test containers Ruby. We have a Clojure, Elixir. I think we have a Haskell implementation. And also currently as we speak, there is work going on to adopt a test containers PHP implementation there- Most of those were created as community implementations, right, and the test containers maintainers. We try to, kind of sort of share our expertise and like get everyone under the same umbrella. So all the implementations in different languages share some vocabulary and work kind of sort of similarly, right? So if you are moving between languages, you would recognize the basic abstractions and you would be like, ah, I would like a test containers module here, right. So you can work effectively because it's a massive, massive [00:16:00] undertaking to have like some sort of, not even support, but like a system that kind of works for like 60, 70 technologies, right? It's the maintaining team cannot be experts in all of that. So there was a lot of reliance on the community contributions or partnerships with actual vendors where they say like, ah, we like test containers so much, we would like to collaborate and put engineering time together with your team's time and make, for example, local stack test containers module. Really good for both integration tasks. And but also for making the best use of actual local stack, right? More, more, there is like, you know, Red Pandas. I think you actually put like a great example out. Local stack is for everyone's working on AWS and against AWS services, local stack is the thing to use. It is just amazing. And running it in test containers was like a blast. [00:17:00] So it is exactly one of the things I had to use. No, LocalStack team is super, super interesting, right? So they were- So the project is localstack.cloud. And they are building this like emulator, like a single machine emulator for AWS services. They have a ton of compatibility tasks. So it actually works the same way as AWS, but it's not distributed. And it's amazing. You can run it locally. You can like, if you want to download, run it locally. But you also can run it in Docker container with test containers. So it becomes part of your application code and your CI pipelines. So really, really interesting. And also specifically one thing is, right, there are lots of other services, cloud services that, as a developer, I would like, and I'm sure other people would like to test against locally, right? Like you never want to go and like create spare accounts or use your production account for testing. And [00:18:00] then it's not even about like actual paying for the cloud service, right? That's- Those are like some cents for like- It's not that expensive for accidental access, right? But like, it's just the- Something will break the, like, some configurations will linger and like you want this ephemeral environment. And I know LocalStack recently has created, the thing for Snowflake, right, which is the managed cloud database. Snowflake is a database, right? Or is it a data lake? I think it's both depending on who you ask. It's, yeah, yeah, it's a, so like it's a managed cloud service, right? And LocalStack now offers also something for Snowflake. So if you were using that and you were like, Oh, how do I test it locally? Please consider the LocalStack and maybe test containers combination there. That is interesting. I didn't know that they now move [00:19:00] further than the AWS ecosystem, which was already a massive undertaking. I think they support like, what is it? Like 80 to 90 percent of all the services, everything that kind of makes sense. It's completely insane. Apart from that, what I mean, you said that basically all of the technologies are encapsulated in those test container modules. What do you think is like the most famously used module? Would it be local stack or is it Postgres? No, I- So I think, I don't think we publish this data very much, but we, it's a little bit hard to also see, cause there is- Those Are open source projects so there were no analytics built in, like what are people using? We do see some data from, for example, Maven Central about the actual, the amount of libraries that being pulled. But is also like, it doesn't correspond to the actual like usage usage, you know, [00:20:00] cause you only pull that once. And then probably that library would get into your internal artifactory or something. And then your whole organization is going to use that, but it's one download, right? So, but like there is some sort of like a sense which are popular. Postgres is massively, massively popular with test containers. And I also think because there are also very many technologies that are based on Postgres, right? So there are flavors of that. And if people internally create libraries to use those with test containers, they will probably extend the Postgres container configuration. They will extend the wrapper, and we'll add the additional functionality. For example, installing some extensions or something, right? So currently, like, for example, we have vector databases modules, right? Then there is a PG vector, which is literally Postgres with PG vector extension installed, and [00:21:00] of course the wrapper, their extent, like, so Postgres is a massive, massive open source project. It's not a secret that it's popular. All right. MySQL is popular. Kafka is very popular with test containers. I think in part is because it's actually kind of hard to run Kafka without, because there was actually a few things that you need to figure out. Like in the past you had to mess with ZooKeeper. Do you want ZooKeeper? Do you not want ZooKeeper, right? So if that configuration is for you hidden in the code, it's well, if you can just say, like, give me Kafka and boom, there is Kafka. It's a very important- Yeah, I think the Zookeeper problematic, was one of the reasons why people moved to, Red Panda or why Confluent or the Kafka team now implemented the k-raft or kraft or whatever mode. [00:22:00] No. So, but like, it's massively, massively popular. LocalStack is popular. But also, so you, spoke about Red Panda, but so Red Panda, I think one thing that was driving them, like what was separating them, right, the distinguishing was, because Red Panda is written in, I think either Rust or C, some native languages. Right. So it's- It had a smaller footprint and it had a faster startup, which was very, very good for tasks. And just recently, Apache Kafka project got, bear with me, bear with me, a Docker images of Apache Kafka compiled with GraalVM native image to the binary, right? Sure. There are too many images, image words in that description, but it's literally, it's Kafka compiled to native image, right? And you put it in a container, and then it spins up very quickly and it doesn't use as much resources, right? It's great. [00:23:00] And it's super good for testing specifically because, I didn't know whether you are adventurous enough or trust the technology enough to put that into your production, right, that out of the box. But for your tests, it does wonders because you create those environments on the fly, right? And if say your Kafka containers start not like 10 seconds, but one second, for example, right? Then you were like, it adds up throughout your test suite, right? So like you can be, it can be very, very good. And it makes me personally very, very happy seeing the synergies between multiple projects that I have been involved with through my professional life coming together in this very successful synergy. I hear that. It's the same when you move from startup to startup and you always see the same customers coming back, because they love to be on the edge and trying out new things. It's a great feeling. It's just an amazing feeling. So we crossed the 20 minute [00:24:00] mark. For probably a yes or no answer is sufficient for the next one. Originally TestContainers was created by AtomicJar, which was, acquired by Docker a year and a half ago or two years ago, something. So originally those were the community projects. And then, a few years back, the maintainers of the Java fork, Java, like version of test containers, decided to create a company and concentrate on dealing products around to actually help adoption. And that's how test containers cloud product was born, created, right, for the backend for running containers when you run tests. And yes, in the December of last year, Docker has acquired AtomicJar. So now the whole test containers team is at Docker and integrated into the larger Docker team, which is great. Test containers always relied on Docker environments. So [00:25:00] it's a, I think it's a very strategic move actually for the technologists. Right. So the Docker brand and the company is much, much larger. And them saying like, Ooh, we love this technology. Go says a lot. Right. And also I think it's very, very good for the open source projects as well, because we can get now direct access to the upstream. Runtime bits teams, right? That we can be like, Oh, maybe there could be some performance optimizations. Maybe we can figure out how test containers can work better with compose files. And then- So, it's a, I think it's a very, very positive move for both parties involved. And then we are continuing working on test containers at Docker now. Okay. So, it was a little bit longer than a simple, yes, but okay. Last question. The thing that I always ask as the last question, what do [00:26:00] you think is like the next big thing. What do you see in the future? Could be test containers related, you already mentioned your current stake is a lot of AI and your current interest. So what do you think? So I think it's a fair bet to say that AI, even though the hype is like growing down a little bit currently, I think AI is here to stay with us. Right. And I think that maybe we will get slightly disillusioned in like this AGI, if you will, like, or like generic AI being used as a hammer for all problems, right? But I think that this democratization of AI technologies will continue. And sooner or later, we're going to start using AI components, whether those are LLMs or sort of normal machine learning models or other [00:27:00] things. We're going to start using AI components in our applications to implement some functionality that is hard to implement otherwise. Right. And it could be something fuzzy and small, very contained, right? For example, we're going to be like, your boss comes to you in the bank and says like, we need to look at the credit card transaction logs. It's like, check whether the next one is fraudulent. Do we need to flag it for somebody to look at it? Right. You can write this as a bunch of if else statements, probably, right? Like if the next transaction is like 5,000 miles away from the previous one, maybe look at it, right. But like, it's also hard to grasp that. So maybe there could be in the, like a little model that looks at that input and gives you like a response. And I think there are many problems in software like this, where they're fuzzy enough to being hard to be described in like, just like code, and I think this democratization of AI. Like they're also maybe not the most suitable [00:28:00] for being answered by just say chat GPT or Gemini or any other large language model, maybe that you can. But also probably it's not the most efficient. So I think the AI components that we use in the application, they're going to get dumber down. But the trend of application developers using more AI within their applications will continue, right? And then, so this is very interesting because then application developers need to learn how to work with AI, right? What does it mean? Like, how do you test those things when you have no idea what's happening? What's your security stake on that, right? How would you, what's your, how do you do monitoring? You know how like everyone says like, Oh, AI got dumber suddenly, last week it was so good and now it's like giving me dumb answers. I don't know what is happening. Maybe it's just the service providers turning down the GPU, GPU use a little bit, right? But also maybe because the inputs changed or something changed. Like what do you monitor for in production? [00:29:00] Because when you're going to start building applications with AI, right, your DevOps cycle will change somehow. And it is yet unclear how, but I think it's very, very interesting. And I think it will happen in some form or shape. So if you are currently an application developer, I don't want you to jump on the hype train and be like, Oh, AI is going to write code for me. But rather, eventually you're going to use AI instead of parts of your code for some things. And I think it's a fascinating technology currently to explore. I think that is a perfect closing. Right. So, thank you very much for being here. It was awesome. I think we hit like 28 minutes or something, so that's the perfect timing, at least in my book, I think. Thank you so much. And for the audience, same, same time, same place next week. And I hope you come back and thank you [00:30:00] very much as well for being here. Smash that subscribe button and, yeah, thank you for having me. It was a blast. If anyone has any questions, I'm sure there will be links below. Join, ping me. I'd be happy to talk to you about AI, developer productivity, or like life in general. Perfect.