On the 24th of April 2024, I gave a presentation about Decoding Kotlin at the JetBrains Headquarters in Amterdam, located close to the RAI:
Kotlin MeetUp Amsterdam @ JetBrains! Kotlin Notebooks, Decoding Kotlin & more!
I had a blast on that day and it was great to have met everyone, the different presenters, the organizers and of course the audience. We targeted our presentations to last 30 minutes and in general we succeeded in that. For my presentation it would have also been great to have been able to complement it with practical live examples. But that would have meant that the session would have had to go up to about 1 hour to be honest also considering the buffer needed to answer questions for the audience. If I consider the feedback I got from the audience, I have good confidence that I made my points across.
With this article I want to complement my presentation by explaining in detail the different examples I showed in the slides:
Before we continue I would recommend to checkout the repository called kotlin-mysteries on github. This is the project that supports this article and you can find it under my username jesperancinha over here:
Nullability - Whenever possible?
This is actually a hint to think a bit about what null-safety means in Kotlin. So let's recap about that:
Kotlin promises a guarantee of null-safety. Although we can use nullablemembers in our classes, we really shouldn’t whenever possible.
In practice null-safety in Kotlin means that we can guarantee that certain variables or class members are either nullable or non-nullable throught the course of the running code. I allows us to better predict our code and better localize NPE's (NullPointerException) whenever they occur. While this is true, null-safety, currently only works if we talk about assigning values to variables. There are however lots of examples where we can set the values, which has a subtle difference to assign a value. So let's have a look at an example that people will eventually run into when working with Kotlin and Spring on the server-side.
Let's have a look at the example. For our example we need a database. I provide that in the carparts-database-service. In this example I create a database schema called carparts and another called carparts-data-structures with a simple access using username admin and password admin. The database is created using a commonly known bash script that I provide in that directory called create-multiple-postgresql-databases.sh. Before we continue, let's just start the database and for that let's just use one of these commands:
docker-compose up -d
or just the simplified Makefile script using:
make dcup
Once we have done that, it is probably best to start the service from your IDE. I use Intellij for that. The service is located at the directory carparts-manager on the root of the project. To start the service you can either run a simple command called gradle runBoot or just run the script on the Makefile I made for it and you can call it using make run. Remember that the service cannot start if the database isn't up and running.
Flyway will run and it will execute this script first:
CREATE SEQUENCE car_parts_id_sequence
START WITH 1
INCREMENT BY 1
NO MINVALUE
NO MAXVALUE
CACHE 1;
CREATE TABLE CAR_PARTS (
id BIGINT NOT NULL DEFAULT nextval('car_parts_id_sequence'::regclass),
name VARCHAR(100),
production_date timestamp,
expiry_date timestamp,
bar_code BIGINT,
cost float
);
This script creates a simple database similar to what we may find in production environments these days. It is important to notice that all of the columns, except for the id column can be null in this case. Pay special attention to the column name. This is very important for this example.
The second script to run gives us some data and it is this one:
INSERT INTO CAR_PARTS
(name, production_date, expiry_date, bar_code, cost)
VALUES ('screw', current_date, current_date, 12345, 1.2);
INSERT INTO CAR_PARTS
(name, production_date, expiry_date, bar_code, cost)
VALUES (null, current_date, current_date, 12345, 1.2);
We can observe now that perhaps by mistake or any other reason, we didn't put a name to the second car part. Instead, we are only forcing it to be null.
In our example a simple way to map this database to our Kotlin code is to create an entity like this one in its very, very basic form:
@Table(name = "CAR_PARTS")
@Entity
data class CarPart(
@Id
val id: Long,
val name: String,
val productionDate: Instant,
val expiryDate: Instant,
val barCode: Long,
val cost: BigDecimal
)
This is a typical first approach to mapping the entity. We simply define it also to be a table under a specific name, in this case CAR_PARTS, we mark it to be an @Entity and then finally, we declare all the columns in the form of Kotlin types. They are all type safe, immutable and non-nullable class members. But if you recall from before, we did put a null value into the second element of our table CAR_PARTS.
But before we play with our running service, let's create a simple test to checkout what happens thinking that somehow, someway, we will get both parts. One example of such test could be this one:
class DomainTest : ContainerTest() {
@Autowired
private lateinit var carPartDao: CarPartDao
@Test
fun `should mysteriously get a list with a car part with a name null`() {
// For illustration purposes only.
// It DOES receive a null value.
carPartDao.findAll()
.filter {
it.name == null
}
.shouldHaveSize(1)
}
}
On this test we are only checking if we will get any element of the a list coming from findAll that has a null value for name, which seemingly should not work.
What could be the result of such a test?
In most cases, people coming for the first time starting to code in Kotlin will not even think of such possibility given that Kotlin applies null-safety as one of its core principles. Having said that, we frequently don't really phrase out exactly what that means. Null-safety means that we can predict just by looking at the declaration of a class member of a variable, that its type is nullable or not. But this applies only to assigning. Having this in mind let's run this test and checkout the result:
IntelliJ will let you know that the compiler finds the assertion we are making completely useless. However, as we can see from the resulting test, the non-nullable class member called name now carries a null value. This is something that, if you didn't know about it, you probably didn't expected this. The reason for this is that val and non-nullables is something that the JVM just doesn't understand and with reflection we can still do whatever we want with variables. Under the hood, in its depths, that is what frameworks like the Spring Framework use to be able to perform IoC, validations and use AOP (Aspect Oriented Programming).
From this, we can imagine that using a Spring MVC design pattern the runtime will fail, but not exactly while we read the data from the database. MVC usually implies that, at the service layer, calls to converters from entity to dto and vice-versa will take place. It is here where the problem starts. For many that do not know about this phenomenon, this is exactly where the problem begins. We have learned about null safety, but now we get a difficult to interpret NPE at the service level, where we do not expect an NPE at all given our first impression of what null-safety means. Many people leave investigating if the data is actually being returned null from the database simply because that is exactly what they do not expect.
To illustrate this problem I have created a service with an explicit use of a try-catch exactly during conversion and that logs this when we make a request to list our data. The idea is to log all errors and exclude that data that cannot be converted from an entity to a Dto:
fun CarPart.toDto() = try {
CarPartDto(
id = id,
name = name,
productionDate = productionDate,
expiryDate = expiryDate,
barCode = barCode,
cost = cost
)
} catch (ex: Exception) {
logger.error("Failed to convert data!", ex)
null
} finally {
logger.info("Tried to convert data {}", this)
}
We can now make a call to our running service to the list all endpoint. The implementation looks like this:
@RestController
@RequestMapping
class CarPartsController(
val carPartsService: CarPartsService
) {
@PostMapping("create")
fun createCarPart(@RequestBody carPartDto: CarPartDto) = carPartsService.createCarPart(carPartDto)
@PutMapping("upsert")
fun upsert(@RequestBody carPartDto: CarPartDto) = carPartsService.upsertCarPart(carPartDto)
@GetMapping
fun getCarParts() = carPartsService.getCarParts()
}
And to make a request, we can just use the scratch file I have placed at the carparts-manager service module:
###
GET http://localhost:8080/api/v1/carparts
With this request, we should get a response like this:
[
{
"id": 1,
"name": "screw",
"productionDate": "2024-04-25T00:00:00Z",
"expiryDate": "2024-04-25T00:00:00Z",
"barCode": 12345,
"cost": 1.2
}
]
This represents only the car part that actually has a name. The other one that didn't succeed during the conversion gets filtered out, however, nonetheless still gets logged out to the console:
java.lang.NullPointerException: Parameter specified as non-null is null:
method org.jesperancinha.talks.carparts.carpartsmanager.dto.CarPartDto.<init>,
parameter name
at
(...)
2024-04-25T21:13:52.316+02:00 INFO 151799 --- [carparts-manager] [nio-8080-exec-1] o.j.t.c.c.converter.Converters
:
Tried to convert data
CarPart(id=2, name=null, productionDate=2024-04-25T00:00:00Z, expiryDate=2024-04-25T00:00:00Z, barCode=12345, cost=1.2)
And here we can see that the data is coming through to our runtime anyway. It is only at the moment that we try to assign a null value via our own code, that we get a NPE, which is, in this case, at the time we try to convert our entity to a DTO.
The Spring Framework, like many other frameworks, resort to reflection to be able to inject values into the field and class members of our instances and reflection is a part of how these frameworks handle objects. But to focus precisely on where this phenomenon happens we can have a look a at a simpler example that shows a direct way of setting a null value into a data class in kotlin:
data class CarPartDto(
val id: Long,
val name: String,
val productionDate: Instant,
val expiryDate: Instant,
val barCode: Long,
val cost: BigDecimal
)
This example can be found on module carparts-null-play. In this class I declared all class members to be non-nullable, but, as mentioned above, we can still set a null value to name:
fun main() {
val carPartDto = CarPartDto(
id = 123L,
name = "name",
productionDate = Instant.now(),
expiryDate = Instant.now(),
cost = BigDecimal.TEN,
barCode = 1234L
)
println(carPartDto)
val field: Field = CarPartDto::class.java
.getDeclaredField("name")
field.isAccessible = true
field.set(carPartDto, null)
println(carPartDto)
assert(carPartDto.name == null)
println(carPartDto.name == null)
}
If we run this program, we will get this result in the console:
CarPartDto(id=123, name=name, productionDate=2024-04-26T06:45:51.335693902Z, expiryDate=2024-04-26T06:45:51.335696975Z, barCode=1234, cost=10)
CarPartDto(id=123, name=null, productionDate=2024-04-26T06:45:51.335693902Z, expiryDate=2024-04-26T06:45:51.335696975Z, barCode=1234, cost=10)
true
And this just proves the point that although null-safety is a big a concern in Kotlin and it makes the code more readable and more predictable, it doesn't play a role in setting a value, while it does play 100% a role in assigning a value.
There are multiple ways to approach this problem, when it arises, especially when we are using frameworks that provision CRUD interfaces and it is a good idea to just mention a few of them:
Database Migration - If possible, make sure that there are no more null values in the database and create a constraint in the table itself to prevent nulls from being create
Handle the null values earlier - If nulls aren't expected, we could manually handle these null's. The downside is that your IDE will probably keep signalling a warning and you'll have to supress those warnings one way or the other if so.
Use another framework - This is probably a more costly operation and it is not always clear if using another framework will solve this specific problem
Inline and crossinline - Why does this matter
You probably have already heard of inline in Kotlin. If you haven't then understanding it is quite easy and to explain it, we can say something like this:
Inline and crossline can be used incombination with each other. Inline creates bytecode copies of the code per each call point and they can even help avoid type erasure. Crossinline improves readability and some safety, but nothing really functional
In this in my experience the problem wasn't so much to figure out how the code should compiler and where to use crossinline, but the question was why was the compiler asking us, the developers, to use crossinline. For this I have created another example located on module located at the root of the project. There, we can find this example:
fun main() {
callEngineCrossInline {
println("Place key in ignition")
println("Turn key or press push button ignition")
println("Clutch to the floor")
println("Set the first gear")
}.run { println(this) }
}
inline fun callEngineCrossInline(crossinline startManually: () -> Unit) {
run loop@{
println("This is the start of the loop.")
introduction {
println("Get computer in the backseat")
startManually()
return@introduction
}
println("This is the end of the loop.")
}
println("Engine started!")
}
fun introduction(intro: () -> Unit) {
println(LocalDateTime.now())
intro()
return
}
In this example, we are simply creating a program that calls a higher-order inline function called callEngineCrossInline that passes on an argument that is a function and that get's called inside of it via another higher order function, which is not inlined and receiver a new function that calls startManually as part of its body. The code does compiler and there does not seem to be a problem with it and here, we are using crossinline. But let's think about this a bit more in detail. The compiler is going to try to compile this and when we decompile that into Java, it creates something like this:
public final class IsolatedCarPartsExampleKt {
public static final void main() {
int $i$f$callEngineCrossInline = false;
int var1 = false;
String var2 = "This is the start of the loop.";
System.out.println(var2);
introduction((Function0)(new IsolatedCarPartsExampleKt$main$$inlined$callEngineCrossInline$1()));
var2 = "This is the end of the loop.";
System.out.println(var2);
String var4 = "Engine started!";
System.out.println(var4);
Unit var3 = Unit.INSTANCE;
Unit $this$main_u24lambda_u241 = var3;
int var6 = false;
System.out.println($this$main_u24lambda_u241);
}
public static final void callEngineCrossInline(@NotNull final Function0 startManually) {
Intrinsics.checkNotNullParameter(startManually, "startManually");
int $i$f$callEngineCrossInline = false;
int var2 = false;
String var3 = "This is the start of the loop.";
System.out.println(var3);
introduction((Function0)(new Function0() {
public final void invoke() {
String var1 = "Get computer in the backseat";
System.out.println(var1);
startManually.invoke();
}
public Object invoke() {
this.invoke();
return Unit.INSTANCE;
}
}));
var3 = "This is the end of the loop.";
System.out.println(var3);
String var4 = "Engine started!";
System.out.println(var4);
}
public static final void introduction(@NotNull Function0 intro) {
Intrinsics.checkNotNullParameter(intro, "intro");
LocalDateTime var1 = LocalDateTime.now();
System.out.println(var1);
intro.invoke();
}
public static void main(String[] args) {
main();
}
}
public final class IsolatedCarPartsExampleKt$main$$inlined$callEngineCrossInline$1 extends Lambda implements Function0 {
public IsolatedCarPartsExampleKt$main$$inlined$callEngineCrossInline$1() {
super(0);
}
public final void invoke() {
String var1 = "Get computer in the backseat";
System.out.println(var1);
int var2 = false;
String var3 = "Place key in ignition";
System.out.println(var3);
var3 = "Turn key or press push button ignition";
System.out.println(var3);
var3 = "Clutch to the floor";
System.out.println(var3);
var3 = "Set the first gear";
System.out.println(var3);
}
public Object invoke() {
this.invoke();
return Unit.INSTANCE;
}
}
It is important to notice that callEngineCrossInline gets inlined up until the call to introduction and the function that we pass through via introduction gets also inlined. Now let' think about how would this have worked if we had used instead of return@introduction, something like return@loop or even return@callEngineCrossInline. Can you image how inline would have worked here? And if you can, do you see how complicated would that be to make generic for all kinds of functions or methods in Kotlin, that would make non-local returns? Neither do I and this is part of the reason why crossinline exists. In this specific case, if we do not use crossinline the compiler it will not allow us to build on the source code. It will mention that it is mandatory. But even if we try to make a non-local return in this case, the compiler will still fail saying that we are making a non-local return and so we will get these warnings respectively:
and
But the big question was, if the compiler already knows that non-local returns are a big problem in making inline code, or in other words, creating multiple copies of the bytecode per call point, why do we need to even put a crossinline before our parameter declaration? Maybe not in this case, but cross inline works like a standard and while in this specific cases it only guarantees an improvement in code readability, there are cases where crossinline actually has code safety functionality and for that I created this example:
object SpecialShopNonLocalReturn {
inline fun goToStore(chooseItems: () -> Unit) {
println("Walks in")
chooseItems()
}
@JvmStatic
fun main(args: Array<String> = emptyArray()) {
goToStore {
println("Make purchase")
return@main
}
println("Never walks out")
}
}
This example is looks quite easy to understand and the compiler will show no problems with this. It simulates the idea of going into a store, purchasing some items and walking out of the store. But in this specific example we don't really walk out of the store.
If you are using Intellij, odds are, that you are not getting any warning with this code. In this code we see that with return@main we are making a non-local return to the main function. The results in the println("Never walks out") never even being called. And if we decompile the resulting byte code, we will find something interesting:
public final class SpecialShopNonLocalReturn {
@NotNull
public static final SpecialShopNonLocalReturn INSTANCE = new SpecialShopNonLocalReturn();
private SpecialShopNonLocalReturn() {
}
public final void goToStore(@NotNull Function0 chooseItems) {
Intrinsics.checkNotNullParameter(chooseItems, "chooseItems");
int $i$f$goToStore = false;
String var3 = "Walks in";
System.out.println(var3);
chooseItems.invoke();
}
@JvmStatic
public static final void main(@NotNull String[] args) {
Intrinsics.checkNotNullParameter(args, "args");
SpecialShopNonLocalReturn this_$iv = INSTANCE;
int $i$f$goToStore = false;
String var3 = "Walks in";
System.out.println(var3);
int var4 = false;
String var5 = "Make purchase";
System.out.println(var5);
}
public static void main$default(String[] var0, int var1, Object var2) {
if ((var1 & 1) != 0) {
int $i$f$emptyArray = false;
var0 = new String[0];
}
main(var0);
}
}
The compiler doesn't create bytecodes for the println("Never walks out") call. It essentially ignores this dead-code. This is where crossinline may be used and can be very helpful. Let's look at a variation of this example where we do get out of the store:
object SpecialShopLocalReturn {
inline fun goToStore(crossinline block: () -> Unit) {
println("Walks in")
block()
}
@JvmStatic
fun main(args: Array<String> = emptyArray()) {
goToStore {
println("Make purchase")
return@goToStore
}
println("Walks out")
}
}
In this example, crossinline signas the compiler to check the code for non-local returns. In this case, if we try to make a non-local return, the compiler should warn you of an error, and if you are using IntelliJ for it, it will show something like this:
And in this case, we don't need to worry about the decompiled code or how this looks in the bytecode, because crossinline assures us that every call to goToStore will never accept a function that will return to @main:
public final class SpecialShopLocalReturn {
@NotNull
public static final SpecialShopLocalReturn INSTANCE = new SpecialShopLocalReturn();
private SpecialShopLocalReturn() {
}
public final void goToStore(@NotNull Function0 block) {
Intrinsics.checkNotNullParameter(block, "block");
int $i$f$goToStore = false;
String var3 = "Walks in";
System.out.println(var3);
block.invoke();
}
@JvmStatic
public static final void main(@NotNull String[] args) {
Intrinsics.checkNotNullParameter(args, "args");
SpecialShopLocalReturn this_$iv = INSTANCE;
int $i$f$goToStore = false;
String var3 = "Walks in";
System.out.println(var3);
int var4 = false;
String var5 = "Make purchase";
System.out.println(var5);
String var6 = "Walks out";
System.out.println(var6);
}
public static void main$default(String[] var0, int var1, Object var2) {
if ((var1 & 1) != 0) {
int $i$f$emptyArray = false;
var0 = new String[0];
}
main(var0);
}
}
So in this case we succeed to get out of the store. As we can see from all of the above crossinline doesn't really have a functional transformation functionality for the code. It is, instead, used as a marker to warn the developers of the code intent making it more readable and as shown in this last case, giving it some level of protection against developer mistakes. This doesn't stop a developer, not knowing better, to remove crossinline to only make the code compile
Tail cal optimization - What is the catch?
TCO exists already for a long time and it is used widely now in many programming languages. Some programming languages use it under the hood, and other programming languages, like Kotlin or Scala, use special keywords to signal to the compiler to perform TCO. We can describe TCO like this:
Since the late 50’s TCO was alreadya theory intentend to be applied toTail Recursivity. It allows tailrecursive functions to betransformed into iterative functionsin the compiled code for betterperformance
The idea is not only to improve performance of the code we are using, but most importantly, to avoid stackoverflow errors. To better understand this, let's have a look at the examples that I have placed in the module carparts-tailrec located at the root of this project:
sealed interface Part {
val totalWeight: Double
}
sealed interface ComplexPart : Part {
val parts: List<Part>
}
data class CarPart(val name: String, val weight: Double) : Part {
override val totalWeight: Double
get() = weight
}
data class ComplexCarPart(
val name: String,
val weight: Double,
override val parts: List<Part>
) :
ComplexPart {
override val totalWeight: Double
get() = weight
}
data class Car(
val name: String,
override val parts: List<Part>
) : ComplexPart {
override val totalWeight: Double
get() = parts.sumOf { it.totalWeight }
}
In this example I'm declaring a few data classes that I will use to create a kind of data tree set with different nodes where each nod has a weight corresponding to each car part. Each car can contain different lists of car parts that can be comples or simple. If they are comples, than that means that they are composed of may other parts with different weights having itself already some composition with a separate weight to support them. This is why we find here a Car, a ComplexCarPart, and a CarPart. With this, we can then create two cars in a list like this:
listOf(
Car(
"Anna", listOf(
CarPart("Chassis", 50.0),
CarPart("Engine", 100.0),
CarPart("Transmission", 150.0),
ComplexCarPart(
"Frame", 500.0,
listOf(
CarPart("Screw", 1.0),
CarPart("Screw", 2.0),
CarPart("Screw", 3.0),
CarPart("Screw", 4.0),
)
),
CarPart("Suspension", 200.0),
CarPart("Wheels", 100.0),
CarPart("Seats", 50.0),
CarPart("Dashboard", 30.0),
CarPart("Airbags", 20.0)
)
),
Car(
"George", listOf(
ComplexCarPart(
"Chassis", 300.0,
listOf(
CarPart("Screw", 1.0),
CarPart("Screw", 2.0),
CarPart("Screw", 3.0),
CarPart("Screw", 4.0),
)
),
CarPart("Engine", 300.0),
CarPart("Transmission", 150.0),
CarPart("Seats", 50.0),
CarPart("Dashboard", 30.0),
CarPart("Airbags", 20.0)
)
)
)
In order to calculate the total weight of the car, I am using this function which looks good but it does have a story behind it than just the declaration of it:
tailrec fun totalWeight(parts: List<Part>, acc: Double = 0.0): Double {
if (parts.isEmpty()) {
return acc
}
val part = parts.first()
val remainingParts = parts.drop(1)
val currentWeight = acc + part.totalWeight
return when (part) {
is ComplexPart -> totalWeight(remainingParts + part.parts, currentWeight)
else -> totalWeight(remainingParts, currentWeight)
}
}
Looking at totalWeight, we can see that all possible last calls to this function recursively are the call to the function itself. This is already enough, but a great tale sign of tail recursive functions is the fact that an accumulator, represented as acc in this case is passed on as a parameter for this function. This is the reason why this function is said to be tail recursive. The compiler will not make a build if we use another kind of recursive function. The keyword tailrec serves two purposed in this case. It tells the compiler that it should consider this function as a candidate for tail call optimization and it informs the developer during design time, if the function remains being a tail recursive function. It serves as a guide for both developers and the compiler, we could say.
A great question that was raised during my presentation was if the compiler can recognize tail recursive functions without using tailrec. The compiler can do that but it will not apply TCO to it unless we apply tailrec before the function declaration. If we do not apply tailrec, nothing unusual will happen:
public static final double totalWeight(@NotNull List parts, double acc) {
Intrinsics.checkNotNullParameter(parts, "parts");
if (parts.isEmpty()) {
return acc;
} else {
Part part = (Part)CollectionsKt.first(parts);
List remainingParts = CollectionsKt.drop((Iterable)parts, 1);
double currentWeight = acc + part.getTotalWeight();
return part instanceof ComplexPart ? totalWeight(CollectionsKt.plus((Collection)remainingParts, (Iterable)((ComplexPart)part).getParts()), currentWeight) : totalWeight(remainingParts, currentWeight);
}
}
Nothing special happened as expected, but with tailrec applied, then we get something very, very different:
public static final double totalWeight(@NotNull List parts, double acc) {
Intrinsics.checkNotNullParameter(parts, "parts");
while(!parts.isEmpty()) {
Part part = (Part)CollectionsKt.first(parts);
List remainingParts = CollectionsKt.drop((Iterable)parts, 1);
double currentWeight = acc + part.getTotalWeight();
if (part instanceof ComplexPart) {
List var8 = CollectionsKt.plus((Collection)remainingParts, (Iterable)((ComplexPart)part).getParts());
parts = var8;
acc = currentWeight;
} else {
parts = remainingParts;
acc = currentWeight;
}
}
return acc;
}
I have some slides at the bottom of this presentation and as I am writing this text, I noticed that even in this case, the transformation is slightly different than the one I show in the slides. I the slides, the example comes from decompiling the byte code in the same way as I do here, but in this particular case the compiler did something fundamentally different. In this case, the while loop keeps going until the parts List is empty, where as in the slides the generated example does an endless while loop until it returns when the parts list is empty. In any case, the principle is the same. Using tailrec our original tail-recursive function is transformed into an iterative function. What this does is only create one single stack where the whole process occurs. In this case we avoid making multiple recursive calls. But the big advantage is the fact that we will never get a StackOverflowException exception this way. We may get an OutOfMemoryExcpetion or any other related to resource exhaustion, but never really an overflow related error. Performance is a part of it purely because although time and space complexities for the tailrecursive function and the iterative function is mathematically the same, there is still an small overhead in generating the different call stack frames.
But we could have done this anyway in the code in Kotlin also and so why didn't we do that? The best answer to that is in my opinion that Kotlin relies at its core in immutable principles and null-safety. Also in general terms it is regarded as bad practice to reuse input parameters and change their values. Also, with a terrible reputation, is the usage of loops in the code. Coding using a tail recursive function leads to code that is very easy to understand and to follow its execution. That is what tailrec aims to provide. We can implement beautiful, but otherwise very poorly efficient and prone to StackOverflowException code, and still make it work as it should in the bytecode. Understanding tailrec, what it does and which problems it tries to solve is crucial when trying to understand our code and potentially finding the source of bugs.
Data classes and Frameworks - Why doesn't it work ... and why it does?
This is probably the most enigmatic issue of working with Data classes in different frameworks. Whether you have been working with the Spring Framework, Quarkus, or even the odd case with the old way of deploying applications using JEE war packages, you may have come across a common way of solving problems where the annotations don't seem to do anything by applying @field: as a prefix to you annotation of choice. Or maybe you have found that not even that works.
Kotlin provides use-site targets thatallow us to specify where particularannotations have to be applied.Sometimes we need them andsometimes we don’t
For this example I have created and example using the spring framework and it is located on the carpart-data-structure module. To run the example for this module we will need to start the docker containers using the docker-compose.yaml file at the root of the project. So let's first go there and run docker-compose up -d. After the service is running, let's have a look at the following entity:
@Table(name = "CAR_PARTS")
@Entity
data class CarPart(
@Id
val id: Long,
@Column
@field:NotNull
@field:Size(min=3, max=20)
val name: String,
val productionDate: Instant,
val expiryDate: Instant,
val barCode: Long,
@field:Min(value = 5)
val cost: BigDecimal
)
This entity will apply the validation correctly, but why doesn't it work exactly in the same way as if we remove the @field use-site target? Let's first have a look at what happens in this specific case. The decompiled bytecode looks like:
public final class CarPart {
@Id
private final long id;
@Column
@NotNull
@Size(
min = 3,
max = 20
)
@org.jetbrains.annotations.NotNull
private final String name;
@org.jetbrains.annotations.NotNull
private final Instant productionDate;
@org.jetbrains.annotations.NotNull
private final Instant expiryDate;
private final long barCode;
@Min(5L)
@org.jetbrains.annotations.NotNull
private final BigDecimal cost;
(...)
}
We can see that the annotations have been applied correctly to the fields and this decompiled code gives us a guarantee that everything is working. The integration tests for this case should also work flawlessly and we could try the running application using the test-requests.http file that I have created for it:
###
POST http://localhost:8080/api/v1/carparts/create
Content-Type: application/json
{
"id": 0,
"name": "brakesbrakesbrakesbrakesbrakesbrakesbrakes",
"productionDate": 1713787922,
"expiryDate": 1713787922,
"barCode": 12345,
"cost": 1234
}
By running this request, we should expect a validation error and if we perform this request we should be getting something like:
2024-04-26T14:01:49.818+02:00 ERROR 225329 --- [carparts-manager] [nio-8080-exec-1] o.a.c.c.C.[.[.[/].[dispatcherServlet] : Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: org.springframework.transaction.TransactionSystemException: Could not commit JPA transaction] with root cause
jakarta.validation.ConstraintViolationException: Validation failed for classes [org.jesperancinha.talks.carparts.carpartsdatascructures.domain.CarPart] during persist time for groups [jakarta.validation.groups.Default, ]
List of constraint violations:[
ConstraintViolationImpl{interpolatedMessage='size must be between 3 and 20', propertyPath=name, rootBeanClass=class org.jesperancinha.talks.carparts.carpartsdatascructures.domain.CarPart, messageTemplate='{jakarta.validation.constraints.Size.message}'}
And this of course makes perfect sense. However if we remove @field and let Kotlin decide that for us, then, after running the application, we will get no error and the response will just be this one:
POST http://localhost:8080/api/v1/carparts/create
HTTP/1.1 200
Content-Type: application/json
Transfer-Encoding: chunked
Date: Fri, 26 Apr 2024 12:04:12 GMT
{
"id": 0,
"name": "brakesbrakesbrakesbrakesbrakesbrakesbrakes",
"productionDate": "2024-04-22T12:12:02Z",
"expiryDate": "2024-04-22T12:12:02Z",
"barCode": 12345,
"cost": 1234
}
Response file saved.
> 2024-04-26T140412.200.json
Response code: 200; Time: 345ms (345 ms); Content length: 164 bytes (164 B)
This just means that now the annotations do not work for an entity that has been declared like this one:
@Table(name = "CAR_PARTS")
@Entity
data class CarPart(
@Id
val id: Long,
@Column
@NotNull
@Size(min=3, max=20)
val name: String,
val productionDate: Instant,
val expiryDate: Instant,
val barCode: Long,
@Min(value = 5)
val cost: BigDecimal
)
So why doesn't the data gets validated in this last case? As Kotlin advances, one of the goals it, just like in many other programming languages is to reduce what we call boiler-plate code as much as possible. However, having said that, some thing stop working as they used to when we evolve in that direction. With the advent of data class and Java records, we also remove the places where Java developers have been used to placing annoations. Using this decorative style of programming used to be very easy to do because we would find the getter and setters, parameters and fields easy to see in the code. Kotlin data class cramps everything up into a single line per class member. By doing that, we need to tell Kotlin where it should apply the annotation simply because there is no visible way to do so. To data, we can use Kotlin use-site targets, which are the answer to that problem. And it is true that for most cases, the @field will solve these problems for us. But there is a reason for that the is frequently overlook. Kotlin has rules for that and we can read them in their website and they go like this:
If you don't specify a use-site target, the target is chosenaccording to the @Target annotation of the annotation beingused. If there are multiple applicable targets, the first applicabletarget from the following list is used:
param
property
field
And so, just to give an example if we look at property @Size and check what it says in its implementation we find this:
@Target({ METHOD, FIELD, ANNOTATION_TYPE, CONSTRUCTOR, PARAMETER, TYPE_USE })
@Retention(RUNTIME)
@Repeatable(List.class)
@Documented
@Constraint(validatedBy = { })
public @interface Size {
(...)
}
It basically contains all possible targets and that means that we can apply this annotation anywhere. But there is a problem with this, if we follow the rules provided by Kotlin we will see that @Size will be applied in param and this is bad news for our code. To understand why, let's have a look at the decompiled code of the entity that doesn't validate anything anymore:
public CarPart(long id,
@jakarta.validation.constraints.NotNull
@Size(min = 3,max = 20) @NotNull String name,
@NotNull Instant productionDate,
@NotNull Instant expiryDate, long barCode,
@Min(5L) @NotNull BigDecimal cost) {
Intrinsics.checkNotNullParameter(name, "name");
Intrinsics.checkNotNullParameter(productionDate, "productionDate");
Intrinsics.checkNotNullParameter(expiryDate, "expiryDate");
Intrinsics.checkNotNullParameter(cost, "cost");
super();
this.id = id;
this.name = name;
this.productionDate = productionDate;
this.expiryDate = expiryDate;
this.barCode = barCode;
this.cost = cost;
}
And indeed, we now observe that @Size has been applied to a parameter. But now you may ask why this doesn't work? What is the difference between putting @Size in a parameters or in a field? The Spring Framework uses AOP (Aspect Oriented Programming) to be able to validate data either via entities or Dtos. However this comes out of the box to work very nicelly with fields but it is not ready to work with params. If we don't do something special, by default, applying the mentioned annotation to the contructor parameters will never take any effect.
Making sure we know how to use site targets in Kotlin is of crucial importance. I cannot stress enough the time that can be lost by trying to fix problems like this without knowing the rules that Kotlin establishes for us in order to be able to apply annotations correctly.
Delegates and otheruse-site targets - But how can we use it?
Delegates are something many of use already use without thinking about it in Kotlin. Namely use make a lot of use of by lazy which allow us to save some time during startup and only use resources when they are absolutely necessary. It can be overused as well, but this is just one example of it. However we do have at our disposal one particular use-site target that can be very intriguing.
Delegation is a great part of the Kotlin programming language and it is quite different than what we are used to seeing in Java
The use-site target I will discuss in this last segment is the @delegate use site-target. With it, we can apply a decoration or stereotype to a delegate and to see that working let's have a look at the following example locate in module carparts-use-site-targets at the root folder of the project:
interface Horn {
fun beep()
}
class CarHorn : Horn {
override fun beep() {
println("beep!")
}
}
class WagonHorn : Horn {
override fun beep() {
println("bwooooooo!")
}
}
In this example I am only creating the return type Horn where I also declare two subclasses CarHorn and WagonHorn. To return values of these types I have then created a specific delegate for them with only a getValue in mind:
class SoundDelegate(private val initialHorn: Horn) {
operator fun getValue(thisRef: Any?, property: KProperty<*>): Horn {
return initialHorn
}
}
This delegate will only return the value that is being set on its field on creation.
Finally I created two annotations that will do nothing specifically in this case, but they will allow us to see how this gets applied to a delegate:
annotation class DelegateToWagonHorn
annotation class DelegateToCarHorn
And finally to demonstrate this, a code where we can see all of the above being applied:
class HornPack {
@delegate:DelegateToWagonHorn
val wagonHorn: Horn by SoundDelegate(CarHorn())
@delegate:DelegateToCarHorn
val carHorn: Horn by SoundDelegate(WagonHorn())
}
If we decompile the bytecode from this into Java we will see something that is very obvious, but at the same time it is very important to draw some attention to it:
public final class HornPack {
static final KProperty[] $$delegatedProperties;
@DelegateToWagonHorn
@NotNull
private final SoundDelegate wagonHorn$delegate = new SoundDelegate((Horn)(new CarHorn()));
@DelegateToCarHorn
@NotNull
private final SoundDelegate carHorn$delegate = new SoundDelegate((Horn)(new WagonHorn()));
@NotNull
public final Horn getWagonHorn() {
return this.wagonHorn$delegate.getValue(this, $$delegatedProperties[0]);
}
@NotNull
public final Horn getCarHorn() {
return this.carHorn$delegate.getValue(this, $$delegatedProperties[1]);
}
static {
KProperty[] var0 = new KProperty[]{Reflection.property1((PropertyReference1)(new PropertyReference1Impl(HornPack.class, "wagonHorn", "getWagonHorn()Lorg/jesperancinha/talks/carparts/Horn;", 0))), Reflection.property1((PropertyReference1)(new PropertyReference1Impl(HornPack.class, "carHorn", "getCarHorn()Lorg/jesperancinha/talks/carparts/Horn;", 0)))};
$$delegatedProperties = var0;
}
}
There are two imporant aspects when having a look at this code. The resulting decompiled code, shows us that the annotations created have been applied to the delegates and if we look at another aspect of it, we can see that we have two more accessors made available for us to be able to access the two horns that we have created and these methods are: getWagonHorn and getCarHorn. The interesting about this bit is that it seems to suggest that we can apply a use-site target to an annotation that we want to apply to a delegate and maybe use a use-site target to an annotation that we want to apply to a getter of the property that we want to use in our code via the delegate. To test this I have created another example, which is located in a module we have already seen before in carparts-data-structures:
@Service
data class DelegationService(
val id: UUID = UUID.randomUUID()
) {
@delegate:LocalDateTimeValidatorConstraint
@get: Past
val currentDate: LocalDateTime by LocalDateTimeDelegate()
}
In this case, the DelegationServer is composed of only one field where its assignment is being done via a delegate that we have create to be one that returns a LocalDateTime. The idea of injecting a service with an annotated delegate is to allow Spring to perform its operations and wrap this delegate in a CGLIB proxy. But before we continue, let's first have a look at the implementation of the LocalDateTimeValidatorConstraint, which I didn't had the time to explain what it does and its purpose during the presentation:
@Target(AnnotationTarget.FIELD, AnnotationTarget.FUNCTION)
@Retention(AnnotationRetention.RUNTIME)
@Constraint(validatedBy = [LocalDateTimeValidator::class])
@MustBeDocumented
annotation class LocalDateTimeValidatorConstraint(
val message: String = "Invalid value",
val groups: Array<KClass<*>> = [],
val payload: Array<KClass<*>> = []
)
class LocalDateTimeValidator : ConstraintValidator<LocalDateTimeValidatorConstraint, LocalDateTimeDelegate> {
override fun initialize(constraintAnnotation: LocalDateTimeValidatorConstraint) {
}
override fun isValid(value: LocalDateTimeDelegate, context: ConstraintValidatorContext): Boolean {
return when(Locale.getDefault().country ){
"NL","US" -> true
else -> false
}
}
}
What this validator does is to invalidate our delegate if the default Local is anything other than "NL" or "US". To this, we can use the same test-requests.http file, but here we need to perform a request to another endpoint to test this:
###
POST http://localhost:8080/api/v1/carparts/create/extended
Content-Type: application/json
{
"id": 0,
"name": "brakes",
"productionDate": 1713787922,
"expiryDate": 1713787922,
"barCode": 12345,
"cost": 1234
}
Before we make this request though, make sure that you have reverted the changes we made before to test our code. Once that is done, if we make this request we should get a normal response back and we should find something like this in the service logs:
2024-04-26T14:54:57.294+02:00 INFO 234172 --- [carparts-manager] [nio-8080-exec-1] o.j.t.c.c.converter.Converters : Tried to convert data CarPart(id=0, name=brakes, productionDate=2024-04-22T12:12:02Z, expiryDate=2024-04-22T12:12:02Z, barCode=12345, cost=1234)
2024-04-26T14:54:57.325+02:00 INFO 234172 --- [carparts-manager] [nio-8080-exec-1] j.t.c.o.j.t.c.c.c.LocalDateTimeValidator : 2024-04-26T14:54:57.325284145
We get this LocalDateTime time at the end of the logs. The implementation is easy to follow and looks like this:
@PostMapping("create/extended")
fun createCarPartExtended(
@RequestBody @Valid carPartDto: CarPartDto,
@Valid delegationService: DelegationService,
) = carPartServiceExtended.createCarPart(carPartDto)
.also {
logger.info("{}", delegationService.currentDate)
}
We are in this case using a carPartServiceExtended here and this is just a service that I am createing using a @get site target. It is only another instance of carPartService created using a delegate and declared as a bean like this:
@SpringBootApplication
class CarPartsDataStructureApplication(
carPartDao: CarPartDao
) {
@get:Bean("carPartServiceExtended")
val carPartServiceExtended: CarPartsService by CarPartsService(carPartDao)
}
But let's now focus at what happens to the DelegationService where we are using both use-site targets. The field currentDate is also annotated with @get: Past, which means that we should only accept LocalDateTime in the past. This means that depending on how your Locale is configured on your machine, this code may fail or not. LocalDateTime wise, it should always work because, no matter what, our LocalDateTime will always be in the past. But lets now change the code to make it impossible to get a positive validation. Let's change it to validate to Future:
@Service
data class DelegationService(
val id: UUID = UUID.randomUUID()
) {
@delegate:LocalDateTimeValidatorConstraint
@get: Future
val currentDate: LocalDateTime by LocalDateTimeDelegate()
}
And make the delegation validation check for a non existing locale like for example CatLand:
class LocalDateTimeValidator : ConstraintValidator<LocalDateTimeValidatorConstraint, LocalDateTimeDelegate> {
override fun initialize(constraintAnnotation: LocalDateTimeValidatorConstraint) {
}
override fun isValid(value: LocalDateTimeDelegate, context: ConstraintValidatorContext): Boolean {
return when(Locale.getDefault().country ){
"CatLand" -> true
else -> false
}
}
}
And a great question to be asked at this point even before we continue is how many validation fails could we find before making the request? Is it 0, 1 or 2? If we restart the code and run the same request we will get a curious error. To start out we a 400 error:
{
"timestamp": "2024-04-26T13:08:04.411+00:00",
"status": 400,
"error": "Bad Request",
"path": "/api/v1/carparts/create/extended"
}
And the service logs will corroborate this story, but they will also tell us the validation error that have occurred:
2024-04-26T15:08:04.405+02:00 WARN 237106 --- [carparts-manager] [nio-8080-exec-1] .w.s.m.s.DefaultHandlerExceptionResolver : Resolved [org.springframework.web.bind.MethodArgumentNotValidException: Validation failed for argument [1] in public org.jesperancinha.talks.carparts.carpartsdatascructures.dto.CarPartDto org.jesperancinha.talks.carparts.carpartsdatascructures.controller.CarPartsController.createCarPartExtended(org.jesperancinha.talks.carparts.carpartsdatascructures.dto.CarPartDto,org.jesperancinha.talks.carparts.org.jesperancinha.talks.carparts.carpartsdatascructures.service.DelegationService) with 2 errors: [Field error in object 'delegationService' on field 'currentDate$delegate': rejected value [org.jesperancinha.talks.carparts.LocalDateTimeDelegate@6653aa1c]; codes [LocalDateTimeValidatorConstraint.delegationService.currentDate$delegate,LocalDateTimeValidatorConstraint.currentDate$delegate,LocalDateTimeValidatorConstraint]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [delegationService.currentDate$delegate,currentDate$delegate]; arguments []; default message [currentDate$delegate]]; default message [Invalid value]] [Field error in object 'delegationService' on field 'currentDate': rejected value [2024-04-26T15:08:04.390037846]; codes [Future.delegationService.currentDate,Future.currentDate,Future.java.time.LocalDateTime,Future]; arguments [org.springframework.context.support.DefaultMessageSourceResolvable: codes [delegationService.currentDate,currentDate]; arguments []; default message [currentDate]]; default message [must be a future date]] ]
Two important error messages that we should have a good look at are these [Invalid value], which refers to the validation provided for the delegate itself and [must be a future date], which refers to the currentDategetValue of the delegate.
So let's now have a look at how this looks like in the decompiled code:
public class DelegationService {
static final KProperty[] $$delegatedProperties;
@NotNull
private final UUID id;
@LocalDateTimeValidatorConstraint
@NotNull
private final LocalDateTimeDelegate currentDate$delegate;
(...)
@Past
@NotNull
public LocalDateTime getCurrentDate() {
LocalDateTime var1 = this.currentDate$delegate.getValue(this, $$delegatedProperties[0]);
Intrinsics.checkNotNullExpressionValue(var1, "getValue(...)");
return var1;
}
We can clearly see in this case that @LocalDateTimeValidatorConstraint has been applied to the delegate and @Past has been applied to the getter of the property we want it to be applied to.
What is great about what we just saw in this case is that we now know and understand how this particular annotation @delegate can be used. I have, however not yet seen a use-case for this in server-side development using Spring. It could be that this has special use cases for Spring or perhaps in other frameworks. In spite of that it is good to have this present when developing in Kotlin.
Conclusion
My idea with this presentation was to promote a few of ideas:
Better understanding of the Kotlin Language
Don’t fight the Spring Framework or anything else like Quarkus. They are not evil and they are not magic.
Read the Kotlin documentation and only use Google as a last resort.
Nothing is perfect and Kotlin also falls into that category and recognizing that, allow us to be better
Finally I would like to say a massive thank you to the Kotlin Dutch User Group, shout out to Raphael de Lio for reaching out, Xebia NL and JetBrains for organizing this event. Thanks eveyone for coming to the event!
Further I'd like to mention that the presentations of Jolan Rensen and Jeroen Rosenberg were really worth watching. To the date of this publication I could only find Jolan's work on DataFrameAndNotebooksAmsterdam2024.
Finally I just want to say that I am inherently someone that looks at stuff with critical thinking in mind and a lot of curiosity. This means that it is because I like Kotlin so much that I pay attention to all the details surrounding the language. There are things that Kotlin offers that may offer different challenges and it may play out different in the future but in my opinion it is a great engineering invention for the software development world.
Source code and slides
If you are interest in know more about TailRec you may also find interesting a video I made about it right over here:
And if you just want a quick, easy listening way of learning about the data classes case I mention in this presentation I also have a video that talks about that over here:
At the end of this presentation I mention a custom validation using Spring. If you know the AssertTrue and AssertFalse annotations you may ask why I didn't use those. That is because both only validate for boolean returned values. I have made videos about both, but for this presentation this the one of interest where I explain how to make custom validations:
About me
Homepage -https://joaofilipesabinoesperancinha.nl
YouTube - JESPROTECH https://www.youtube.com/channel/UCzS_JK7QsZ7ZH-zTc5kBX_ghttps://www.youtube.com/@jesprotech
GitHub - https://github.com/jesperancinha
Hackernoon - https://hackernoon.com/u/jesperancinha
DevTO - https://dev.to/jofisaes
Medium - https://medium.com/@jofisaes