• history,
• key features,
• pros & cons,
• use cases,
• and finally- examples of companies relying on this relational database.

And a friendly reminder- this PostgreSQL introduction is a continuation of a top 8 databases series I covered here.

Explainer Animation

If you enjoy whiteboard animations, then right here you can find a video version of this article:

If you find this content useful, please leave a subscription  😉

PostgreSQL History

As usual, let’s start everything by learning more about PostgreSQL genesis.

PostgreSQL history started in the 1980s at the University of California, Berkeley when Michael Stonebraker began working on a new project, which would be the successor of Ingres. The goal of this project was to address the problems plaguing the databases of that time.

Initially, the project was called POSTGRES, highlighting that it is the successor of Ingres (which was created at the same university). But then, in 1996, the name was changed to PostgreSQL to emphasize its support for SQL. In 2007, the developer team decided to stick with the name PostgreSQL and Postgres as an alias.

But back to the development topic, the project target was to introduce support for data types and relationships between them. And, although the team took a lot of ideas from Ingres, the codebase was created from scratch.

The PostgreSQL introduction took place in 1986 when the team released official papers describing the system. Two years later, they performed a demo of a prototype at the ACM SIGMOD Conference. The team was gradually releasing new versions with new features and changes. Nevertheless, the number of feature requests and users began to surpass them, and the project ended in 1994, and POSTGRES Version 4.2 was released under the MIT license.

As the official date of the PostgreSQL release, we take 8th July of 1996, when Marc Fournier created the first non-university server for Postgres development.

What Is PostgreSQL Then?

So, with all of that being said, let’s answer the question- what is PostgreSQL?

Well, it’s an open-source relational database management system (aka RDBMS), based on the POSTGRES Version 4.2 (which we’ve already covered).

One of the key features of PostgreSQL is its support for a wide range of data types. Moreover, it also supports data types for special purposes, like arrays, JSON, and XML.

In addition to this, Postgres is also extendable and comes with a number of features that make it well-suited for a wide range of applications, like:

• transactions
• stored procedures & views
• triggers
• and support for complex queries.

What’s worth mentioning here as well, the Postgres license lets everyone use, modify and distribute it entirely for free. Regardless of whether we do it commercially or for personal use.

PostgreSQL Advantages

Nextly, let’s take a while to go through the introduction of PostgreSQL’s advantages:

• First of all, it’s open source. Postgres usage is entirely free regardless of whether we use it commercially, or not. Whatsoever, we can always modify and adjust it to our needs. This makes PostgreSQL a great choice for everyone, who desires a high-quality DBMS at no cost.
• Secondly, performance. It is known for its great performance, with the ability to handle large amounts of data and high levels of traffic.
• Thirdly, PostgreSQL is community-driven. From its very beginning, Postgres is developed by many people and companies working together to deliver a high-quality solution.
• Additionally, it has an active community and a lot of resources available online. Just like in previous articles, in this PostgresSQL introduction, I must use this argument. Tons of materials, libraries, forums, blog posts, etc. is a great advantage, which helps us incorporate it into our projects with ease. Moreover, in case of any problems we can easily find solutions on the internet, for example using StackOverflow.
• Whatsoever, Postgres supports a wide variety of programming languages. We can write SQL queries or custom functions with C or C++. Of course, we can find probably hundreds of libraries to write code with Java, Kotlin, and many more.
• Lastly, it comes with plenty of features and data types support out of the box. If you would like to learn more about its features, then I highly encourage you to visit its documentation here. Not to mention that we can always extend and customize its behavior due to its liberal licensing.

PostgreSQL Disadvantages

With all of that great things being covered in our PostgreSQL introduction, let’s see its drawbacks:

• Firstly, it’s comparatively slower and it is not as well supported as MySQL when it comes to different applications or hostings.
• Secondly, complexity. PostgreSQL is a powerful ORDBMS, but it may require more resources and expertise to set up and manage than some other database systems.
• Lastly, being an open-source solution means that there’s not a single organization, or a company standing behind it and it does not come with any warranty.

PostgreSQL Use Cases

As the next thing, let’s see what PostgreSQL is well suited for:

• First of all, web-based apps. PostgreSQL with its great support for different (even complex) data types and other features can be a great solution for content management systems, e-commerce platforms, etc.
• Scientific and financial applications. Postgres is often used in apps, which require processing large amounts of data, support for data analysis, and transactions support.
• Moreover, everywhere, where we need to process spatial data. Postgis, an extension for PostgreSQL, adds support for geographic objects allowing location queries to be run in SQL.

Please keep in mind that these are not all possible use cases. And if you need a mature relational database management system, which supports various data types, complex queries, and large datasets, then PostgreSQL can be a great choice.

Which Companies Use Postgres?

As the last thing, let’s take a second to see the list of companies using PostgreSQL.

As the first one- Apple. It uses Postgres to store and manage data for a number of its internal systems, including its online store and customer support systems. They even replaced MySQL with Postgres as an embedded DB in the OS X Lion.

And as the next one, Instagram. Yes, a giant with over 2 billion users makes use of PostgreSQL as one of the main databases for their system.

Of course, the list of all companies is much longer, but let’s see a couple more:

• Skype
• Yahoo
• Spotify
• Salesforce
• Pinterest
• Uber and Airbnb.

Summary Of The PostgreSQL Introduction

And that’s all for this article, in which we cover what exactly PostgreSQL is. Together, we walked through its genesis, advantages, disadvantages, and use cases, as well.

I hope you enjoy this series about the most popular databases on my blog. Please keep in mind, that I have other articles, in which you can learn how to work with Postgres in practice, for example:

• REST API With Ktor, Ktorm, and PostgreSQL
• YouTube Course On REST API With Spring WebFlux and Kotlin

Alternatively, you might wan to check other articles in a series:

• Redis Database Explained
• MySQL Essentials: A Fast-Paced Introduction

Finally, the task for you: please leave a comment with your thoughts or feedback in the comments section below.

The post A Quick Introduction To PostgreSQL appeared first on Codersee blog- Kotlin on the backend.

In this article, we will take a look at the 8 most popular databases and their key features, as well as common use cases.

So, no matter whether you are a database administrator, programmer, or simply would like to learn more about the different databases available, this article will provide you with all the necessary information to start with.

Doodly Animation

If you find this content useful, please leave a thumb up, or a sub. This way you can help me, so that I can help you 😉

What Does “The Most Popular Databases” Mean?

Well, before we start I wanna make sure that we are on the same page when referring to the “most popular”.

Depending on who wrote the article, this term can have various meanings. For some, it will be the frequency of Google searches. For others, it may be data provided by the https://db-engines.com page.

And for the purpose of this blog post, I decided to use the Stack Overflow Developer Survey 2022 results, which present, as follows:

In the next chapters, I will walk you through each of the 8 most popular databases shortly.

Please keep in mind that this article is a general overview and expect that more thorough continuations will be added in the upcoming weeks 🙂

MySQL Database

With that being said, let’s start with the winner- MySQL.

It’s an open-source relational database management system (aka RDBMS). It was created in 1994 by the Swedish company MySQL AB (which was later acquired by Sun Microsystems and then by Oracle Corporation). Originally, it was developed as a proprietary database system but later it was open-sourced under the GNU General Public License.

Advantages:

• It’s free and open-source, so it can be a great choice for smaller companies and organizations preferring to cut costs.
• Moreover, MySQL is widely supported by web hosting providers, which means that it can be easily integrated with web-based applications.
• Additionally, it has a low learning curve and a simple and intuitive user interface, so it is not only a great choice for beginners but also for less experienced teams.
• Finally, a large and active community of users and developers means a lot of online resources and support available.

Disadvantages:

• First of all, it is not designed for applications that require real-time data processing or high availability. And technically- this is not a drawback, but rather something you should be aware of.
• Whatsoever, MySQL is hard to scale and not intended for large amounts of data.
• Lastly, it does not have built-in support for full-text search or geospatial data and some advanced features that are available in other database systems.

As I mentioned above, I will create a separate blog post about the king of the most popular databases. But for now, I can invite you to check out my other how-to guides with MySQL:

• Deploy Kotlin Spring Boot App with MySQL on Kubernetes
• Spring Boot on Google GKE with Cloud SQL and Kotlin

PostgreSQL (aka Postgres)

Nextly, let’s take a look at the second place with a 3% difference- PostgreSQL (also known as Postgres).

It was created in 1986 by a team of researchers at the University of California and is a powerful and open-source object-relational database management system. PostgreSQL is a highly popular and widely used DBMS, known for its flexibility, reliability, and performance.

Advantages:

• Just like MySQL, PostgreSQL is open-source, free to use, and has a large, active community.
• It is highly reliable and performs well even under heavy workloads. This makes it a good choice for applications that require high availability and scalability.
• Moreover, Postgres comes with support for a lot of features and capabilities out of the box. For example triggers, stored procedures, and multi-row transactions.

Disadvantages:

• Firstly, PostgreSQL may not be as widely supported or compatible with technologies, as some other database systems.
• Secondly, it may not be as fast or scalable as other database management systems, such as NoSQL databases. And whereas it works well for large datasets, it is not the speed demon.
• Whatsoever, Postgres is a pretty complex system, so it might be a bit difficult to learn.

And just like previously, please expect new articles to come. But, again, if you would like to learn backend development with Kotlin and Postgres, then I can recommend:

• REST API With Spring WebFlux and Kotlin Course
• other tutorials on how to use it with Spring Boot, or Ktor

SQLite

As the next step, let’s take a while to the no 3 in the most popular databases summary- SQLite.

SQLite is an embedded SQL database engine. The main difference between it and most of SQL databases is that it is self-contained and does not run a separate server process. To put it simply- it’s a library, which reads and writes directly to plain disk files.

It was created in 2000 by Dwayne Richard Hipp using a C programming language and since then, it become the most deployed database engine- used, among others of top web browsers.

Advantages:

• First of all, SQLite is a really lightweight database and it can be successfully used in low-memory environments. With all features enabled, its size can be as little as 750KiB.
• Additionally, the database file format is portable and can be migrated between 32-bit ad 64-bit systems.
• Moreover, SQLite is easy to learn and requires zero configuration, which makes it a great choice for beginners (and not only!)
• Finally, it is really fast and can bring much better performance, than the filesystem.

Disadvantages:

• Firstly- it’s not suitable for client/server applications. Although it can work over the network, the filesystem latency might be an issue.
• Secondly (and lastly), SQLite is not a good choice for high-volume websites or environments demanding high concurrency. Whereas it does not have a limit for concurrent reads, only one writer can write at any instant in time.

MongoDB (aka Mongo)

And the prize for 4th place in our most popular databases ranking goes to MongoDB.

MongoDB is an open-source document-oriented database. It was created in 2009 by MongoDB Inc., which makes it not only the first NoSQL representative in the ranking but also the youngest one (so far). In contrast to previous examples, MongoDB is an example of a NoSQL database, so instead of tables and rows, Mongo stores JSON-like documents, which are organized in collections.

Advantages:

• First of all, Mongo was built with environments demanding high performance and scalability in mind.
• Secondly, it’s flexible. MongoDB does not require you to define a schema for your collections (SQL tables counterparts), which can help you reduce the time spent on schema changes. Moreover, it can be a great choice everywhere, where a schema is hard to define.
• Finally, just like MySQL and Postgres, MongoDB is widely supported and really easy to learn.

Disadvantages:

• No out-of-the-box support for joins. Unlike relational databases, Mongo by design does not support joins between collections, so if you have to work with highly connected data sets, then it might not be the best choice.
• Moreover, MongoDB documents cannot exceed 16MB.
• Lastly, the lack of joins oftentimes leads to data duplication, which may become troublesome to maintain in the future.

If you would like to learn Mongo in practice, then check out my practical guides about MongoDB with Spring Boot, Ktor, and Micronaut.

Microsoft SQL Server

Following, the 5th position on the most popular databases list is taken by Microsoft SQL Server.

It was developed by Microsoft and first released in 1989 and is another relational database in our ranking. Microsoft SQL Server lets us communicate using T-SQL (Transact-SQL), which is an extension of SQL bringing additional features, like functions, or local variables. Whatsoever, it comes with an entire range of products, which we can optionally configure together to make it a really powerful tool.

Advantages:

• Firstly, Microsoft SQL Server has really good support and documentation.
• Additionally, it’s relatively easy to set up and use.
• Whatsoever, SQL Server provides a range of security features to protect against unauthorized access to the data and ensure data integrity.
• Lastly, a strong point is data recovery in case anything happens.

Disadvantages:

• First of all, pricing. And although there is a free option available, Enterprise or Standard editions are pretty expensive.
• Secondly, compatibility. Prior to the SQL Server 2017, it could be run on Windows-based servers, but this is the first version, which can be deployed on Linux.
• Lastly, it may require additional Microsoft software purchases to fully make use of it in your case.

Redis

And the 6th most popular database of 2022 is Redis.

It was created by Salvatore Sanfilippo, also known as antirez, in 2009, which makes it a relatively young database in our summary.

Redis is an open-source, in-memory data structure store that is commonly used as a database, cache, and message broker. It is known for its high performance, flexibility, and scalability, making it a popular choice for a wide range of applications. Whatsoever, it is oftentimes used in conjunction with Microservices and containers, to support the needs of modern, distributed applications.

Advantages:

• As the first one- Redis is an in-memory data store, which results in much higher data processing, than disk-based databases.
• Additionally, it’s highly scalable and can be easily expanded to support the growing needs of our application.
• Lastly, Redis is an open-source project with a large community and a huge amount of online resources to use and learn from. Moreover, it is compatible and pretty easy to use with most of the popular frameworks.

Disadvantages:

• So the first advantage is the disadvantage, as well. The in-memory data store is actually faster, but we have to keep in mind that it is more expensive.
• Moreover, it is a key-value store (like a dictionary), which means that the application(s) connecting to Redis have to be aware of the structure, which requires additional config.
• Lastly, Redis is doing fine when dealing with key-based operations, but is not that efficient, when it comes to searching through the data set.

If you would like to learn more about it in detail, then check out my more detailed article: Redis Database Explained

MariaDB

As the one before last in the most popular databases list, let’s see what exactly MariaDB is.

First of all, MariaDB is a fork of the MySQL DBMS, created by the original developers of MySQL in response to the acquisition of MySQL by Oracle Corporation. The acquisition raised concerns among many users and developers about the future direction of the MySQL project, which resulted in MariaDB being released in 2009.

It’s known for its performance, reliability, and ease of use, and is used by many organizations around the world. Moreover, it is designed to be backward-compatible with MySQL and can be used as a drop-in replacement for it in most cases.

Advantages:

• Firstly, MariaDB is known for its performance and reliability.
• Additionally, it is an open-source project with a large and active community of users and developers. And I know that this argument repeats often in this article. But trust me- a wealth of resources, such as documentation and tutorials, available online comes with plenty of other perks.
• Thirdly, as mentioned above it can be used as a MySQL drop-in replacement in most cases. This allows companies to replace the database without changing the other tools. However, please remember that this is not always the case and sometimes may require additional work.
• And lastly, it is widely supported by various programming languages and frameworks.

Disadvantages:

• As the first one, some of the MySQL Enterprise Edition features are not available out of the box. And although this can be done through plugins, this is an additional, independent tool we have to incorporate.
• Whatsoever, migrating from MySQL may result in additional adjustments and config.
• And as the last thing- the support may be pretty expensive.

Elasticsearch

And finally, we come to the last position in the top 8 most popular databases of 2022- Elasticsearch.

Elasticsearch is an open-source, full-text search and analytics engine created by Shay Banon in 2010.

It is designed to be scalable and flexible, making it a popular choice for building search and data analysis applications. Moreover, it’s based on the Lucene search engine library and provides a RESTful API for interacting with the data. Elasticsearch is known for its powerful search and analytics capabilities, and can be used to search, analyze, and visualize data from a variety of sources, including structured, unstructured, and time-series data.

Advantages:

• Elasticsearch is an open-source project, supported by a large and active community of users.
• Similarly, it’s compatible with plenty of programming languages and frameworks, which helps in its implementation.
• Lastly, it is a highly scalable search and analytics engine, which is intended for fast full-text searches. Thus, it can be a great solution for website searches.

Disadvantages:

• Elasticsearch may not be as fast or efficient as some other search and analytics platforms in certain scenarios- for example, a large amount of data.
• Moreover, it is more difficult to learn compared to similar tools. But this comes with being more powerful, as well.
• Lastly, Elasticsearch should not be used as primary storage.

The Most Popular Databases Summary

And that’s all for this article about the 8 most popular databases. As I already mentioned, this is the initial, bird’s-eye view post and you can expect more detailed articles to come.

Let me know your thoughts in the comments section below!

The post A Quick Intro To The 8 Most Popular Databases appeared first on Codersee blog- Kotlin on the backend.

Hello friend! In this post, I would like to show you how to solve a pretty common issue when mapping Postgres jsonb to Jackson JsonNode in Spring Data JPA.

If you are not a regular reader of my blog, then I’m pretty sure that you wanted to map a PostgreSQL jsonb field in your Spring Boot with Spring Data JPA application to Jsonnode and you saw something like that:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘entityManagerFactory’ defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Could not determine recommended JdbcType for `com.fasterxml.jackson.databind.JsonNode`

If that’s the case, then you’ve come to the right place. 😁

2. Quick Setup

Before we learn how to solve the issue with Postgres jsonb mapping to JsonNode, let’s take a minute to reproduce the error.

Note: If you would like to cut straight to the point and see how to implement a mapper, then you can skip this part and navigate directly to chapter 3.

2.1. PostgreSQL Table With jsonb column

Firstly, we need to have a PostgreSQL table with jsonb column:

As we can see, it’s just a dummy table called some_table, which contains 4 example columns.

And to be on the same it page, I created it and populated with the following SQL script:

create table some_table(
  id serial not null primary key,
  string_column text not null,
  integer_column smallint not null,
  jsonb_column jsonb not null
);

insert into 
  some_table(string_column, integer_column, jsonb_column)
values
  ('value-1', 1, '{ "some_field": "some_value"}'),
  ('value-2', 2, '{ "another_field": "some_value_2"}');

[elementor-template id=”9007393″]

2.2. Spring Boot and Dependencies

Nextly, let’s create an example Spring Boot project. As always, we can do that with the Spring Initializr page.

Please configure it according to your needs, but do not forget to add the PostgreSQL Driver and Spring Data JPA dependencies. Additionally, we will need to enrich the project with Jackson Databind:

implementation("org.springframework.boot:spring-boot-starter-data-jpa")
implementation("com.fasterxml.jackson.core:jackson-databind:2.14.1")
runtimeOnly("org.postgresql:postgresql")

2.3. application.yaml

With that being done, let’s specify the connection details:

spring:
  datasource:
    url: "jdbc:postgresql://localhost:5432/postgres"
      username: postgres
      password: postgres

2.4. Implement Entity

As the next step, let’s open up the project and add the SomeEntity data class:

@Entity
@Table(name = "some_table")
data class SomeEntity(
  @Id
  val id: Long? = null,
  val stringColumn: String,
  val integerColumn: Int,
  val jsonbColumn: JsonNode
)

As we can see, we use the @Entity annotation to mark the class as the entity and @Table to explicitly provide Spring with the table name. Moreover, the necessary @Id annotation specifies the primary key of our entity and the jsonbColumn will contain the value of the Postgres jsonb column mapped to JsonNode.

2.5. Create JPA repository

With that being done, let’s add the SomeEntityRepository interface:

interface SomeEntityRepository : CrudRepository<SomeEntity, Long>

This time, the only thing we have to do is to extend the CrudRepository interface, which comes with a basic CRUD operations.

2.6. Create JPA repository

Lastly, let’s add the logic, which will be run when the application starts:

@Component
class MyRunner(
  private val repository: SomeEntityRepository
) : CommandLineRunner {

  override fun run(vararg args: String?) {
    repository.findAll()
      .forEach(::println)
  }
}

As we can see, a repository instance will be injected into this simple component. Moreover, we implement the run method of CommandLineRunner, which instructs Spring Boot to run after the app starts (if you would like to learn more about it, then check out my article about the Spring Boot console app).

Finally, we want to print out all entities, nevertheless, such a stack trace appears to our eyes:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'entityManagerFactory' defined in class path resource [org/springframework/boot/autoconfigure/orm/jpa/HibernateJpaConfiguration.class]: Could not determine recommended JdbcType for `com.fasterxml.jackson.databind.JsonNode`
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1751) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:599) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:521) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1130) ~[spring-context-6.0.2.jar:6.0.2]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:905) ~[spring-context-6.0.2.jar:6.0.2]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:584) ~[spring-context-6.0.2.jar:6.0.2]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:730) ~[spring-boot-3.0.0.jar:3.0.0]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:432) ~[spring-boot-3.0.0.jar:3.0.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:308) ~[spring-boot-3.0.0.jar:3.0.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1302) ~[spring-boot-3.0.0.jar:3.0.0]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1291) ~[spring-boot-3.0.0.jar:3.0.0]
at com.codersee.jsonbtojsonnode.JsonbToJsonNodeApplicationKt.main(JsonbToJsonNodeApplication.kt:14) ~[main/:na]
Caused by: org.hibernate.type.descriptor.java.spi.JdbcTypeRecommendationException: Could not determine recommended JdbcType for `com.fasterxml.jackson.databind.JsonNode`
at org.hibernate.type.descriptor.java.spi.UnknownBasicJavaType.getRecommendedJdbcType(UnknownBasicJavaType.java:37) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.boot.model.process.internal.InferredBasicValueResolver.from(InferredBasicValueResolver.java:222) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.mapping.BasicValue.buildResolution(BasicValue.java:507) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.mapping.BasicValue.resolve(BasicValue.java:315) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.mapping.BasicValue.resolve(BasicValue.java:305) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.lambda$processValueResolvers$4(InFlightMetadataCollectorImpl.java:1767) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at java.base/java.util.ArrayList.removeIf(ArrayList.java:1682) ~[na:na]
at java.base/java.util.ArrayList.removeIf(ArrayList.java:1660) ~[na:na]
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.processValueResolvers(InFlightMetadataCollectorImpl.java:1766) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.boot.internal.InFlightMetadataCollectorImpl.processSecondPasses(InFlightMetadataCollectorImpl.java:1752) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.boot.model.process.spi.MetadataBuildingProcess.complete(MetadataBuildingProcess.java:300) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.metadata(EntityManagerFactoryBuilderImpl.java:1350) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.hibernate.jpa.boot.internal.EntityManagerFactoryBuilderImpl.build(EntityManagerFactoryBuilderImpl.java:1421) ~[hibernate-core-6.1.5.Final.jar:6.1.5.Final]
at org.springframework.orm.jpa.vendor.SpringHibernateJpaPersistenceProvider.createContainerEntityManagerFactory(SpringHibernateJpaPersistenceProvider.java:66) ~[spring-orm-6.0.2.jar:6.0.2]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.createNativeEntityManagerFactory(LocalContainerEntityManagerFactoryBean.java:376) ~[spring-orm-6.0.2.jar:6.0.2]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.buildNativeEntityManagerFactory(AbstractEntityManagerFactoryBean.java:409) ~[spring-orm-6.0.2.jar:6.0.2]
at org.springframework.orm.jpa.AbstractEntityManagerFactoryBean.afterPropertiesSet(AbstractEntityManagerFactoryBean.java:396) ~[spring-orm-6.0.2.jar:6.0.2]
at org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean.afterPropertiesSet(LocalContainerEntityManagerFactoryBean.java:352) ~[spring-orm-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1797) ~[spring-beans-6.0.2.jar:6.0.2]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1747) ~[spring-beans-6.0.2.jar:6.0.2]
... 15 common frames omitted

3. Map Postgres jsonb to JsonNode

So at this point, we can see that Spring “Could not determine recommended JdbcType for `com.fasterxml.jackson.databind.JsonNode`“.

One of the possible solutions is to implement a custom AttributeConverter and mark it with @Converter annotation:

@Converter(autoApply = true)
class MyConverter : AttributeConverter<JsonNode, String> {

  private val mapper = ObjectMapper()

  override fun convertToDatabaseColumn(node: JsonNode): String {
    return node.asText()
  }

  override fun convertToEntityAttribute(data: String): JsonNode {
    return mapper.readTree(data)
  }
}

As we can see, the AttributeConverter interface comes with two methods: convertToDatabaseColumn and convertToEntityAttribute, which will be invoked when mapping between the entity and the database.

Additionally, we have set the autoApply to true (the default is false) to inform the persistence provider that it should use this converter to map between Postgres jsonb and JsonNode. Without it, we would have to explicitly annotate the desired field:

@Convert(converter = MyConverter::class)
val jsonbColumn: JsonNode

Of course, the choice is up to you and may vary depending on your project, but most of the time you would like to choose the first solution.

Finally, if we run the application, we should see that everything is working, as expected and logs contain the following lines:

SomeEntity(id=1, stringColumn=value-1, integerColumn=1, jsonbColumn={"some_field":"some_value"})
SomeEntity(id=2, stringColumn=value-2, integerColumn=2, jsonbColumn={"another_field":"some_value_2"})

4. Summary

And that’s all for this short, hands-on article on how to map Postgres jsonb to JsonNode. As always, you can find the source code in this GitHub repository.

As always, happy to hear your thoughts on how would you solve this issue.

The post Postgres jsonb to Jackson JsonNode With Spring Data JPA appeared first on Codersee blog- Kotlin on the backend.

In this article, which is a continuation of a Flyway series we will learn how to add Flyway to an existing Spring Boot Project.

If you are not sure what this tool can bring to your project, then check out my first post on Flyway with Spring Boot, where you can find all the necessary information.

2. Project Setup

This time, we won’t spend time preparing a sample project (well, if you are here then there’s a 95% chance that you already have some project).

Nevertheless, if you don’t have one, but still would like to learn how to add a flyway to an existing Spring Boot project, then no worries. You can find an example project in this GitHub repository.

Note: If you use the project from the linked repository, then please run the script.sql file (can be found in /resources/sql directory) against your database, before you start the application.

3. Strategy For Adding Flyway To Existing Spring Boot Project

Before we start anything, I just wanted to emphasize that we should always create a backup, before performing any operations on our databases. Regardless of how widely used, or well-tested the tool is, something can always go wrong and we have to be ready to roll out plan B.

3.1 Add Flyway Dependency

As the first thing (after the backup!), let’s add the necessary dependency to our Spring Boot project.

For gradle, it will look as follows:

implementation 'org.flywaydb:flyway-core:9.8.3'

On the other hand, when working with Maven, we need to add these lines:

<dependency>
  <groupId>org.flywaydb</groupId>
  <artifactId>flyway-core</artifactId>
  <version>9.8.3</version>
</dependency>

3.2 Verify Spring Boot App

Nextly, let’s run our Spring Boot application and verify, whether anything changed:

org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Found non-empty schema(s) "custom_schema" but no schema history table. Use baseline() or set baselineOnMigrate to true to initialize the schema history table.
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1751) ~[spring-beans-6.0.2.jar:6.0.2]

As we can clearly see, Flyway throws a pretty descriptive message:

Found non-empty schema(s) “custom_schema” but no schema history table. Use baseline() or set baselineOnMigrate to true to initialize the schema history table.

And basically, this error clearly states what we need to do when adding a Flyway to an existing project. But before that, we have to take care of a couple of things.

3.3 Generate Production Database DDL

Firstly, we have to generate an SQL script containing the DDL our production database, including:

• procedures
• views,
• triggers etc…

I won’t go into details here, because DDL script generation depends on the database you are using. Nevertheless, it can be done pretty easily with tools like DataGrip, DBeaver, or pg_dump.

And when the SQL script with DDL is ready, let’s save it in the Flyway migrations directory (in Spring Boot the default one is resources/db/migration). Of course, we have to remember about the Flyway files naming convention, so the file name should be something, like V1__my_custom_init_script.sql.

3.4 Clear Irrelevant Data

If you have only one database instance in your project, then feel free to skip this point. Otherwise, we have to make sure that the generated DDL script will work in every other environment. And we can achieve it in two ways:

• if we don’t care about the data in other environments- then we can run the Flyway clean command.
• however, if we do actually care about the data in our non-production environments, then there’s no other choice than to manually verify and align other database structures with the main one.

The choice here is yours and will depend on many factors, like the importance of test data, amount of environments, etc.

Nevertheless, if you would like to clean your database before adding a Flyway to your existing project, then you can do that with the following:

flyway -defaultSchema="custom_schema" \
  -url="jdbc:postgresql://{URL}:{PORT}/{DB_NAME}" \
  -cleanDisabled="false" \
  -user="{USERNAME}" \
  clean

Please keep in mind that with this command, we will be prompted to enter the password manually (and alternatively, we can pass the -password flag).

3.5 Add Flyway To Existing Spring Boot Project

Finally, with all that being done, we can return to the Spring Boot project and make the necessary changes.

Let’s navigate to the application.yaml file and insert these settings:

spring:
  flyway:
    baselineOnMigrate: true
    defaultSchema: "custom_schema" # not necessary, but most likely you'll want to point to a specific schema

As we can see, with baselineOnMigrate set to true, we instruct Flyway to automatically call baseline when migration is executed against a non-empty schema. Additionally, we set the defaultSchema value, but this is totally optional.

As a result, the flyway_schema_history table is added to our schema:

and populated with precisely one record.

Please remember that the baselineOnMigrate set to true is only required once- when we do the initial deployment and can be removed after that.

3.6 Add Flyway Using Commands

Basically, with all of the above being done we are finished and can enjoy the benefits of Flyway in our project.

However, if you would like to learn how to add Flyway to an existing project using CLI, then you can do exactly the same with the following command:

flyway -defaultSchema="custom_schema" \ 
-url="jdbc:postgresql://{HOST}:{PORT}/{DB_NAME}" \
-user="USERNAME" \
-baselineOnMigrate="true" \
-locations="filesystem:." migrate

Or, alternatively, run baseline only:

flyway -defaultSchema="custom_schema" \ 
-url="jdbc:postgresql://{HOST}:{PORT}/{DB_NAME}" \
-user="USERNAME" \
-locations="filesystem:." baseline

And then the migrate command:

flyway -defaultSchema="custom_schema" \ 
-url="jdbc:postgresql://{HOST}:{PORT}/{DB_NAME}" \
-user="USERNAME" \
-locations="filesystem:." migrate

These commands might be useful if you would like to extract migrations to a separate directory (and/or repository).

4. Add Flyway To An Existing Spring Boot Project Summary

And that would be all in this article about how to add Flyway to an existing Spring Boot project. If you’d like to get to know Flyway better, then I highly encourage you to visit their documentation.

Let me know in the comments section in case of any questions 🙂

The post Add Flyway To An Existing Spring Boot Project appeared first on Codersee blog- Kotlin on the backend.

If you are not sure, whether this is something for you, then please check the introduction, which will help you to make the decision:

If you would like to see the course, then you can find all videos in this playlist.

This is my first YouTube course and I will be forever thankful if you would like to share some feedback with me. You can do that in the comments section here, under the video, or by using the contact form. I am constantly trying to improve the content quality, but that’s simply impossible without hearing from you and your perspective 🙂

Let me know what you think about this course and what topics would you like to see after Spring WebFlux.

Finally, I wanted to share that Codersee was announced as one of the Top 10 Kotlin Programming Blogs on the web by FeedSpot. On this occasion, I would like to thank you, dear reader, for your input and for being a part of this journey. Thanks and see you in the near future! 🙂

The post YouTube Course On REST API With Spring WebFlux and Kotlin appeared first on Codersee blog- Kotlin on the backend.

In this step-by-step guide, we will learn how to deploy a Spring Boot application With AWS App Runner and Amazon RDS (using PostgreSQL).

Although as an example, we will create and containerize the Spring Boot app, the knowledge you’re going to gain today is universal and you will be able to easily apply it in any project.

Important note: Please delete ALL RESOURCES created in AWS after finishing this tutorial. Otherwise, you might easily forget about them and be charged for using them.

In my articles, I focus on the practice and getting things done.

However, if you would like to get a strong understading of DevOps concepts, which are more and more in demand nowadays, then I highly recommend to check out KodeKloud courses, like this Docker and Kubernetes learning paths.

2. What Is AWS App Runner?

Basically, it is a fully managed service, which allows us, the developers, to deploy containerized applications with ease. It takes care of plenty of things, like load balancing, scaling, and deployment out of the box so that we can focus on the source code. Definitely, a tool worth considering when working in smaller teams. If you are interested in more details, please refer to this web page.

3. What Is Amazon RDS?

Well, to be even more specific, Amazon Relational Database Service (RDS), is a fully managed cloud database service. Like the App Runner, it takes care of many things related to database setup, operating, and scaling. Moreover, we can choose between several engines, like:

• PostgreSQL
• MySQL
• SQL Server, and a few more…

Similarly, I highly encourage you to visit this AWS page to see more information.

4. Create Spring Boot Project

As I mentioned in the beginning, in this post we will create a simple Spring Boot application connecting to PostgreSQL, however, we won’t get into many details here, so that we can focus on AWS App Runner and Amazon RDS part.

If you don’t want to spend more time here, please see this GitHub repository and skip to step 5.

4.1. Imports

As the first step, let’s take care of imports in our project. Let’s add the following to the standard Spring Boot imports:

implementation("org.springframework.boot:spring-boot-starter-data-jdbc")
implementation("org.springframework.boot:spring-boot-starter-web")
implementation("org.flywaydb:flyway-core:8.5.11")
runtimeOnly("org.postgresql:postgresql")

Please note, that we’ve added a Flyway, as well. It will be useful to set up the database effortlessly.

4.2. Flyway Migration

With that being done, let’s add a new file called V1__Init_And_Create_User_Table.sql inside the /resources/db/migration directory:

CREATE TABLE app_user (
  id SERIAL NOT NULL PRIMARY KEY,
  email TEXT NOT NULL
);

INSERT INTO app_user(id, email) values (1, 'john@doe.com');

4.3. Add Model and Repository

Following, let’s create an AppUser data class:

@Table("app_user")
data class AppUser(
    @Column("id") val id: Long? = null,
    @Column("email") val email: String
)

And related AppUserRepository:

interface AppUserRepository : CrudRepository<AppUser, Long>

4.4. Expose Test Endpoints

Nextly, let’s expose two test endpoints:

@RestController
@RequestMapping("/api/users")
class AppUserController(
    private val appUserRepository: AppUserRepository
) {

    @GetMapping
    fun getAllUsers(): List<AppUser> =
        appUserRepository
            .findAll()
            .toList()

    @GetMapping("/non-db")
    fun getAllUsersNonDb(): List<AppUser> =
        listOf(
            AppUser(1L, "piotr@wolak.com")
        )
}

These two endpoints will be helpful to validate if our Spring Boot App is deployed successfully on App Runner and connected to Amazon RDS.

4.5. Configure application.yaml

After that, let’s configure application.yaml file:

spring:
  datasource:
    url: jdbc:postgresql://${DB_HOST:localhost}:5432/
    username: postgres
    password: password

As we can see, these 3 values will be used to connect our application to the PostgreSQL instance. Whatsoever, we’ve introduced the DB_HOST environment variable, so that we will be able to inject database URL easily later.

4.6. Create JAR File

With that being set, let’s build our service and create a JAR file. When working with gradle (and gradle wrapper), we can simply run the following:

./gradlew build

With that, a new file called app-runner-rds-0.0.1-SNAPSHOT.jar is generated inside the /build/libs directory.

4.7. Prepare a Dockerfile

Finally, let’s create a Dockerfile inside the root directory:

FROM openjdk:17-jdk-alpine
RUN mkdir /work
WORKDIR /work
COPY /build/libs/app-runner-rds-0.0.1-SNAPSHOT.jar app.jar
EXPOSE 8080
ENTRYPOINT ["java","-jar","/work/app.jar"]

5. Create IAM User

With all of the above being done, we can start the process of deployment to App Runner with Amazon RDS. Nevertheless, please make sure that you have already installed AWS Command Line Interface on your machine. We will need it to push our Docker image later.

If yes, then the first thing we have to do is create an IAM user. Let’s head to the Amazon Management Console and type “users” in the search bar:

Let’s select Users (IAM feature) and on the next page, hit the Add users button:

As the next step, let’s specify the User name and credential type. Please select the Access key – Programmatic access option:

With this setting, we will be able to push our Docker image using CLI.

Then, let’s click the Next: Permissions button and Create group on the next page:

Similarly, let’s specify the Group name and select the AdministratorAccess option:

Please keep in mind that in real-life scenarios, we would stick to the principle of least privilege and I picked the above option for the simplicity of this tutorial.

Nextly, let’s click the Next: Tags button:

We don’t need any tags for the purpose of this article, so let’s hit Next: Review:

Everything seems to be perfect, so let’s finalize the process with Create user button.

The user has been created successfully and we can hit the Download .csv button:

With that being done, let’s open the download directory with the terminal and run:

aws configure import --csv file://new_user_credentials.csv

# Result:
Successfully imported 1 profile(s)

The output indicates everything went perfectly. Nevertheless, we can double-check by simply going to the .aws directory and checking the credentials file. On Windows, it is C:\Users\[your-user]\.aws :

[app-runner-rds-user]
aws_access_key_id = <your-key-id>
aws_secret_access_key = <your-access-key>

Important note: Please remember that these data are confidential and you should never share them with anyone.

6. Push the Docker image to ECR

After that, we can push our Docker image to the Elastic Container Registry. This way, we will share it with our App Runner instance later.

Let’s get back to the Amazon Management Console and search for “registry” and select the appropriate option:

On the next page, let’s hit the Create repository button:

Following, let’s set the visibility settings to Private and specify the name of our registry. Let’s keep the rest of the settings with their defaults and hit the Create repository button:

Then, we should see that our repository was created successfully:

After that, let’s open our project root directory in the terminal and run the following:

docker build -t [your-repo-uri]:latest .

This step is required and URI has to match our private repository URI so that we will be able to push it.

Following, let’s authenticate to the ECR registry:

aws ecr get-login-password --profile app-runner-rds-user --region us-east-1 | docker login --username AWS --password-stdin [you-repo-uri]

#Result: 
Login Succeeded

Finally, let’s push our previously created Docker image (if you don’t remember the tag, you can simply run docker images command):

docker push [our_image_tag]

Of course, this step may take some time, but in the end, we should see the following in our repository:

7. Create RDS Service

As the next step, let’s create a new RDS (Managed Relational Database Service) instance.

Let’s type rds in the search bar and select the first option:

After that, let’s click the Create database button.

On the next page, let’s specify the Standard create along with PostgreSQL type, version, and Free tier template:

Following, let’s specify the database identifier with username and password (in our case, postgres and password values):

For the storage, let’s go with the minimum 20Gib value and disabled autoscaling:

Nextly, let’s configure the connectivity:

For the Database authentication, let’s keep the defaults and proceed to the Additional configuration:

After we disable all unnecessary options, let’s hit Create database. Similarly, this process may take a while.

Finally, we should see the green bar and the following details page:

Do not close it, we need these data later.

The last thing we would like to do here is to click the security group and add the All traffic Inbound rule for the 0.0.0.0/0 source (for simplicity):

8. Configure App Runner

With all of that being done, we can finally switch to the AWS App Runner setup process.

Let’s navigate to the App Runner page and hit the Create an App Runner service button. On the next page, let’s select the previously pushed image and create a new service role:

Nextly, let’s specify the service name and add one environment variable:

• DB_HOST– database endpoint value from RDS Management Console

In the Auto scaling, let’s select the Custom configuration and Add new. Then, let’s set the Maximum size to 1 (although it is not necessary, it is always a good practice to limit scaling to our needs):

For the Health check and Security, let’s leave the defaults. However, for Networking, let’s select Custom VPC and click Add new.

Although the connector name is not relevant, the rest of the values should be set properly:

• VPC and Security groups– to match database instance values
• Subnets– at least one value from RDS networking

After that, let’s click Next and on the next page, the Create & Deploy button.

After some time, we should see a green bar once again indicating that the service is up and running:

7. Testing

Finally, let’s test that everything is working, as expected:

curl https://{your-default-domain}.awsapprunner.com/api/users/non-db

# Result:
[
  {
    "id": 1,
    "email": "piotr@wolak.com"
  }
]

curl https://{your-default-domain}.awsapprunner.com/api/users

# Result:
[
  {
    "id": 1,
    "email": "john@doe.com"
  }
]

As we can see, both endpoints returned expected data, which indicates that everything is working fine.

8 Summary

And that would be all for this a bit long article. Together, we have learned how to deploy a Spring Boot application With AWS App Runner and Amazon RDS (using PostgreSQL).

I hope you enjoyed it and I will be thankful if you would like to leave a comment if you liked it. Thanks!

The post Deploy Spring Boot App With AWS App Runner And Amazon RDS appeared first on Codersee blog- Kotlin on the backend.

In this short article, I would like to show you how to configure Flyway when working with Spring WebFlux and Spring Data R2DBC.

In my previous tutorial, we’ve learned a lot about Flyway itself and how to set it up correctly, when working with Spring Boot. Nevertheless, if we would like to apply this knowledge when building a reactive web application with Spring WebFlux and Spring Data R2DBC, we would have to take a different approach. And that’s what we are going to focus on in this tutorial.

2. Imports

2.1. Create Spring WebFlux With R2DBC Project

As the first step, let’s create a skeleton project. We can do so by simply going to the Spring Initializr page and selecting the following dependencies:

• Spring Reactive Web
• Spring Data R2DBC
• PostgreSQL Driver

After that, we should see the following in our build.gradle.kts file:

implementation("org.springframework.boot:spring-boot-starter-data-r2dbc")
implementation("org.springframework.boot:spring-boot-starter-webflux")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor")
runtimeOnly("org.postgresql:postgresql")
runtimeOnly("org.postgresql:r2dbc-postgresql")

// other dependencies

2.2. Add Flyway Dependency

Unfortunately, we can’t add the Flyway dependency to our Spring WebFlux project on the same page, so we have to do it manually:

implementation("org.flywaydb:flyway-core:8.5.11")

With that being done, we have to reload Gradle changes and wait till the additional dependency is fetched to our environment.

3. Configure application.properties (or YAML) File

As the next step, let’s head to the application.properties file (or application.yaml in my case) and configure database connection along with Flyway:

spring:
  r2dbc:
    url: r2dbc:postgresql://localhost:5432/
    username: postgres
    password: password
  flyway:
    url: jdbc:postgresql://localhost:5432/
    user: postgres
    password: password

Please keep in mind that the values used for port, username, and password may be different in your case. Moreover, in the case of the Flyway connection, we have to use JDBC.

4. Implement Flyway Migration

Nextly, let’s prepare an SQL file, which we would like to run as the first migration. Let’s call it V1__My_First_Migration.sql:

CREATE TABLE person(
  id SERIAL NOT NULL PRIMARY KEY,
  email TEXT NOT NULL
);

As we can see, this simple script will create a new table called person. Please remember to put this file in the correct directory- resources/db/migration/ in our case.

5. Add Flyway Configuration For R2DBC

As the next step, let’s run our application to see if our Flyway migration is working. Technically, the app started successfully. Nevertheless, logs do not contain anything about applied migration and when we check the database we can clearly see, that migration was not run.

To change that, let’s add a FlywayConfiguration file to our project:

@Configuration
class FlywayConfiguration(private val env: Environment) {

    @Bean(initMethod = "migrate")
    fun flyway(): Flyway {
        return Flyway(Flyway.configure()
            .dataSource(
                env.getRequiredProperty("spring.flyway.url"),
                env.getRequiredProperty("spring.flyway.user"),
                env.getRequiredProperty("spring.flyway.password"))
        )
    }
}

Alternatively, we can use @Value annotation, as well:

@Configuration
class FlywayConfiguration(
    @Value("\${spring.flyway.url}") private val url: String,
    @Value("\${spring.flyway.user}") private val user: String,
    @Value("\${spring.flyway.password}") private val password: String
) {

    @Bean(initMethod = "migrate")
    fun flyway(): Flyway {
        return Flyway(
            Flyway.configure()
                .dataSource(url, user, password)
        )
    }
}

As we can see, this configuration is responsible for creating a new Flyway bean with the URL, username, and password obtained from our application.yaml file.

If we rerun our app once again, we will see the following:

Creating Schema History table "public"."flyway_schema_history" ...
Current version of schema "public": << Empty Schema >>
Migrating schema "public" to version "1 - My First Migration"
Successfully applied 1 migration to schema "public", now at version v1 (execution time 00:00.034s)

The above message clearly indicates that the migration was run and we can expect a new table in our database.

6. Summary

And that would be all for this short article on how to configure Flyway migrations with Spring WebFlux and R2DBC.

Let me know if you found this material useful in the comment sections below, or by using the contact form.

Finally, if you would like to see the whole source code, please refer to this GitHub repository.

The post Flyway Migrations With Spring WebFlux (R2DBC) appeared first on Codersee blog- Kotlin on the backend.

In this blog post, I would like to show you what exactly Flyway is and how to configure it, when working with Spring Boot.

Together, we will spend some time to figure out:

• why do we even need it in our project?
• how can we quickly configure Flyway with Spring Boot?
• finally, how can we customize its behavior?

2. What is Flyway and What Problems Does It Solve?

To put it simply, Flyway is a database versioning tool. Just like Git is a version control system for our codebase, Flyway takes care of database schema management.

Let’s imagine that we’ve started a Spring Boot project, which communicates with the PostgreSQL database. We’ve been working on our own, implementing new features, adding new tables, and changing the schema many times. At some point, a new programmer joins our team, so we simply export our local or remote database and send him the script, so that he will be able to set up a local environment.

So far so good, but what will happen when more people join our team? Or we would like to create more testing environments? Of course, we could save the script and update it each time, we change anything, but this can introduce plenty of problems, for example:

• when multiple people are editing schema asynchronously, how can we make sure that all their scripts are applied in a specific environment?
• how can we be sure that all developers have exactly the same (or a specific) version on their local?
• how can we determine the correct order of scripts, so that nothing breaks when applying them?

If you are not convinced after reading the above points, then imagine working without Git when cooperating with others and exchanging written code manually 🙂 Technically- doable- nevertheless definitely not recommended.

Of course, Flyway is not the only tool we could use with Spring Boot, however, it is definitely easy to work with.

[elementor-template id=”9007393″]

3. Setup Flyway With Spring Boot

With all of that being said, let’s switch to the practice part. If you would like to simply download the skeleton for this tutorial, I’ve uploaded it to this GitHub repository. Nevertheless, I highly encourage you to set up the Spring Boot project on your own and follow below steps. This way, you’re definitely gonna learn more.

3.1. Imports

As the first step, let’s add the necessary imports:

implementation("org.flywaydb:flyway-core:10.13.0")
runtimeOnly("org.flywaydb:flyway-database-postgresql:10.13.0")
implementation("org.postgresql:postgresql:42.7.3")
implementation("org.springframework.boot:spring-boot-starter-data-jdbc") //alternatively, we could use spring-boot-starter-data-jpa

The above 4 are the bare minimum required to work with Flyway and Spring Boot.

3.2. Edit Application Properties

As the next step, we have to make sure that we’ve set up the database connection properly.

The easiest way to do so is via application.yaml (or application.properties) file:

spring:
  datasource:
    url: "jdbc:postgresql://localhost:5432/your-db-name"
    username: db-username
    password: db-password

By default, Flyway will use the @Primary Data Source (which we’ve configured above).

3.3. Add Migrations Directory

Nextly, we have to add /db/migration directory to the /resources. If we don’t do so and try to run the application, it will fail with the following error:

Flyway failed to initialize: none of the following migration scripts locations could be found:

classpath:db/migration

Of course, we can customize that, and we will learn how to do that later.

3.4. Validation

For now, let’s check if our setup is working correctly (please make sure that you have a database with an empty schema up and running).

After we run the application, we should see the following in the logs:

Flyway Community Edition 8.5.11 by Redgate
...
Successfully validated 0 migrations (execution time 00:00.009s)
No migrations found. Are your locations set up correctly?
Creating Schema History table "public"."flyway_schema_history" ...
Schema "public" is up to date. No migration necessary.

Moreover, we can see that a new table called flyway_schema_history was added to our schema.

In brief, Flyway uses this table to manage versions and control applied, modified or added scripts. If you would like to see more theory, then this article from their documentation is a great source for that.

3.5. Add a Migration

Finally, let’s add our first migration file called V1__Create_User_Table.sql. This “strange” filename is the default Flyway naming convention, which consists of:

• a prefix– B for baseline, R for repeatable,  U for undo, and V for versioned migrations
• migration number– number 1 in our case. If we would like it to be 1.1, then we should start the name with V1_1
• double underscore– separates version from migration description
• description– underscores are translated to spaces, so our migration will become Create User Table

With that being said, let’s add an example SQL script:

CREATE TABLE IF NOT EXISTS app_user (
  id SERIAL NOT NULL PRIMARY KEY,
  name TEXT NOT NULL
);

After that, let’s run our Spring Boot application once again and check out printed logs:

Migrating schema "public" to version "1 - Create User Table"

As can be seen, migration was applied successfully and a new table was added to our database.

4. Flyway Customization With application.properties

4.1. Enable/Disable Flyway

By default, Flyway is enabled when added to Spring Boot. Nevertheless, if we would like to change that, we can set the enabled flag:

spring:
  flyway:
    enabled: false

4.2. Set Custom Migrations Directory

As I’ve mentioned earlier, the default directory is /db/migration, but it can be customized with spring.flyway.locations property:

spring:
  flyway:
    locations: "classpath:my-custom-dir,classpath:my-custom-dir-2"

As we can see, we can provide multiple directories. When working with Spring Boot, a classpath means resources directory. Alternatively, we can refer to any directory in the system using filesystem instead.

It’s worth mentioning that when we specify multiple directories, then at least one of them has to exist (but not all of them). Moreover, we can’t duplicate migration versions among our directories.

So, in practice:

/resources
    /my-custom-dir
        V2__one.sql
    /my-custom-dir-2
        V1__one.sql

These two directories are treated, just like one and if we would specify V1 version twice, then the FlywayException would be thrown:

Found more than one migration with version 1

4.3. Vendor Aware Directory

Additionally, Flyway allows us not only to specify hard-coded paths, but we can use a vendor placeholder, which will be resolved on the fly:

spring:
  flyway:
    locations: "classpath:my-custom-dir/{vendor}"

With the above config, Flyway will look for migrations files inside the /resources/my-custom-dir/postgresql directory. You can find a full list of supported vendors right here.

5. Flyway Java (Kotlin) Configuration

It’s worth mentioning that Flyway allows us to apply migrations written in Java. When working with Spring Boot, they will be auto-configured with any bean implementing a JavaMigration interface. However, this interface requires us to implement a few methods, so the recommended way is to extend the BaseJavaMigration abstract class instead (and sufficient in most cases).

Given that, let’s have a look at the example written in Kotlin (which technically does not differ too much from its Java counterpart):

@Component
class V1__Create_User_Table : BaseJavaMigration() {
    override fun migrate(context: Context) {
        val update = context.connection.createStatement()
        update.execute(
            """
            CREATE TABLE IF NOT EXISTS app_user (
                id SERIAL NOT NULL PRIMARY KEY, 
                name TEXT NOT NULL
            );
            """.trimIndent()
        )
    }
}

@Component
class V2__Drop_User_Table : BaseJavaMigration() {
    override fun migrate(context: Context) {
        val update = context.connection.createStatement()
        update.execute("DROP TABLE app_user;")
    }
}

As we can see, the only thing we have to do is to override the migrate method, and that class names should follow the default Flyway naming convention.

If you are wondering whether we can mix SQL files with Spring Beans configuration, then the answer is yes (however we should be really cautious about it).

6. Flyway With Spring Boot Summary

And that would be all for this tutorial about Flyway With Spring Boot. I believe the knowledge presented here will be sufficient to easily incorporate this tool into any Spring Boot project. Moreover, if you’d like to learn how to incorporate Flyway to an existing Spring Boot project, or how to use it with Spring WebFlux, then check out the flyway tag.

Let me know what you think about database version control tools. Do you like Flyway, or maybe you prefer some other tool?

Have a great day and see you in the next articles! 🙂

The post Flyway Migrations With Spring Boot appeared first on Codersee blog- Kotlin on the backend.

In this step-by-step tutorial, I would like to show you how to create a simple REST API using Ktor, Ktorm, and PostgreSQL (aka Postgres).

Ktor is a free, open-source framework, which allows us to easily build connected applications. When it comes to server-side applications, we can treat it as a lightweight and powerful alternative to Spring Boot. Additionally, it was written in Kotlin programming language by its’ creators- the JetBrains team.

Ktorm, on the other hand, is a lightweight and efficient ORM Framework for Kotlin directly based on pure JDBC. Furthermore, combined with PostgreSQL makes an interesting combination for a REST API tech stack.

2. Generate Ktor Project

With all of that being said, let’s create the Ktor project (and in the next chapter, I will show you how to add Ktorm and PostgreSQL dependencies).

If you are the owner of IntelliJ Ultimate Edition, then it ships with Ktor support out of the box. Nevertheless, in this tutorial, we’ll focus on the alternative: https://start.ktor.io, which is pretty similar to the Spring Initializr page.

As the first step, let’s specify the project name along with other, necessary data:

Please remember about selecting the same Ktor version, so that the code from this tutorial will work for sure. Don’t worry about a sample code generated, we will take care of it later.

As the next step, let’s click the Add plugins button and add the kotlinx.serialization– we’ll use it to serialize our objects into JSON format:

As we can see, two other plugins have been added automatically: ContentNegotiation and Routing. Please do not remove them, we will need them later.

[elementor-template id=”9007393″]

2. Add Ktorm and PostgreSQL To Ktor

Nextly, let’s open up our Ktor Postgres project with IntelliJ and go to the gradle.properties file.

Unfortunately, the Project Generator does not allow us to add all of the necessary imports, so we will have to do it manually:

postgresql_driver_version=42.3.1
ktorm_version=3.4.1

As the next step, let’s add a few lines within the build.gradle.kts file and reload the Gradle project:

val ktorm_version: String by project
val postgresql_driver_version: String by project

dependencies {

// other dependencies

implementation("org.ktorm:ktorm-core:$ktorm_version")
implementation("org.ktorm:ktorm-support-postgresql:$ktorm_version")
implementation("org.postgresql:postgresql:$postgresql_driver_version")

}

As we can see, these dependencies are necessary in order to connect our Ktor application with the Postgres database.

3. Clean Up The Code

After that, let’s clean up the project a bit and delete the Routing.kt along with its invocation within the Application.kt.

The second file should now be looking, as follows:

fun main() {
  embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
    configureSerialization()
  }.start(wait = true)
}

Then, let’s go to the Serialization.kt file and remove unnecessary code, as well:

fun Application.configureSerialization() {
  install(ContentNegotiation) {
    json()
  }
}

The above code will be necessary for JSON serialization.

4. Prepare PostgreSQL With Docker Compose

Basically, if you already have a PostgreSQL instance running on your machine, you can just skip the docker-compose part and just run the below SQL script.

Either way, let’s create the sql folder in the root directory and add the init-db.sql file:

CREATE TABLE book
(
  id   SERIAL NOT NULL,
  name varchar NOT NULL
);

INSERT INTO book(name) VALUES ('Book #1');
INSERT INTO book(name) VALUES ('Book #2');

We can clearly see, that after the script is run, a new table will be added and populated with test data.

Following, let’s go to the root of the Ktor project and add the docker-compose.yaml file with Postgres definition:

version: "3.9"
services:
  postgres-sandbox:
    image: postgres:14
    ports:
      - '5438:5432'
    volumes:
      - ./sandbox-db:/var/lib/postgresql/data
      - ./sql/init-db.sql:/docker-entrypoint-initdb.d/init-db.sql
    environment:
      - POSTGRES_NAME=postgres
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres

The above code is responsible for a few things:

• pulling the Docker image for PostgreSQL
• publishing the container’s 5432 port to the host’s 5438 (to put it simply- we will connect from our application using port 5438)
• running initialization script prepared above
• and finally, setting credentials through environment variables

To start our PostgreSQL instance, let’s run the below command:

docker-compose up

# To validate:
docker ps
CONTAINER ID IMAGE       COMMAND                CREATED       STATUS       PORTS                  NAMES
2f5716235039 postgres:14 "docker-entrypoint.s…" 6 minutes ago Up 6 minutes 0.0.0.0:5438->5432/tcp ktor-ktorm-postgresql_postgres-sandbox_1

As we can see, everything is working fine and now we will be able to connect our Ktor app with Postgres.

5. Implement DTOs

Following, let’s prepare request and response classes, which will be used in our REST API. Technically, we don’t have to do that, but it’s a good practice to keep them separate.

Let’s start with the BookRequest:

@Serializable
data class BookRequest(
  val name: String
)

Similarly, let’s implement the BookResponse and ErrorResponse:

@Serializable
data class BookResponse(
  val id: Long,
  val name: String
)

@Serializable
data class ErrorResponse(val message: String)

You might have noticed, that both classes are annotated with @Serializable (from kotlinx.serialization library). According to the documentation:

Applying Serializable to the Kotlin class instructs the serialization plugin to automatically generate implementation of KSerializer for the current class, that can be used to serialize and deserialize the class.

To put it simply, this annotation is necessary if we want to operate on these two classes in our REST API.

6. Add Entity And Columns Bindings

With all of that being done, let’s add the code responsible for object-relational mapping:

interface Book : Entity<Book> {
  companion object : Entity.Factory<Book>()

  val id: Long?
  var name: String
}

object Books : Table<Book>("book") {
  val id = long("id").primaryKey().bindTo(Book::id)
  val name = varchar("name").bindTo(Book::name)
}

It may seem a bit counterintuitive in the beginning, so let’s spend some time understanding it better.

The first one, Book, is a representation of the entity class, which will be used to operate on it. You might have already noticed, that it’s been declared as an interface- that’s a design requirement of Ktorm. (Ktorm 2.5 introduced the possibility to declare them as classes, but this approach has some limitations, which you can read more about here).

Additionally, you might be curious, what exactly does this line do?

companion object : Entity.Factory<Book>()

Basically, interfaces cannot be instantiated, but to help us with that, Ktorm provides an abstract class Entity.Factory. It overloads the invoke operator so that we will be able to create Book instances and initialize its properties pretty easily:

val newBook = Book {
  name = bookRequest.name
}

Finally, the second part, an object called Books is responsible for binding our entity with the database table.

It allows us to specify columns types, and names and wire them with our entity through the .bindTo() method. Whatsoever, we will use this object later, when working with our Ktor Postgres database connection.

7. Connect To PostgreSQL

As the next step, let’s implement the BookService and connect to the database:

class BookService {

  private val database = Database.connect(
    url = "jdbc:postgresql://localhost:5438/postgres",
    driver = "org.postgresql.Driver",
    user = "postgres",
    password = "postgres"
  )
}

As we can see, the connect function allows us to specify Ktor Postgres connection details. Please make sure that these values match the actual state in your case. As a result, we receive a new database object, which we will use to perform DB operations.

8. Implement Service Methods

With all of that being done, we can finally add logic responsible for books management.

Let’s go to the BookService and add 4 methods:

fun createBook(bookRequest: BookRequest): Boolean {
  val newBook = Book {
    name = bookRequest.name
  }

  val affectedRecordsNumber = 
    database.sequenceOf(Books)
      .add(newBook)

  return affectedRecordsNumber == 1
}

fun findAllBooks(): Set<Book> =
  database.sequenceOf(Books).toSet()

fun findBookById(bookId: Long): Book? =
  database.sequenceOf(Books)
    .find { book -> book.id eq bookId }

fun updateBookById(bookId: Long, bookRequest: BookRequest): Boolean {
  val foundBook = findBookById(bookId)
    foundBook?.name = bookRequest.name

  val affectedRecordsNumber = foundBook?.flushChanges()

  return affectedRecordsNumber == 1
}
fun deleteBookById(bookId: Long): Boolean {
  val foundBook = findBookById(bookId)

  val affectedRecordsNumber = foundBook?.delete()

  return affectedRecordsNumber == 1
}

8.1. Create Book

Let’s take a while to analyze each method:

fun createBook(bookRequest: BookRequest): Boolean {
  val newBook = Book {
    name = bookRequest.name
  }

  val affectedRecordsNumber = 
    database.sequenceOf(Books)
      .add(newBook)

  return affectedRecordsNumber == 1
}

As we can see, the createBook() creates a new Book instance (thanks to the companion object) and inserts it into the table using the database object.

The sequenceOf(), instantiates a new EntitySequence object. It exposes a sequence API, which we can use to perform DB operations, like querying or inserting data.

Finally, the .add() method inserts our entity into the database and returns the number of affected records.

The last line of this function simply validates, whether the number of affected records is equal to one (any other number indicates that something went wrong).

8.2. Fetching Books

Nextly, let’s check the querying functions:

fun findAllBooks(): Set<Book> =
  database.sequenceOf(Books).toSet()

fun findBookById(bookId: Long): Book? =
  database.sequenceOf(Books)
    .find { book -> book.id eq bookId }

They are pretty much the same, except for the fact, that in the second case the .find{} function will be translated to the WHERE clause in the generated SQL.

8.3. Update Book

As the next one, let’s check out the updateBookById():

fun updateBookById(bookId: Long, bookRequest: BookRequest): Boolean {
  val foundBook = findBookById(bookId)
  foundBook?.name = bookRequest.name

  val affectedRecordsNumber = foundBook?.flushChanges()

  return affectedRecordsNumber == 1
}

We can clearly see, that we can operate on the fetched entity, just like on any object in Kotlin. Nevertheless, for the changes to take effect, we must invoke the .flushChanges() function, which returns the affected records number.

8.4. Delete Book

Finally, the deleteBookById():

fun deleteBookById(bookId: Long): Boolean {
  val foundBook = findBookById(bookId)

  val affectedRecordsNumber = foundBook?.delete()

  return affectedRecordsNumber == 1
}

Although it’s performing a different action, the mechanism behind it remains the same.

9. Implement Book Routes

With all of that being done, we can finally add the necessary routes.

In Ktor, Routing is a core plugin responsible for handling incoming requests. If you’ve ever been working with Spring Boot before, then you may find it similar to the Controller concept.

9.1. Edit Application.kt

As the first step, let’s get back to the Application.kt and add configureBookRoutes() invocation:

fun main() {
  embeddedServer(Netty, port = 8080, host = "0.0.0.0") {
    configureBookRoutes()
    configureSerialization()
  }.start(wait = true)
}

9.2. Create BookRoutes.kt

Secondly, let’s add the BookRoutes.kt file with the necessary function:

fun Application.configureBookRoutes() {
  routing {
    route("/book") {
      val bookService = BookService()
      createBook(bookService)
      getAllBooksRoute(bookService)
      getBookByIdRoute(bookService)
      updateBookByIdRoute(bookService)
      deleteBookByIdRoute(bookService)
    }
  }
}

private fun Book?.toBookResponse(): BookResponse? =
  this?.let { BookResponse(it.id!!, it.name) }

To put it simply, the routing block installs a Routing feature in our application. On the other hand, the route builds a route to match the specified path. This way, we can keep relevant functions together and avoid duplicating the /book path for each of them.

When it comes to the toBookResponse(), this extension function will help us to convert Book entities into BookResponses.

9.3. Add Create Book Route

With this in mind, let’s implement the rest of the code step by step. As the first one, let’s add the createBook():

fun Route.createBook(bookService: BookService) {
  post {
    val request = call.receive<BookRequest>()

    val success = bookService.createBook(bookRequest = request)

    if (success)
      call.respond(HttpStatusCode.Created)
    else
      call.respond(HttpStatusCode.BadRequest, ErrorResponse("Cannot create book"))
  }
}

The above code will be responsible for handling POST /book requests. As a word of explanation, the call.receive<BookRequest>() deserializes the content of the request (JSON) into the BookRequest instance. When the content cannot be translated, it throws ContentTransformationException (it’s worth to remember in real-life scenarios).

On the contrary, the call.respond() is responsible for sending responses to the client. We can clearly see that if creation is successful, the 201 Created is returned. However, if this is not the case, then the 400 Bad Request with an appropriate message is returned.

9.4. Implement Get All Books Route

As the next step, let’s add the getAllBooksRounte():

fun Route.getAllBooksRoute(bookService: BookService) {
  get {
    val books = bookService.findAllBooks()
      .map(Book::toBookResponse)

    call.respond(message = books)
  }
}

This one will be triggered on GET /book requests returning the list of books.

9.5. Add Get Book By Id Route

With that being done, let’s add the getBookByIdRoute() responsible for handling GET /book/{id} requests:

fun Route.getBookByIdRoute(bookService: BookService) {
  get("/{id}") {
    val id: Long = call.parameters["id"]?.toLongOrNull()
      ?: return@get call.respond(HttpStatusCode.BadRequest, ErrorResponse("Invalid id"))

    bookService.findBookById(id)
      ?.let { foundBook -> foundBook.toBookResponse() }
      ?.let { response -> call.respond(response) }
      ?: return@get call.respond(HttpStatusCode.BadRequest, ErrorResponse("Book with id [$id] not found"))
  }
}

This time, let’s focus on the call.parameters[“id”]. The Parameters property of the call is simply a map of case-insensitive names to a collection of String values. That’s why we have to use a safe ?.toLongOrNull() to obtain the value correctly. If the id is invalid, then the 400 Bad Request is returned.

The rest of the code is pretty straightforward- either the found book is mapped to the BookResponse instance and returned to the client, or a 400 Bad Request informing that the book with a given id was not found.

9.6. Insert Update and Delete Routes

Finally, let’s add the two routes responsible for handling PATCH /book/{id} and DELETE /book/{id} requests:

fun Route.updateBookByIdRoute(bookService: BookService) {
  patch("/{id}") {
    val id: Long = call.parameters["id"]?.toLongOrNull()
      ?: return@patch call.respond(HttpStatusCode.BadRequest, ErrorResponse("Invalid id"))

    val request = call.receive<BookRequest>()
    val success = bookService.updateBookById(id, request)

    if (success)
      call.respond(HttpStatusCode.NoContent)
    else
      call.respond(HttpStatusCode.BadRequest, ErrorResponse("Cannot update book with id [$id]"))
  }
}

fun Route.deleteBookByIdRoute(bookService: BookService) {
  delete("/{id}") {
    val id: Long = call.parameters["id"]?.toLongOrNull()
      ?: return@delete call.respond(HttpStatusCode.BadRequest, ErrorResponse("Invalid id"))

    val success = bookService.deleteBookById(id)

    if (success)
      call.respond(HttpStatusCode.NoContent)
    else
      call.respond(HttpStatusCode.BadRequest, ErrorResponse("Cannot delete book with id [$id]"))
  }
}

As we can see, these functions use already described concepts, so we won’t go into much detail.

10. Test With cURL

Finally, let’s run the application and test it manually. For simplicity, we will use the cURL command. Alternatively, we could do that through Postman, or Insomnia.

10.1. GET /book

Let’s start by listing all books to validate both the endpoint and SQL init script:

curl localhost:8080/book

# Output
[
  {
    "id":1,
    "name": "Book #1"
  },
  {
    "id":2,
    "name":"Book #2"
  }
]

As we can see, everything is working correctly.

10.2. POST /book

As the second step, let’s create a new book:

curl -X POST 'localhost:8080/book' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "name": "Book #3"
  }'

# Then again: 
curl localhost:8080/book

# Output
[
  {
    "id":1,
    "name": "Book #1"
  },
  {
    "id":2,
    "name":"Book #2"
  },
  {
    "id":3,
    "name":"Book #3"
  }
]

10.3. GET /book/{id}

As the third step, let’s check whether fetching book details works:

curl localhost:8080/book/3

# Output:
{
  "id":3,
  "name":"Book #3"
}

curl localhost:8080/book/1234

# Output:
{
  "message":"Book with id [1234] not found"}
}

10.4. PATCH /book/{id}

After that, let’s validate the PATCH endpoint:

curl -X PATCH 'localhost:8080/book/3' \
  --header 'Content-Type: application/json' \
  --data-raw '{
    "name": "Book #3- edited"
  }'

# Then, let's check with GET:
curl localhost:8080/book/3

# Output:
{
  "id": 3,
  "name": "Book #3- edited"
}

10.5. DELETE /book/{id}

Finally, let’s delete created book:

curl -X DELETE 'localhost:8080/book/3'

# Then, let's check with GET:
curl localhost:8080/book/3

# Output:
{
  "message":"Book with id [3] not found"
}

11. Ktor With Ktorm And PostgreSQL Summary

And that would be all for this step-by-step guide on how to create REST API with Ktor, Ktorm, and PostgreSQL. I really hope you will find this material useful when working with Ktor. If you would like to see the source code, please refer to this GitHub repository.

If you find this one useful, then you may enjoy my newer article about Ktor and MongoDB.

As always- if you would like to ask me about anything, please do that in the comments section below, or through the contact form.

Previous articles, you might be interested in:

• Exception Handling with @RestControllerAdvice and @ExceptionHandler
• 16 IntelliJ Shortcuts That Will Boost Your Productivity
• SOLID Principles with Kotlin Examples

The post REST API With Ktor, Ktorm and PostgreSQL appeared first on Codersee blog- Kotlin on the backend.