Apache Spark works very well in reading several files for data extraction, in this post we'll create an example of reading a CSV file using Spark, Java and Maven. Maven org.apache.spark spark-core_2.12 3.1.0 org.apache.spark spark-sql_2.12 3.1.0 CSV Content Let's suppose that file's name below is movies.csv. title;year;rating The Shawshank Redemption;1994;9.3 The Godfather;1972;9.2 The Dark Knight;2008;9.0 The Lord of the Rings: The Return of the King ;2003;8.9 Pulp Fiction;1994;8.9 Fight Club;1999;8.8 Star Wars: Episode V - The Empire Strikes Back;1980;8.7 Goodfellas;1990;8.7 Star Wars;1977;8.6 Creating a SparkSession SparkConf sparkConf = new SparkConf(); sparkConf.setMaster("local[*]"); sparkConf.setAppName("app"); SparkSession sparkSession = SparkSession.builder() .config(sparkConf) .getOrCreate(); Running the Read Dataset ds = sparkSession.read() .format("CSV") .option("sep",";") .option("inferSchema", "true") .option("header", "true") .load("movies.csv"); ds.select("title","year","rating").show(); Result Understanding some parameters .option("sep", ";"): Defines the use of a default delimiter for file reading, in this case the delimiter is a semicolon (;) .option("inferSchema", "true"): The inferSchema parameter makes it possible to infer the file(s) in order to understand (guess) the data types of each field .option("header", "true"): Enabling the header parameter makes it possible to use the name of each field defined in the file header .load("movies.csv"): movies.csv is the name of the file to be read Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Spark: The Definitive Guide: Big Data Processing Made Simple is a complete reference for those who want to learn Spark and about the main Spark's feature. Reading this book you will understand about DataFrames, Spark SQL through practical examples. The author dives into Spark low-level APIs, RDDs and also about how Spark runs on a cluster and how to debug and monitor Spark clusters applications. The practical examples are in Scala and Python. Beginning Apache Spark 3: With Dataframe, Spark SQL, Structured Streaming, and Spark Machine Library with the new version of Spark, this book explores the main Spark's features like Dataframes usage, Spark SQL that you can uses SQL to manipulate data and Structured Streaming to process data in real time. This book contains practical examples and code snippets to facilitate the reading. High Performance Spark: Best Practices for Scaling and Optimizing Apache Spark is a book that explores best practices using Spark and Scala language to handle large-scale data applications, techniques for getting the most out of standard RDD transformations, how Spark SQL's new interfaces improve performance over SQL's RDD data structure, examples of Spark MLlib and Spark ML machine learning libraries usage and more. Maven: The Definitive Guide Written by Maven creator Jason Van Zyl and his team at Sonatype, Maven: The Definitive Guide clearly explains how this tool can bring order to your software development projects. In this book you'll learn about: The POM and Project Relationships, The Build Lifecycle, Plugins, Project website generation, Advanced site generation, Reporting, Properties, Build Profiles, The Maven Repository and more. Well that’s it, I hope you enjoyed it!

Before starting to talk about access to states, it is necessary to explain what states or State are. What are States? What's Terraform State? Terraform State is a way for Terraform to manage the infrastructure, configurations and resources created in order to maintain a mapping of what already exists and control the update and creation of new resources. A basic example is when we create an S3 Bucket, an EC2 instance or an SQS via Terraform. All these resources are mapped in the state and are managed by Terraform. State locations Local By default Terraform allocates state locally in the terraform.tfsate file. Using the State locally can work well for a specific scenario where there is no need to share the State between teams. Remote Unlike Local, when we have teams sharing the same resources, using State remotely becomes essential. Terraform provides support so that State can be shared remotely. We won't go into detail on how to configure it, but it's possible to keep State in Amazon S3, Azure Blob Storage, Google Cloud Storage, Alibaba Cloud OSS, and other cloud services. The State is represented by the terraform.tfsate file in JSON format, here is an example of a S3 Bucket mapped on State: { "version": 4, "terraform_version": "0.12.3", "serial": 3, "lineage": "853d8b-4ee1-c1e4-e61e-e10", "outputs": {}, "resources": [ { "mode": "managed", "type": "aws_s3_bucket", "name": "s3_bucket_xpto", "provider": "provider.aws", "instances": [ { "schema_version": 0, "attributes": { "acceleration_status": "", "acl": "private", "arn": "arn:aws:s3:::bucket.xpto", "bucket": "bucket.xpto", "bucket_domain_name": "bucket.xpto", "bucket_prefix": null, "bucket_regional_domain_name": "bucket.xpto", "cors_rule": [], "force_destroy": false, "grant": [], "hosted_zone_id": "Z3NHGSIKTF", "id": "bucket.xpto", "lifecycle_rule": [], "logging": [], "object_lock_configuration": [], "policy": null, "region": "us-east-1", "replication_configuration": [], "request_payer": "BucketOwner", "server_side_encryption_configuration": [], "tags": { "Environment": "development" }, "versioning": [ { "enabled": false, "mfa_delete": false } ], "website": [], "website_domain": null, "website_endpoint": null }, "private": "Ud4JbhV==" } ] } ] } Accessing and updating the State Despite the State being allocated in a JSON file, it is not recommended to change it directly in the file. Terraform provides the use of the Terraform state commands executed via CLI so that small modifications can be made. Through the CLI, we can execute commands in order to manipulate the State, as follows: terraform state [options] [args] Sub-commands: list List the resources in the state mv Move an item in state pull Extract the current state and list the result on stdout push Update a remote state from a local state file rm Remove an instance from state show Show state resources 1. Listing State resources Command: terraform state list The above command makes it possible to list the resources being managed by State Example: $ terraform state list aws_s3_bucket.s3_bucket aws_sqs_queue.sqs-xpto In the example above, we have as a result, an S3 and an SQS Bucket that were created via terraform and are being managed by State. 2. Viewing a resource and its attributes Command: terraform state show [options] RESOURCE_ADDRESS The above command makes it possible to show in detail a specific resource and its attributes Example: $ terraform state show aws_sqs_queue.sqs-xpto # aws_sqs_queue.sqs-xpto: resource "aws_sqs_queue" "sqs-xpto" { arn = "arn:aws:sqs:sqs-xpto" content_based_deduplication = false delay_seconds = 90 fifo_queue = false id = "https://sqs-xpto" kms_data_key_reuse_period_seconds = 300 max_message_size = 262144 message_retention_seconds = 345600 name = "sqs-xpto" receive_wait_time_seconds = 10 tags = { "Environment" = "staging" } visibility_timeout_seconds = 30 } 3. Removing resources from the State Command: terraform state rm [options] RESOURCE_ADDRESS The above command removes one or more items from the State. Unlike a terraform destroy command, which removes the State resource and remote objects created in the cloud. Example: $ terraform state rm aws_sqs_queue.sqs-xpto Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Terraform: Up & Running: Writing Infrastructure as Code is a book focused on how to use Terraform and its benefits. The author make comparisons with several other IaC (Infrastructure as code) tools such as Ansible and Cloudformation (IaC native to AWS) and especially how to create and provision different resources for multiple cloud services. Currently, Terraform is the most used tool in software projects for creating and managing resources in cloud services such as AWS, Azure, Google Cloud and many others. If you want to be a complete engineer or work in the Devops area, I strongly recommend learning about the topic. AWS Cookbook is a practical guide containing 70 familiar recipes about AWS resources and how to solve different challenges. It's a well-written, easy-to-understand book covering key AWS services through practical examples. AWS or Amazon Web Services is the most widely used cloud service in the world today, if you want to understand more about the subject to be well positioned in the market, I strongly recommend the study. Well that’s it, I hope you enjoyed it!

Java 8 Streams introduced different methods for handling collections. One of these methods is the findFirst(), which allows returning the first element of a Stream through an Optional instance. Example Output Item: Monica Souza Using filter Output Item: Andre Silva Note that it returned the first name with last name Silva from the collection. Not using Streams If we use the traditional way without using Streams. The code would look like this, filtering by last name "Silva" In this case we depend on the break to end the execution. Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Head First Design Patterns: Building Extensible and Maintainable Object-Oriented Software is a book that through Java examples shows you the patterns that matter, when to use them and why, how to apply them to your own designs, and the object-oriented design principles on which they're based. Head First Java is a complete learning experience in Java and object-oriented programming. With this book, you'll learn the Java language with a unique method that goes beyond how-to manuals and helps you become a great programmer. Through puzzles, mysteries, and soul-searching interviews with famous Java objects, you'll quickly get up to speed on Java's fundamentals and advanced topics including lambdas, streams, generics, threading, networking, and the dreaded desktop GUI. Well that’s it, I hope you enjoyed it!

What's Delta Lake? Delta Lake is an open-source project that manages storage layer in your Data lake. In practice it's an Apache Spark abstraction reusing the same mechanisms offering extra resources such as ACID transactions support. Everyone knows that keeping data integrity in a data pipeline is a critical task in face of high data read and write concurrency. Delta Lake provides audit history, data versioning and supports DML operations such as deletes, updates and merges. For this tutorial, we're going to simulate a data pipeline locally focusing on Delta Lake advantages. First, we'll load a Spark Dataframe from a JSON file, a temporary view and then a Delta Table which we'll perform some Delta operations. Last, Java as programming language and Maven as dependency manager, besides Spark and Hive to keep our data catalog. Maven org.apache.spark spark-core_2.12 3.0.1 org.apache.spark spark-sql_2.12 3.0.1 org.apache.spark spark-hive_2.12 3.0.1 io.delta delta-core_2.12 0.8.0 The code will be developed in short snippets to a better understanding. Setting Spark with Delta and Hive String val_ext="io.delta.sql.DeltaSparkSessionExtension"; String val_ctl="org.apache.spark.sql.delta.catalog.DeltaCatalog"; SparkConf sparkConf = new SparkConf(); sparkConf.setAppName("app"); sparkConf.setMaster("local[1]"); sparkConf.set("spark.sql.extensions",var_ext); sparkConf.set("spark.sql.catalog.spark_catalog",val_ctl); SparkSession sparkSession = SparkSession.builder() .config(sparkConf) .enableHiveSupport() .getOrCreate(); Understanding the code above We define two variables val_ext and val_ctl by assigning the values ​​to the keys (spark.sql.extensions and spark.sql.catalog.spark_catalog). These are necessary for configuring Delta together with Spark We named the Spark context of app Since we are not running Spark on a cluster, the master is configured to run local local[1] Spark supports Hive, in this case we enable it in the enableHiveSupport( ) Data Ingest Let's work with Spark Dataframe as the data source. We load a Dataframe from a JSON file. order.json file {"id":1, "date_order": "2021-01-23", "customer": "Jerry", "product": "BigMac", "unit": 1, "price": 8.00} {"id":2, "date_order": "2021-01-22", "customer": "Olivia", "product": "Cheese Burguer", "unit": 3, "price": 21.60} {"id":3, "date_order": "2021-01-21", "customer": "Monica", "product": "Quarter", "unit": 2, "price": 12.40} {"id":4, "date_order": "2021-01-23", "customer": "Monica", "product": "McDouble", "unit": 2, "price": 13.00} {"id":5, "date_order": "2021-01-23", "customer": "Suzie", "product": "Double Cheese", "unit": 2, "price": 12.00} {"id":6, "date_order": "2021-01-25", "customer": "Liv", "product": "Hamburger", "unit": 1, "price": 2.00} {"id":7, "date_order": "2021-01-25", "customer": "Paul", "product": "McChicken", "unit": 1, "price": 2.40} Creating a Dataframe Dataset df = sparkSession.read().json("datasource/"); df.createOrReplaceGlobalTempView("order_view"); Understanding the code above In the previous section, we're creating a Dataframe from the JSON file that is inside the datasource/ directory, create this directory so that the structure of your code is more comprehensive and then create the order.json file based on the content shown earlier . Finally, we create a temporary view that will help us in the next steps. Creating a Delta Table Let's create the Delta Table from an SQL script. At first the creation is simple, but notice that we used different types of a table used in a relational database. For example, we use STRING instead of VARCHAR and so on. We are partitioning the table by the date_order field. This field was chosen as a partition because we believe there will be different dates. In this way, queries can use this field as a filter, aiming at better performance. And finally, we define the table as Delta Table from the USING DELTA snippet. String statement = "CREATE OR REPLACE TABLE orders (" + "id STRING, " + "date_order STRING," + "customer STRING," + "product STRING," + "unit INTEGER," + "price DOUBLE) " + "USING DELTA " + "PARTITIONED BY (date_order) "; sparkSession.sql(statement); Understanding the code above In the previous section we're creating a Delta table called orders and then we execute the creation. DML Operations Delta supports Delete, Update and Insert operations using Merge Using Merge together with Insert and Update In this step, we are going to execute a Merge that makes it possible to control the flow of inserting and updating data through a table, Dataframe or view. Merge works from row matches, which will be more understandable in the next section. String mergeStatement = "Merge into orders " + "using global_temp.order_view as orders_view " + "ON orders.id = orders_view.id " + "WHEN MATCHED THEN " + "UPDATE SET orders.product = orders_view.product," + "orders.price = orders_view.price " + "WHEN NOT MATCHED THEN INSERT * "; sparkSession.sql(mergeStatement); Understanding the code above In the snippet above we're executing the Merge operation from the view order_view created in the previous steps. In the same section we have a condition orders.id = orders_view.id that will help in the following matches. If the previous condition is true, that is, MATCHED is true. The data will be updated. Otherwise, NOT MATCHED. Data will be inserted. In the case above, the data will be inserted, because until then there was no data in the orders table. Run the command below to view the inserted data. sparkSession.sql("select * from orders").show(); Update the datasource/order.json file by changing the product, price field and run all snippets again. You will see that all records will be updated. Update operation It is possible to run Update without the need to use Merge, just run the command below: String updateStatement = "update orders " + "set product = 'Milk-Shake' " + "where id = 2"; sparkSession.sql(updateStatement); Delete operation String deleteStatement = "delete from pedidos where id = 2"; sparkSession.sql(deleteStatement); In addition to being able to execute the Delete command, it is possible to use this command with Merge. Understanding Delta Lake Transaction Log (DeltaLog) In addition to supporting ACID transactions, delta generates some JSON files that serve as a way to audit and maintain the history of each transaction, from DDL and DML commands This mechanism it is even possible to go back to a specific state of the table if necessary. For each executed transaction a JSON file is created inside the _delta_log folder. The initial file will always be 000000000.json, containing the transaction commits. In our scenario, this first file contains the commits for creating the orders table. For a better view, go to the local folder that was probably created in the root directory of your project called spark-warehouse. This folder was created by Hive to hold resources created from JSON files and parquets. Inside it will have a folder structure as shown below: Note that the files are created in ascending order from each executed transaction. Access each JSON file and you will see each transaction that was executed through the operation field, in addition to other information. 00000000000000000000.json "operation":"CREATE OR REPLACE TABLE" 00000000000000000001.json "operation":"MERGE" 00000000000000000002.json "operation":"UPDATE" 00000000000000000003.json "operation":"DELETE" Also note that the parquet files were generated partitioned into folders by the date_order field. Hope you enjoyed!

Since Java 8 released in 2014, dozens of new features have been added, including improvements in the JVM and functions to make the developer's life easier. Among these features are the Expression Lambda (EL) which was the starting point for the entry of Java into the world of functional programming, improvement in the data API and the no need to create implementations of existing Interfaces with the use of Default methods . And the other news is the Streams API, the focus of this post. The Stream API is a new approach to working with Collections making the code cleaner and smarter. The Stream API works with on-demand data processing and provides dozens of features to manipulate Collections, reducing code and simplifying development in a kind of pipeline that will be explained later. To understand better, let's create a simple code of a list of objects and we'll create some conditions in order to extract a new list with the desired values. In this example we will not use Streams API. Let's create a Class representing the City entity which will have the following attributes: name, state and population. And finally a method called listCities that loads a list of objects of type City. Filtering In the next code, we will iterate through a list, and within the iteration will check which cities have a population greater than 20 and then add them to a secondary list. Until then, there are technically no problems with the above code. But it could be cleaner. Now, applying Stream API, here's a new example. Notice the difference, we invoke the method listCities() and because it returns a Collection, it is possible to invoke the Stream method. From the call of the stream() method, the pipeline starts. More examples of Filter Example 1: Output: City: Hollywood /State: CA /Population: 30 City: Venice /State: CA /Population: 10 Example 2: Output: City: Venice /State: CA /Population: 10 How does a pipeline work? Following the previous example, the pipeline is a sequential process that differentiates between intermediate and final operations. In the example, the Stream is invoked from a data source (list of objects of type City) that works on demand, the filter method is an intermediate operation, that is, it processes the data until the collect method is invoked, resulting in a final operation. Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Head First Design Patterns: Building Extensible and Maintainable Object-Oriented Software is a book that through Java examples shows you the patterns that matter, when to use them and why, how to apply them to your own designs, and the object-oriented design principles on which they're based. Head First Java is a complete learning experience in Java and object-oriented programming. With this book, you'll learn the Java language with a unique method that goes beyond how-to manuals and helps you become a great programmer. Through puzzles, mysteries, and soul-searching interviews with famous Java objects, you'll quickly get up to speed on Java's fundamentals and advanced topics including lambdas, streams, generics, threading, networking, and the dreaded desktop GUI. Well that’s it, I hope you enjoyed it!

Pitest is a tool that provides state-of-the-art mutation testing. It was written to run on Java applications or on applications that run on top of a JVM. How it works? Pitest works by generating mutant codes that will make changes to the bytecodes in the source code, changing logic, removing and even changing method return values. Thus, it's possible to evaluate the code in a more complete way, looking for failures and making the code reliable. Basic example of mutation See the code below: if(name == ""){ return "Empty name"; }else{ return name; } Running Pitest on the code above, it's able to modify the bytecodes by changing the code above to: if(name != ""){ return name; }else{ return "Empty name"; } Pitest enables you identifying possible points of failure and guide you to create tests based on these points. Hands on time Maven junit junit 4.12 test org.pitest pitest 1.6.2 Let's create a simple class representing a bank account called BankAccount and to make it simple we'll implement the logic inside the entity class itself. In this class, we have a method called userValid that checks if the user is valid. package com.bank.entity; public class BankAccount { public String bankName; public String user; public Double balance; public Boolean userValid(BankAccount bankAccount){ if(bankAccount.user != ""){ return true; }else{ return false; } } } Without creating any test classes, run this command via Maven: mvn org.pitest:pitest-maven:mutationCoverage See that a report was generated in the console with the mutations created based on the conditions that must be tested. Looking at the console above, we have 3 mutations or point of failures, which means we need to eliminate these mutations from our code. Pitest provides another way to validate these mutations through a report generated by an HTML page. You can access these generated reports through /target/pit-reports/ folder In the image below we can see a report that summarizes the coverage percentage. Note that we have 0% of coverage. In the BankAccount.java.html file shows details about uncovered code parts. Note that in the image above there is a list of Active mutators that are mutations based on the code we created. Removing these mutations Based on the reports, let's create some tests to reach out 100% of coverage. First of all, we need to create a class test. For this example, we're going to create a class test called BankAccountTest.java. To do this, there's an attention point. You'll have to create this class with the same package's name used for the BankAccount.java. Thus, create inside src/test/java a package called com.bank.entity and finally BankAccountTest.java. package com.bank.entity; import org.junit.Assert; import org.junit.Test; public class BankAccountTest { @Test public void userValid_NegateConditionalsMutator_Test(){ BankAccount bankAccount = new BankAccount( "Chase","Jonas", 2000.00); Boolean actual = bankAccount.userValid(bankAccount); Boolean expected = true; Assert.assertEquals(expected, actual); } @Test public void userValid_BooleanFalseReturnValsMutator_Test(){ BankAccount bankAccount = new BankAccount( "Chase","", 2000.00); Boolean actual = bankAccount.userValid(bankAccount); Boolean expected = false; Assert.assertEquals(expected, actual); } } We have created only 2 test method for our coverage: 1.userValid_NegateConditionalsMutator_Test(): Covers filled-in usernames. The constructor passed the "Chase" argument representing the username so that the method's condition is validated. And finally validates the condition of the method to be returned as TRUE. To this case, we already eliminated two mutations according to statistic report. BooleanTrueReturnValsMutator NegateConditionalsMutator 2. userValid_BooleanFalseReturnValsMutator_Test(): Validates the possibility of the method returning the Boolean value FALSE. Rerun the Maven command below: mvn org.pitest:pitest-maven:mutationCoverage Application console Note that for every mutation generated, 1 was eliminated. Covering 100% coverage in tests. References: https://pitest.org Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Unit Testing Principles, Practices, and Patterns: Effective Testing Styles, Patterns, and Reliable Automation for Unit Testing, Mocking, and Integration Testing with Examples in C# is a book that covers Unit Testing Principles, Patterns and Practices teaches you to design and write tests that target key areas of your code including the domain model. In this clearly written guide, you learn to develop professional-quality tests and test suites and integrate testing throughout the application life cycle. Mastering Unit Testing Using Mockito and JUnit is a book that covers JUnit practices using one of the most famous testing libraries called Mockito. This book teaches how to create and maintain automated unit tests using advanced features of JUnit with the Mockito framework, continuous integration practices (famous CI) using market tools like Jenkins along with one of the largest dependency managers in Java projects, Maven. For you who are starting in this world, it is an excellent choice. Well that’s it, I hope you enjoyed it!

Before understanding about ActiveMQ, we have to think about common problems in applications that need to scale and better integrate their services. Today the information transmitted flow is infinitely greater than 10 years ago and it is almost impossible to measure capacity in terms of scalability that an application can support. Use case To understand better, let's imagine that you were hired to design an architecture for an e-commerce that will sell tickets for NFL games. As always you have little time to think about an architecture. The first idea is simple and quick, the result is this drawing below. Thinking about the number of accesses and requests per second, do you think it is a resilient architecture? Does the database scale? Does the bank support multi-access? And if the bank for some reason falls down, will the purchase be lost? We can improve this architecture a little more, making it a little more professional and resilient. Let's go. Let's understand this last drawing. Now, when placing a purchase order, the orders are sent to a message server (Broker). The idea of ​​the Broker is basically a service capable of allocating messages. These are usually texts or text in Json format. In this drawing we can say that the Queue allocates customer data, number of tickets, values, etc. And finally, there is an application that does all the management of orders/purchases. This application reads/removes messages from the Broker and can perform validations before writing to the database. Now, let's assume that one of the requirements is for each sale, the customer must receive an invoice for the ticket. As the new architecture is well decoupled, it's easier to "plug in" a new application to do this job. Then you thought of a new design, follow: Now the application that manages the purchases, in addition to recording the sale after retrieving the messages in Queue 1, it also sends a message to Queue 2, where it will allocate the customers' invoices. And a new application that manages invoices retrieves these messages and records them on a specific database for the financial area. But what are the benefits of this new architecture? The architecture is more resilient, asynchronous and fault-tolerant. In case of, one of the applications fails for some reason, the message returns to the queue until the applications are reestablished. And finally, it facilitates the integration of new applications. What about ActiveMQ? What does he have to do with it? ActiveMQ is the service that provides the messaging server. In the design, it would be the message servers (Brokers). To understand even better, let's create a practical example of how to configure and use ActiveMQ with Spring Boot and JMS. Creating the Project To create this Spring Boot project we're going to use Spring Initializr to generate our project faster. Therefore, access the following https://start.spring.io to create it and choose the dependencies. Fill in the fields and select the 2 dependencies (ActiveMQ 5 and Spring Web) as shown in the image. Generate the file and import it into your project. Pom file Following the pom.xml that was created by Spring Initializr. 4.0.0 org.springframework.boot spring-boot-starter-parent 2.4.2 com.spring.active.mq spring-boot-active-mq 0.0.1-SNAPSHOT spring-boot-active-mq Demo project for Spring Boot 1.8 org.springframework.boot spring-boot-starter-activemq org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test test org.springframework.boot spring-boot-maven-plugin Installing ActiveMQ Let's download ActiveMQ to make the process more transparent. But there is also the possibility of using the built-in version of Spring Boot, but this time we will present it in a more traditional way. For this example, we're going to use the classic version of ActiveMQ. Download ActiveMQ here: https://activemq.apache.org/components/classic/download/ Steps to install here: https://activemq.apache.org/getting-started After installation, start the server according to the documentation. application.properties file In the created Spring-Boot application, fill in the application.properties file spring.activemq.broker-url=tcp://127.0.0.1:61616 spring.activemq.user=admin spring.activemq.password=admin The first line sets the messages server URL The second and subsequent lines are the authentication data. Ticket class public class Ticket { private String name; private Double price; private int quantity; public Ticket(){} public Ticket(String name, Double price, int quantity){ this.name = name; this.price = price; this.quantity = quantity; } public String getName() { return name; } public void setName(String name) { this.name = name; } public Double getPrice() { return price; } public void setPrice(Double price) { this.price = price; } public int getQuantity() { return quantity; } public void setQuantity(int quantity) { this.quantity = quantity; } @Override public String toString() { return String.format("Compra de ingresso -> " + "Name=%s, Price=%s, Quantity=%s}", getName(), getPrice(), getQuantity()); } } In the SpringBootActiveMqApplication class previously created by the generator, make the following change. @SpringBootApplication @EnableJms public class SpringBootActiveMqApplication { public static void main(String[] args) { SpringApplication.run(SpringBootActiveMqApplication.class, args); } @Bean public JmsListenerContainerFactory defaultFactory( ConnectionFactory connectionFactory, DefaultJmsListenerContainerFactoryConfigurer configurer) { DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory(); configurer.configure(factory, connectionFactory); return factory; } @Bean public MessageConverter jacksonJmsMessageConverter() { MappingJackson2MessageConverter converter = new MappingJackson2MessageConverter(); converter.setTargetType(MessageType.TEXT); converter.setTypeIdPropertyName("_type"); return converter; } } The @EnableJms annotation is the mechanism responsible for enabling JMS. The defaultFactory method configures and registers the factory to connect the queues using JMS. And finally, the jacksonJmsMessageConverter method converts the messages passed from JSON to the type that will be passed in the JmsTemplate that we will see soon. All of these methods use the @Bean annotation. Methods annotated with @Bean are managed by the Spring container. TicketController class In the TicketController class, we create a method called buyTicket that will be responsible for sending messages to the queue called compra_queue (purchase_queue) through a POST request. In this method we're using a JmsTemplate type object that allows objects to be converted and sent to the queue using JMS. package com.spring.active.mq.springbootactivemq.Controller; import com.spring.active.mq.springbootactivemq.pojo.Ticket; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.http.MediaType; import org.springframework.jms.core.JmsTemplate; import org.springframework.web.bind.annotation.PostMapping; import org.springframework.web.bind.annotation.RequestBody; import org.springframework.web.bind.annotation.RestController; @RestController public class TicketController { @Autowired private JmsTemplate jmsTemplate; @PostMapping(value = "/buy", consumes = MediaType.APPLICATION_JSON_VALUE) public void buyTicket(@RequestBody Ticket ticket){ jmsTemplate.convertAndSend("compra_queue", new Ticket(ticket.getName(), ticket.getPrice(), ticket.getQuantity())); } } EventListener class The EventListener class is a sort of "listener". The @JmsListener annotation defines this listener characteristic. In this same annotation, it is possible to configure the queue's name that will be "listened to" by the method. In short, all messages sent to the queue compra_queue (purchase_queue) will be received by this method. package com.spring.active.mq.springbootactivemq.listener; import com.spring.active.mq.springbootactivemq.pojo.Ticket; import org.springframework.jms.annotation.JmsListener; import org.springframework.stereotype.Component; @Component public class EventListener { @JmsListener(destination = "compra_queue", containerFactory = "defaultFactory") public void receiveMessage(Ticket ticket) { System.out.println("Mensagem da fila:" + ticket); } } Accessing Broker service - ActiveMQ After starting the service according to the documentation, access the service console through a browser http://127.0.0.1:8161/ Creating queue To create the queue, click on the Queues option on the top red bar, as shown in the image below. In the Queue Name field, type the queue's name as shown in the image above and click on the Create button. It's done, queue was created! Starting application Via terminal, access your project directory and run the Maven command below or launch via IDE mvn spring-boot:run Sending messages We will use Postman to send messages, if you don't have Postman installed, download it by accessing this link https://www.postman.com/downloads/ After installation, access Postman and fill in the fields as shown in the image below. Json content {"name":"Joao","price":2.0,"quantity":4} By clicking on the Send button, access the application's console and it will be possible to view the message sent and transmitted in the queue. Access the ActiveMQ console again and you will be able to see the message log that was sent to the queue. The Number of Consumers column is the number of consumers in the queue, which in this case is just 1. The Messages Enqueued column shows the number of messages that were sent and, finally, the Messages Dequeued column is the number of messages that were removed from the queue . Here I have a SpringBoot project with ActiveMQ repository: https://github.com/jpjavagit/jms-active-mq. It's worth checking out! Books to study and read If you want to learn more about and reach a high level of knowledge, I strongly recommend reading the following book(s): Spring Microservices in Action is a book that covers the principles of microservices using Spring, Spring Boot applications using Spring Cloud, resiliency, how to deploy and real-life examples of good development practices Spring MVC Beginner's Guide: Beginner's Guide is a book covering fundamental Spring concepts such as architecture, request flows, Bean validation, how to handle exception flows, using REST and Ajax, Testing and much more. This book is an excellent choice for anyone wanting to learn more about the fundamentals of Spring. Spring is a Java Framework containing different projects, Spring MVC being one of them. By acquiring a good Spring MVC foundation, you will be able to tackle challenges using any Spring Framework project. Learn Microservices with Spring Boot: A Practical Approach to RESTful Services using RabbitMQ, Eureka, Ribbon, Zuul and Cucumber is a book covers the main features of the Spring ecosystem using Spring Boot, such as creating microservices, event-based architecture, using RabbitMQ as a messaging feature, creating RESTful services and much more. This book is an excellent choice for anyone who wants to learn more about how Spring Boot and it's features. Well that’s it, I hope you enjoyed it!

There are different ways to create resources in AWS, you can create a Bucket S3, SQS, RDS and among many other resources manually. But to deal with infrastructure and its management, creating resources manually becomes unsustainable. Another way is using IaC tools - Infrastructure as code that allows you to create, manage and provision resources in the cloud with less effort. At AWS we can use CloudFormation to help us create the resources you want to use. How it works? Starting from a template in JSON or YAML format and then uploading this file to CloudFormation on AWS. Very simple. To better understand this process, let's create an S3 Bucket and an SQS queue through CloudFormation, following what was described earlier, using a template. There are two ways to create a template, you can use a JSON or YAML file. In this example we will use a template in YAML format. Creating S3 Bucket template Resources: S3Bucket: Type: 'AWS::S3::Bucket' DeletionPolicy: Retain Properties: BucketName: blog.data AccessControl: Private BucketEncryption: ServerSideEncryptionConfiguration: - ServerSideEncryptionByDefault: SSEAlgorithm: "AES256" For the template above, we used some essential parameters for creating the Bucket, the complete list can be consulted in the AWS documentation. Next, let's briefly understand what each parameter means: S3Bucket: is an identifier given to the resource, always create an identifier that makes sense to its context Type: resource type DeletionPolicy: There are three options: Delete: If the CloudFormation stack is deleted, all related resources will be deleted. Be very careful and understand the risks before using this option. Retain: Using this option, you guarantee that when deleting a stack, the related resources will be kept. Snapshot: Option used for resources that support snapshots, for example: AWS::EC2::Volume AWS::ElastiCache::CacheCluster AWS::ElastiCache::ReplicationGroup AWS::Neptune::DBCluster AWS::RDS::DBCluster AWS::RDS::DBInstance AWS::Redshift::Cluster In the Properties, we define the characteristics of the Bucket: BucketName: Bucket name. Remembering that the bucket name must be unique and must follow some name standards according to the documentation AccessControl: It's the access control to the Bucket, there are different access options, as follows: Private PublicRead PublicReadWrite AuthenticatedRead LogDeliveryWrite BucketOwnerRead BucketOwnerFullControl AwsExecRead BucketEncryption: These are the encryption settings of Bucket objects, in this case we use the AES256 algorithm. Uploading and creating the resource 1. In the AWS console, go to CloudFormation 2. Click the Create Stack button 3. Select as prerequisite Template is ready 4. In the Specify template section, select Upload a template file, select the created file by clicking on Choose file and finally click on the Next button. A new page will open for filling in the name of the stack. 5. Click Next and do the same for the next pages. 6. Finally, the resource will be created. This may take a few minutes depending on the feature. Notice that two buckets were created: blog.data: Created via CloudFormation cf-templates-1nwl4b3ve439n-us-east-1: Bucket created automatically when uploading the file at the beginning of the process. Creating SQS template Resources: SQS: Type: 'AWS::SQS::Queue' Properties: QueueName: sqs-blog.fifo ContentBasedDeduplication: true DelaySeconds: 120 FifoQueue: true MessageRetentionPeriod: 3600 Understanding the template: SQS: resource identifier Type: resource type QueueName: SQS queue name. An important detail is the .fifo suffix, necessary if the queue is of the Fifo type. ContentBasedDeduplication: Ensures non-duplication of messages, works only for Fifo-type queues. DelaySeconds: Delay time for each message (in seconds). FifoQueue: How the queue manages the arrival and departure of messages (First-in - First-out). MessageRetentionPeriod: period time messages that will be held in the queue (in seconds) SQS queue created Conclusion CloudFormation is an AWS exclusive tool for resource creation, i.e. if your architecture is built or maintained based on the AWS cloud, CloudFormation is a great choice. If you need to maintain flexibility between clouds, such as the ability to use Google Cloud, Terraform may be a better option as an IaC tool. Well that’s it, I hope you enjoyed it!