{"version":"https:\/\/jsonfeed.org\/version\/1","title":"manuelmazzuola.dev","home_page_url":"https:\/\/manuelmazzuola.dev\/","feed_url":"https:\/\/manuelmazzuola.dev\/feed.json","description":"Manuel Mazzuola's blog","icon":"https:\/\/manuelmazzuola.dev\/apple-touch-icon.png","favicon":"https:\/\/manuelmazzuola.dev\/favicon.ico","expired":false,"author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null},"items":[{"id":"https:\/\/manuelmazzuola.dev\/2023\/05\/15\/how-to-pass-cka-ckad","title":"How to pass the CKA and CKAD","summary":"How to pass the CKA (Certified Kubernetes Administrator) and CKAD (Certified Kubernetes Application Developer)","content_text":"The CKA* certifications exams are easy to ace if you use Kubernetes daily.It\u2019s quite important to get familiar with the K8s online documentation since it is the only documentation you can view.To make sure you are comfortable with https:\/\/kubernetes.io\/docs\/ search for GitHub repositories that contain CKA* exercises andattempt to resolve them with the help of online documentation only.That is all I\u2019ve done to achieve the CKA and the CKAD certifications.","content_html":"<p>The CKA* certifications exams are easy to ace if you use Kubernetes daily.<br \/>It\u2019s quite important to get familiar with the K8s online documentation since it is the only documentation you can view.To make sure you are comfortable with <a href=\"https:\/\/kubernetes.io\/docs\/\">https:\/\/kubernetes.io\/docs\/<\/a> search for GitHub repositories that contain CKA* exercises andattempt to resolve them with the help of online documentation only.<\/p><p>That is all I\u2019ve done to achieve the CKA and the CKAD certifications.<\/p>","url":"https:\/\/manuelmazzuola.dev\/2023\/05\/15\/how-to-pass-cka-ckad","tags":["kubernetes","certification","cka","ckad"],"date_published":"2023-05-15T00:00:00+00:00","date_modified":"2023-05-15T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2023\/04\/06\/i-got-ckad","title":"I got the CKAD (Certified Kubernetes Application Developer)","summary":"I got the CKAD (Certified Kubernetes Application Developer)","content_text":"https:\/\/www.credly.com\/badges\/0fc82058-e871-4bdb-8404-bdf790a243fc","content_html":"<p><a href=\"https:\/\/www.credly.com\/badges\/0fc82058-e871-4bdb-8404-bdf790a243fc\">https:\/\/www.credly.com\/badges\/0fc82058-e871-4bdb-8404-bdf790a243fc<\/a><\/p>","url":"https:\/\/manuelmazzuola.dev\/2023\/04\/06\/i-got-ckad","tags":["ckad","certification","kubernetes"],"date_published":"2023-04-06T00:00:00+00:00","date_modified":"2023-04-06T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2022\/10\/11\/i-got-cka","title":"I got the CKA (Certified Kubernetes Administrator)","summary":"I got the CKA (Certified Kubernetes Administrator)","content_text":"https:\/\/www.credly.com\/badges\/16ffab6c-d4f0-4da6-a36a-531228fe2046","content_html":"<p><a href=\"https:\/\/www.credly.com\/badges\/16ffab6c-d4f0-4da6-a36a-531228fe2046\">https:\/\/www.credly.com\/badges\/16ffab6c-d4f0-4da6-a36a-531228fe2046<\/a><\/p>","url":"https:\/\/manuelmazzuola.dev\/2022\/10\/11\/i-got-cka","tags":["cka","certification","kubernetes"],"date_published":"2022-10-11T00:00:00+00:00","date_modified":"2022-10-11T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2022\/09\/15\/reading-list","title":"My personal del.icio.us","summary":"How to setup a simple but effective reading list","content_text":"I have recently read an article on HackerNews I tracked everything I read on the Internet for a year where the author explains how he created a static site generatorto display the list of articles he read.Often I find myself looking for an article read some time ago, struggling to seek it in the browser history.So I decided to fork the project, do some customization and publish it to have a list of articles I can search easily.Firstly I modified the GitHub workflows to make go point to the correct repository and the author email of the user used to commit new articles.Then I customized the title of the static site, some hrefs and the CSS to make it look like my blog.I added GA and, finally, I created a simple script that I execute in the terminal when I have to add a new item to the list:#!\/bin\/shURL=\"$1\"title=\"$2\"description=\"$3\"image=\"$4\"payload=\"{\\\"event_type\\\":\\\"rl-append\\\",\\\"client_payload\\\":{\\\"URL\\\":\\\"$URL\\\",\\\"Title\\\":\\\"$title\\\",\\\"Description\\\":\\\"$description\\\",\\\"Image\\\":\\\"$image\\\"}}\"curl \\  -X POST \\  -H \"Accept: application\/vnd.github+json\" \\  -H \"Authorization: Bearer $GH_TOKEN\" \\  https:\/\/api.github.com\/repos\/manuelmazzuola\/readingList\/dispatches \\  -d \"$payload\"echo 'OK'Go check it out: https:\/\/manuelmazzuola.dev\/readingList","content_html":"<p>I have recently read an article on HackerNews <a href=\"https:\/\/www.tdpain.net\/blog\/a-year-of-reading\">I tracked everything I read on the Internet for a year<\/a> where the author explains how he created a static site generatorto display the list of articles he read.<\/p><p>Often I find myself looking for an article read some time ago, struggling to seek it in the browser history.<br \/>So I decided to <a href=\"https:\/\/github.com\/manuelmazzuola\/readingList\">fork the project<\/a>, do some customization and publish it to have a list of articles I can search easily.<\/p><p>Firstly I modified the <a href=\"https:\/\/github.com\/manuelmazzuola\/readingList\/actions\">GitHub workflows<\/a> to make <code class=\"language-plaintext highlighter-rouge\">go<\/code> point to the correct repository and the author email of the user used to commit new articles.<\/p><p>Then I customized the title of the static site, some <code class=\"language-plaintext highlighter-rouge\">href<\/code>s and the CSS to make it look like my blog.<br \/>I added <a href=\"https:\/\/github.com\/manuelmazzuola\/readingList\/commit\/b366cb28695f80390a44fab45ef64d4d2026ad77\">GA<\/a> and, finally, I created a simple script that I execute in the terminal when I have to add a new item to the list:<\/p><div class=\"language-bash highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c\">#!\/bin\/sh<\/span><span class=\"nv\">URL<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$1<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">title<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$2<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">description<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$3<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">image<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">$4<\/span><span class=\"s2\">\"<\/span><span class=\"nv\">payload<\/span><span class=\"o\">=<\/span><span class=\"s2\">\"{<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">event_type<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">rl-append<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">client_payload<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:{<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">URL<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\\"<\/span><span class=\"nv\">$URL<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">Title<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\\"<\/span><span class=\"nv\">$title<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">Description<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\\"<\/span><span class=\"nv\">$description<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">,<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">Image<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">:<\/span><span class=\"se\">\\\"<\/span><span class=\"nv\">$image<\/span><span class=\"se\">\\\"<\/span><span class=\"s2\">}}\"<\/span>curl <span class=\"se\">\\<\/span>  <span class=\"nt\">-X<\/span> POST <span class=\"se\">\\<\/span>  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Accept: application\/vnd.github+json\"<\/span> <span class=\"se\">\\<\/span>  <span class=\"nt\">-H<\/span> <span class=\"s2\">\"Authorization: Bearer <\/span><span class=\"nv\">$GH_TOKEN<\/span><span class=\"s2\">\"<\/span> <span class=\"se\">\\<\/span>  https:\/\/api.github.com\/repos\/manuelmazzuola\/readingList\/dispatches <span class=\"se\">\\<\/span>  <span class=\"nt\">-d<\/span> <span class=\"s2\">\"<\/span><span class=\"nv\">$payload<\/span><span class=\"s2\">\"<\/span><span class=\"nb\">echo<\/span> <span class=\"s1\">'OK'<\/span><\/code><\/pre><\/div><\/div><p>Go check it out: <a href=\"https:\/\/manuelmazzuola.dev\/readingList\">https:\/\/manuelmazzuola.dev\/readingList<\/a><\/p>","url":"https:\/\/manuelmazzuola.dev\/2022\/09\/15\/reading-list","tags":["readinglist","nerd","delicious","go"],"date_published":"2022-09-15T00:00:00+00:00","date_modified":"2022-09-15T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2021\/04\/26\/spring-certification-study-notes","title":"My Spring Professional 2021 certification study notes","summary":"My Spring Professional 2021 certification study notes to keep track of everything I have learned","content_text":"The company I work for has a department called The School which allows employeesto sign up for internal courses and obtain certifications if necessary.The courses list is extensive, and this year I have chosen to takethe Spring Professional training course to consolidate my knowledge of the framework.The course allowed receiving a voucher to attend the VMware Spring Professional exam certification,so I decided to give it a try.I don\u2019t really believe in certifications, and I even tend toconsider them a contraindication.As expected the exam was full of nitty-gritty detail questions on Spring internalsand stuff like that, But, I must admit that I\u2019ve learned a lot while I was preparing for the exam.Now I am aware of how Spring and Spring Boot work under the hood, and I better understandSpring features that I don\u2019t use daily at work. I am satisfied with the choice.Study materialI spent 30 hours to prepare the exam, ~1 hour per weekday.The most useful resource has been the officialVMware Spring study guide.It contains a list of questions that lead you through the topics addressed by the exam.Below the answers I wrote while studying.Container, Dependency and IOCWhat is dependency injection and what are the advantages of using it?The dependency injection is the linking action between dependents and dependencies.The dependent component is not allowed to inject dependencies by itself.The DI is delegated to an external authority called Spring IoC container to achieve decoupling between components.This behaviour is commonly called Inversion of Control.The injection process happens at runtime, allowing flexibility because the application behaviourcan be modified by an external configuration.Other advantages are:  Loosely coupled components  High cohesion  Code is more readable  Code is cleaner  Easy testing by mocking dependenciesWhat is an interface and what are the advantages?A java interface specifies behaviour that the implementing class must implementand separates what the caller expects from the implementation.  Interfaces allow an object to be referenced by the methods they support without consideringthe implementation.  Interfaces allow multi-inheritance of types, which is the ability of a class to implementmore than one interface.  Interfaces can help the testing process, by creating implementations mocksWhat is an ApplicationContext?An application context is an instance of any class that implements theApplicationContextinterface.  It provides configuration for the Spring application.  It manages all the beans, created and initialized by the Spring IoC Container.How are you going to create a new instance of an ApplicationContext?To create a new instance of an application context you have to instantiatea class that implements the ApplicationContext interface, for examplethe AnnotationConfigApplicationContext class.Spring boot expose a convenient method SpringApplication.runto bootstrap the application by creating an ApplicationContext instance.Can you describe the lifecycle of a Spring Bean in an ApplicationContext?  Constructor is called          If the bean is created using the constructor DI the 1st and 2nd steps are a single step        Dependencies are injected  @PostConstruct (jsr-250) annoted method is called, it must be void with no arguments  Call method setBeanName(name) declared in the BeanNameAware interface  Call method postProcessBeforeInitialization(bean, beanName) declared in the BeanPostProcess interface  Call method afterPropertiesSet declared in the InitializingBean interface  Call the init method defined in @Bean.initMethod  Call method postProcessAfterInitialization(bean, beanName) declared in the BeanPostProcess interface  Call method postProcessBeforeDestruction(bean, beanName) declared DestructionAwareBeanPostProcessors interface  @PreDestroy (jsr-250) annoted method is called  Call method destroy() declared in the DisposableBean interface  Call destroy method defined in @Bean.destroyMethodHow are you going to create an ApplicationContext in an integration test?  Annotate the test class with @RunWith(SpringJUnit4ClassRunner.class) if using JUnit4,or with @ExtendWith(SpringExtension.class) if using JUnit 5  Annotate the test class with @ContextConfiguration to tell the runner where the definitionsof the beans are          @ContextConfiguration accepts the classes and locations attributes      In Spring Boot just annotate the test class with @SpringBootTest.What is the preferred way to close an application context? Does Spring Boot do this for you?  Call the close() or the registerShutdownHook() of the ConfigurableApplicationContext implementation class.  Call the exit(...) method of the SpringApplication classSpring Boot registers a shutdown hook with the JVM to make sure the application exits appropriately.Furthermore, it exposes an actuator endpoint to shutdown the context on demand.Are beans lazily or eagerly instantiated by default? How do you alter this behaviour?Spring instantiate beans eagerly by default.To make a bean lazy:  Create it as Prototype  Enable lazy component scanning by using the @ComponentScan(lazyInit = true) annotation  Use the @Lazy annotation          The lazy annotation can be used on @Configuration classes, @Bean methods, @Component classes, and along withthe @Autowired annotation.      What is a property source? How would you use @PropertySource?A property source represents a source of name\/value property pairs.The @PropertySource annotation is used for adding a property source to Spring\u2019s Environment.It accepts the list of resource locations of the properties file to be loaded.@Configuration@PropertySource(\"classpath:\/it\/me\/my.properties\")public class AppConfig {  @Autowired  Environment env;  @Bean  public TestBean testBean() {    TestBean testBean = new TestBean();    testBean.setName(env.getProperty(\"testbean.name\"));    return testBean;  }}What is a BeanFactoryPostProcessor and what is it used for?When is it invoked?BeanFactoryPostProcessor is an interface and beans that implement it can alter othersbean definitions.It is invoked during startup of the Spring context, after all bean definitions are loaded.For example, it is used by the property source mechanism to resolve @Value(${placeholder}) placeholdersby injecting values read from the Spring environment and its set of property sources, declared using the@PropertySource annotation.The interface define the method postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory).public class CustomBeanFactory implements BeanFactoryPostProcessor {  @Override  public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {    for (String beanName : beanFactory.getBeanDefinitionNames()) {      BeanDefinition beanDefinition = beanFactory.getBeanDefinition(beanName);      \/\/ manipulate bean definition    }  }}What is a BeanPostProcessor and how is it different to a BeanFactoryPostProcessor? What do they do?When are they called?BeanPostProcess is an interface that adds some processing logic before and after executing the beaninitialization method after the Spring IoC Container instantiates the bean.The interface defines the methods postProcessBeforeInitialization(Object bean, String beanName)and postProcessAfterInitialization(Object bean, String beanName).What does component scanning do?The process of searching the classpath for classes that can contribute to the application context is called component scanning.A class must be annotated with a Stereotype annotation in order to be scanned and registered by component scanning.What is the behaviour of the annotation @Autowired with regards to field injection, constructor injection and method injection?@Autowired annotation populates the annotated attribute with the desired bean.The Bean Factory first looks for a bean of the annotated attribute type, then it tries using the nameif none or more than one are returned it throws a NoSuchBeanDefinitionException exception.If multiple matching it tries to match name and type.@Qualifier or @Primary annotations can be used to alter that behaviour.When @Autowired is used on a field, Reflection is used to populate the field.@Autowired cannot be used to autowire primitive types.How does the @Qualifier annotation complement the use of @Autowired?@Qualifier annotation is used to give a name to a bean.When Spring cannot decide what to autowire based on type, it looks for a bean named with@Qualifier annotation value.What is a proxy object and what are the two different types of proxies Spring can create?A proxy is a wrapper around the target object, it is used to alter the behaviour of the target object.Spring has two types of proxy, the prefered one is the JDK Dynamic Proxy, the other oneis the CGLIB.The former can only proxy by interface, the latter is used when the target class doesnot implement an interface, by subclassing it.The disadvantage of these proxies is that they can only intercept public methods, final classes andfinal methods are not supported too.What does the @Bean annotation do?@Bean annotation indicates that a method produces a bean to be managed by the Spring container.Typically it is used in @Configuration classes, in this case the bean methods may reference other@Bean methods in the same class by calling them directly, this mechanism is called inter-bean references.What is the default bean id if you only use @Bean? How can you override this?The default bean id is the name of the method.You can override it by using the name\/value attribute of the @Bean annotation.Why are you not allowed to annotate a final class with @ConfigurationSpring uses CGLIB to proxy the @Configuration annotated class, and CGLIB proxy does notsupport final classes because they cannot be subclassed.How do you configure profiles? What are possible use cases where they might be useful?A profile can be configured by using the @Profile annotation.The set of profile names for which the annotated component should be registered is indicatedby the value attribute of the annotation.The profile name may contain the NOT operator, !, the annotated component is registered if theprofile is not active.@Profile can be used:  on any class directly or indirectly annotated with @Component  on custom annotations  on any @Bean methodTo select the active profiles the property spring.profiles.active can be used.Profiles may also be activated in integration tests via the @ActiveProfile annotation.Profiles are used to modify the application behaviour, like loading an implementation insteadof another, by simple modifying the external configuration.Can you use @Bean together with @Profile?Yes.Can you use @Component together with @Profile?Sure.How many profiles can you have?Many as you want.How do you inject scalar\/literal values into Spring beans?To inject primitive types you must use the @Value annotation.It supports SpEL language #{systemProperties.myProp}.Alternatively, values may be injected using ${my.app.myProp} style property placeholders.The @Value processing is performed by a BeanPostProcessor, meaning that @Value annotationcan\u2019t be used within BeanPostProcessor or BeanFactoryPostProcessor types.What is Spring Expression Language (SpEL for short)?SpEL is a powerful expression language that supports manipulating an object at runtime.It supports the following functionality:  Literal expressions  Boolean and relational operators  Regular expressions  Class expressions  Accessing properties, arrays, lists, maps  Method invocation  Relational operators  Assignment  Calling constructors  Bean references  Array construction  Inline lists  Ternary operator  Variables  User-defined functions  Collection projection  Collection selection  Templated expressionsWhat is the Environment abstraction in Spring?The Spring Environment represents the environment in which the application is running.It unifies access to all types of property sources, such as property files, JVM system properties,system environment variables and servlet context parameters.Where can properties in the environment come from \u2013 there are many sources for properties \u2013 check the documentation if not sure. Spring Boot adds even more.The Spring boot PropertySource order is:  Devtools global settings properties on your home directory (~\/.spring-boot-devtools.properties when devtools is active).  @TestPropertySource annotations on your tests.  @SpringBootTest#properties annotation attribute on your tests.  Command line arguments.  Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)  ServletConfig init parameters.  ServletContext init parameters.  JNDI attributes from java:comp\/env.  Java System properties (System.getProperties()).  OS environment variables.  A RandomValuePropertySource that only has properties in random.*.  Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)  Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)  Application properties outside of your packaged jar (application.properties and YAML variants).  Application properties packaged inside your jar (application.properties and YAML variants).  @PropertySource annotations on your @Configuration classes.  Default properties (specified using SpringApplication.setDefaultProperties).What can you reference using SpEL?Beans.What is the difference between $ and # in @Value expressions?The former is the property placeholder style, used to inject primitives types,the latter indicates that it is a SpEL expression.Aspect-Oriented Programming (AOP)What is the concept of AOP? Which problem does it solve? What is a cross-cutting concern?AOP, Aspect-Oriented Programming, is a programming paradigm where is possible toalter the behaviour of the existing code without modification of the codeitself.AOP help to avoid tangling and scattering.\/\/ tangling and scattering example, logging is mixed with business-logic\/\/ and scattered across the codebase.public void aMethodService() {  logger.entering(\"MyService\",\"aMethodService\");  anotherService.myMethod();  logger.exiting(\"MyService\",\"aMethodService\");}public void anotherMethodService() {  logger.entering(\"MyService\",\"anotherMethodService\");  suchService.myMethod();  logger.exiting(\"MyService\",\"anotherMethodService\");}A cross-cutting concern is a functionality shared across multiple application modules.Such as logging, because is something that can happen across our application.What is a pointcut, a join point, an advice, an aspect, weaving?A pointcut is an expression that matches join points, for example the executionof a method with a certain name.A join point is a point during the execution of a program, in Spring AOP isalways a method execution.An advice is the code to run by an aspect at a particular join point.An aspect is the class that contains the cross-cutting concern logic.It is annotated with the @Aspect annotation.The weaving is the mechanism used by Spring AOP to proxy the target object to create an advised object.Spring AOP only supports runtime weaving by using JDK Dynamic Proxy or CGLIB, the former is prefered, butyou can switch to the latter by using @EnableAspectJAutoProxy(proxyTargetClass=true).How does Spring solve (implement) a cross-cutting concern?Spring implement a cross-cutting concern by weaving the aspect to the target object, creating a proxy thatcan intercept the execution of the method as specified by the pointcuts.Which are the limitations of the two proxy-types?JDK Dynamic Proxy limitations:  Target class must implement an interface  *-private, protected and final methods are not supported  Methods not defined in the interface are not supported  Does not support self-referencing, if a matched join point method callsanother method of the same class matched by a join point, the advice is calledjust once.CGLIB limitations:  Target class can\u2019t be final, as CGLIB can\u2019t subclass it  Private and final methods are not supported  Protected methods are supported but not recommendedHow many advice types does Spring support? Can you name each one?In Spring AOP there are multiple advice types:  @Before advice, will execute before the join point, it can\u2019t prevent the executionof the method unless it throws an exception  @AfterReturning advice, will execute after the join point completes normally , meaning thatthe target method returns normally without throwing an exception.  @AfterThrowing advice, will execute after the join point execution that ended by throwingan exception  @After advice, will execute after the join point execution, no matter the how execution ended.  @Around advice, which is the most powerful advice, will intercept the target method and surroundthe join point. It can perform custom behaviour before and after target invocation, it can execute ornot the target method and stop the propagation of a possible exception raised by the target.If shown pointcut expressions, would you understand them?Possible pointcut expressions are:  execution([Modifiers] [ReturnType] [FullClassName].[MethodName]([Arguments]) throws [ExceptionType])  whithin(), limits maching to join point of certain types, @Pointcut(\"within(it.manuel..*)\")  target() and this(), the former is used to match a class that implements the specified interface, andthe latter is used to match a specific class type.  @target limits matching to join point where the class of the target method has a specific annotation,@Pointcut(\"@target(org.springframework.stereotype.Service)\")  @args limits matching to join point where the argument of the target method has a specific annotation,@Pointcut(\"@args(it.manuel.annotations.MyArgAnnotation)\")  @annotation limits matching to join point where the target method has a specific annotation,@Pointcut(\"@annotation(org.springframework.beans.factory.annotation.Qualifier)\")The pointcut expression supports wildcards such as + and *, and logical operators like &amp;&amp;, || and !.The + wildcard indicates that the method can also be matched in subclasses it.manuel.+.helloWorld.What is the JoinPoint argument used for?The JoinPoint type is the argument type used for all the advice annotated methods, except @Around,which argument type is ProceedingJoinPoint.What is a ProceedingJoinPoint? Which advice type is it used with?The ProceedingJoinPoint is the argument type for @Around advice.It exposes the .proceed method used to call the method matched by the pointcut.Data Management: JDBC, TransactionsWhat is the difference between checked and unchecked exceptions?A checked exception is a compile-time exception that must be managed.An unchecked exception is a runtime exception, it\u2019s not required to manage it.The latter is used by the data access mechanism to report errors, by throwingexceptions that extend the DataAccessException class.Rollback is automatic only on unchecked exceptions.How do you configure a DataSource in Spring?To access a database using JDBC, the following are needed:  A DB driver to dialogue with the database  The DB connection url  The DB credentialsA DataSource is created using the previous information, and each database librarymust implement it.In Spring to programmatically create a DataSource you have to define a DataSource beanand return an instance of the implementation provided by the DB vendor you are using.In Spring Boot, is possible to define a DataSource by setting propertiesapp.datasource.url=jdbc:h2:mem:helloapp.datasource.username=helloapp.datasource.password=worldSpring Boot also provide a class builder named DataSourceBuilder, it auto-detect the driverbased on what is available on the classpath or based on the JDBC URL.What is the Template design pattern and what is the JDBC template?The template design pattern is one of the most popular, itenables the developer to implement a complex functionality by encapsulatingthe logic in a single method, the so-called template method.The effective implementation is defined later or delegated to subclasses.public abstract class MyTemplateBuilder {  public final JdbcTemplate buildHouse() {    setupFoundation();    buildWalls();    buildRoof();    paintWalls();  }  public abstract setupFoundation();  public abstract buildWalls();  public abstract buildRoof();  public abstract paintWalls();}The JDBC Template is the central class to manage DB operations.Under the hood it manages all the repetitive actions to execute a query,such as opening and closing the connection, handling exceptions and transactions,leaving application code to only provide SQL and extract results.What is a callback? What are the JdbcTemplate callback interfaces that can be used with queries? What is each used for? (You would not have to remember the interface names in the exam, but you should know what they do if you see them in a code sample).A callback is a method called by another method when this one finished the execution,usually, the callback method argument is the return value of the former.The JdbcTemplate query method accepts 3 callback interface types:  ResultSetExtractor&lt;T&gt; interface expose the &lt;T&gt; T extractData(ResultSet set) method  RowMapper&lt;T&gt;expose the &lt;T&gt; List&lt;T&gt; mapRow(ResultSet set, int rowNum) method  RowCallbackHandler expose the void processRow(ResultSet set)  methodResultSetExtractor is supposed to extract the whole ResultSet, it may contain multiple rows,while RowMapper is fed with a row at a time.Both are typically stateless.RowCallbackHandler is fed with a row at time, and it is tipically stateful.Can you execute a plain SQL statement with the JDBC template?JdbcTemplate execute plain SQL queries.When does the JDBC template acquire (and release) a connection, for every method called or once per template? Why?JDBC template acquire a connection for every method called.How does the JdbcTemplate support queries? How does it return objects and lists\/maps of objects?JdbcTemplate support queries for any type.It exposes many overloaded methods:  &lt;T&gt; List&lt;T&gt; query(...)  Map&lt;String, Object&gt; queryForMap(...)  &lt;T&gt; T queryForObject(...)  &lt;T&gt; List&lt;T&gt; queryForList(...)What is a transaction? What is the difference between a local and a global transaction?In a DB context, a transaction is an atomic set of operations, when all operations have completed successfullythe changes are persisted, else if one operation fails, the transaction is rolled back leaving the database pristine.A local transaction is a transaction that involves only local operations,whereas a global transaction involves external operations, such as an external database operationor a message on an external queue.What does declarative transaction management mean?Declarative transaction management is a non-invasive approach for managing transactions providedby Spring AOP.It means that you can enable and tag a class or a method transactional by using annotations.To use transactions is sufficient to add @EnableTransactionManagement to your configurationand annotate your classes or methods with the @Transactional annotation.What is the default rollback policy? How can you override it?By default, a transaction is rolled back when an unchecked exception occurs withinthe transactional code.It is possible to override that policy by configuring the @Transactional annotation parameters:  noRollbackFor to define which exception types must not cause a transaction rollback  noRollbackForClassName to define which exception types must not cause a transaction rollback  rollbackFor to define which exception types must cause a transaction rollback  rollbackForClassName to define which exception types must cause a transaction rollbackWhat is the default rollback policy in a JUnit test, when you use the @RunWith(SpringJUnit4ClassRunner.class) in JUnit 4 or ExtendWith(SpringExtension. class) in JUnit 5, and annotate your @Test annotated method with @Transactional?By default tests do not commit, they always require a rollback.That behaviour can be altered by using the @Rollback(false) annotation orthe @Commit annotation.Are you able to participate in a given transaction in Spring while working with JPA?Yes, by using the TransactionAwareDataSourceProxy class.Which PlatformTransactionManager(s) can you use with JPA?  JpaTransactionManager, also supports direct DataSource access within transaction  JtaTransactionManager, necessary for accessing multiple transaction resources within the same transaction  HibernateTransactionManager, refhttps:\/\/docs.spring.io\/spring-framework\/docs\/current\/reference\/html\/data-access.html#orm-jpa-txWhat do you have to configure to use JPA with Spring? How does Spring Boot make this easier?To enable JPA must be used the @EnableJpaRepositories annotation, it will scan the packageof the annotated configuration class for Spring Data repositories by default.To change the package to scan the basePackages or basePackageClasses attributes can be used.To configure JPA an EntityManagerFactory instance must be provided, for example by using theLocalContainerEntityManagerFactoryBean class.The FactoryBean reads configuration settings from the META-INF\/persistence.xml config file.Spring Boot automagically setup JPA and transactions if the spring-boot-starter-data-jpa module is on theclasspath.Spring Data JPAWhat is a Spring Data Repository interface?How do you define a Spring Data Repository interface? Why is it an interface not a class?A Spring Data Repository interface typically extends one of the interfaces in the Repository&lt;T, ID extends Serializable&gt;hierarchy.Such interfaces expose a ready-to-use set of methods to manipulate entities.Beans of this type are called instant repositories, because Spring looks for these interfaces types and implementsthem at runtime to create beans.Alternatively can be used the @RepositoryDefinition annotation to create a repository, passing to thedomainClass attribute the type of entity and to the idClass attribute the id type.What is the naming convention for finder methods in a Spring Data Repository interface?The supported query keywords are  find\u2026By  read\u2026By  get\u2026By  query\u2026By  search\u2026By  stream\u2026ByTo specify the number of returned records:  \u2026First  \u2026TopTo return only unique results:  \u2026Distinct\u2026How are Spring Data repositories implemented by Spring at runtime?Spring uses a factory bean named RepositoryFactorySupport to create instancesof a given repository interface.It creates a JDK Dynamic proxy implementing the repository interface and apply an advice handling the control to theQueryExecutorMethodInterceptor class.What is @Query used for?The Query annotation is used to declare finder queries directly on repository methods.It accepts both JPQL (only JPA) and native SQL queries.The @Query annotation takes precedence over @NamedQuery annotation.To create update queries add the @Modifying annotation to the @Query annotated method.Spring MVC BasicsWhat is the @Controller annotation used for?The @Controller stereotype annotation is used to indicate that the annotated class provides handler methodsfor HTTP requests.Typically it is used in combination with handler methods annotated with @RequestMapping.How is an incoming request mapped to a controller and mapped to a method?The Spring Web MVC entry point is the DispatcherServlet class, it dispatches HTTP requeststo handlers that match the request pattern.What is the difference between @RequestMapping and @GetMapping?The @GetMapping is equal to @RequestMapping(method = RequestMethod.GET).What is @RequestParam used for?The @RequestParam annotation is used to indicate a query parameter in the request URL.To indicate an optional query parameter set to false the required attribute.A default value can be provided by using the defaultValue attribute.What are the differences between @RequestParam and @PathVariable?The @PathVariable annotation is used to indicate a path parameter in the request URL.What are the ready-to-use argument types you can use in a controller method?The signatures of the handler methods are very flexible, they can receive almost any type-specificto a web application.The following are some parameters types:  @RequestHeader annotated parameters to gather a request header  WebRequest, NativeWebRequest and ServletRequest provide access to headers, request and session attributes  ServletRespose to enrich the response  HttpSession to access the session  Principal to access the credentials  HttpMethod which represents the request method  Locale to get the current locale  BindingResult to access the validation reports  Error to access the validation errors  InputStream, Reader to access the raw body of the request  OutputStream, Writer to access the raw body of the response  @RequestParam annotated parameter to get the query parameters  @PathVariable annotated parameter to access the path variables  @MatrixVariable annotated parameter to access the matrix variables, ex. http:\/\/hello.world\/;g=1;u=3;=9  @CookieValue annotated parameter to access the cookies  Map, Model or ModelMap that contain the data that the view template will render  RedirectAttributes along with @ModelAttribute to specify data to be used by a view which was redirected to  @SessionAttribute or @RequestAttribute to access to request and session attributesWhat are some of the valid return types of a controller method?  HttpEntity, ResponseEntity to return a full response  HttpHeaders to return a response without a body  String to specify a logical view name  View  Map or  Model to specify attributes for the model  ModelAndView to specify model and view to use  void to return an empty response or select the default view  DeferredResult, CompletableFuture, etc. to produce an async response from any thread  Callable&lt;V&gt; to produce an async response in a Spring MVC threadSpring MVC RESTWhat does REST stand for?REpresentationalStateTransferWhat is a resource?A REST resource is something referenced by an URL.I can get the apple number 1 by using theurl \/api\/apple\/1.Is REST secure? What can you do to secure it?It is secure if used safely.For example by using the HTTPS protocol and require some credentials to accessa protected resource.Is REST scalable and\/or interoperable?Yes, it is scalable and interoperable.It does not mandate a specific technology either for the client or server.Which HTTP methods does REST use?REST use the following methods  GET, to get a resource, it is idempotent  POST, to create a resource, it is not idempotent  PUT, to update a resource, it is idempotent  PATCH, to partially update a resource, it is not idempotent  DELETE, to delete a resource, it is idempotent  OPTIONS, to list the supported methods of a resource, it is idempotent  HEAD, to only get the headers related to a resource, it is idempotentWhat is an HttpMessageConverter?The HttpMessageConverter is the interface used for converting from and to HTTP requests and responses.By default the following converters are enabled:  ByteArrayHttpMessageConverter, converts byte arrays  StringHttpMessageConverter, converts Strings  ResourceHttpMessageConverter, converts org.springframework.core.io.Resource for any type of octet stream  SourceHttpMessageConverter, converts javax.xml.transform.Source  FormHttpMessageConverter, converts form data to\/from a MultiValueMap&lt;String, String&gt;.  Jaxb2RootElementHttpMessageConverter, converts Java objects to\/from XML (added only if JAXB2 is present on the classpath)  MappingJackson2HttpMessageConverter, converts JSON (added only if Jackson 2 is present on the classpath)  MappingJacksonHttpMessageConverter, converts JSON (added only if Jackson is present on the classpath)  AtomFeedHttpMessageConverter, converts Atom feeds (added only if Rome is present on the classpath)  RssChannelHttpMessageConverter, converts RSS feeds (added only if Rome is present on the classpath)Is @Controller a stereotype? Is @RestController a stereotype?@Controller is a stereotype annotation.@RestController is a specialization of the @controller annotation, but it can\u2019t be considereda stereotype annotation.What is the difference between @Controller and @RestController?When do you need to use @ResponseBody?@RestController is meta-annotated with @ResponseBody annotation to indicate thatthe value returned by a handler method needs to be bind to the response body, thus serialized.What are the HTTP status return codes for a successful GET, POST, PUT or DELETE operation?  200 Ok for a GET  201 Created for a POST  200 Ok or 204 No Content for a PUT  200 Ok, 202 Accepted or 204 No Content for a DELETEWhen do you need to use @ResponseStatus?The @ResponseStatus annotation is used to specify the response status:  when a controller method completes successfully  along with @ExceptionHandler and @ControllerAdvice for a method that returns an exception  for a specific exception by directly annotating itWhere do you need to use @ResponseBody? What about @RequestBody?The @RequestBody annotation is used to indicate a method parameter should be bound to the bodyof the request.The @ResponseBody annotation indicates the return value should be bound to the web response body.What Spring Boot starter would you use for a Spring REST application?spring-boot-starter-webSecurityWhat are authentication and authorization? Which must come first?Authentication is the process used to check if the credentials provided by the userare valid.Authorization is the process used to check if the authenticated user has the permissionsto access the requested resource.Is security a cross-cutting concern? How is it implemented internally?Security is a cross-cutting concern, it is applied across multiple modules of ourapplication.Spring uses AOP to implement the security mechanism.What is the delegating filter proxy?The DelegatingFilterProxy acts as an interceptor for secured request, delegating the callsto the chained security filter beans, called SpringSecurityFilterChain.What is the security filter chain?The SecurityFilterChain is a list of filters, each one with a particular responsibility.The filters are added or removed from the chain depending on which services are required.The filters ordering is important as there are dependencies between them.It is used by the FilterChainProxy to match the url with the filters to apply.What is a security context?The security context is where are stored the details of the current security context, such asthe details of the Principal that is using the application.What does the ** pattern in an antMatcher or mvcMatcher do?Taking the following pattern example \/admin\/user\/**,they both matches any path that starts with \/admin\/user\/,farther the mvcMatcher also matches the \/admin\/user\/123.json path.Why is the usage of mvcMatcher recommended over antMatcher?mvcMatcher is considered more secure than antMatcher.That is because mvcMatcher use the same rules that Spring MVC uses for matching.Does Spring Security support password encoding?Yes, just define a bean of the encoder type you want to use and use it.Why do you need method security? What type of object is typically secured at the method level (think of its purpose not its Java type).To protect sensitive resources, for example the resource that represents the users.What do @PreAuthorize and @RolesAllowed do? What is the difference between them?The @PreAuthorize annotation is used to run a check before a method invocation to decide ifthe user has the permissions to call the method.I can decide to deny a method execution to users which names start with an m.The @RolesAllowed annotation is used to check if the Principal role is one of the rolesspecified by the annotation.The @Secured and @RolesAllowed annotations are equivalent, the latter is part of theJSR 250 spec.To enable the security annotations the @EnableGlobalMethodSecurity annotation must be used.It exposes some properties:  the prePostEnabled property enables @pre\/@post annotations  the securedEnabled property determines if the @Secured annotation should be enabled  the jsr250Enabled property allows us to use the @RoleAllowed annotationHow are these annotations implemented?Spring AOP.In which security annotation, are you allowed to use SpEL?Yes, all the @pre and @post annotations support SpEL:  @PreAuthorize  @PostAuthorize  @PreFilter  @PostFilterTestingWhat type of tests typically use Spring?Tipically, a Spring application has unit and integration tests.Unit tests are used to test the smallest testable parts of anapplication, isolated from any others units.The dependencies are replaced by stubs and mocks to reproduce the expected behaviour.By default Spring Boot uses Junit 5, but it supports Junit 4 too.Integration tests are used to test the multiple units interactions of anapplication.How can you create a shared application context in a JUnit integration test?The so-called TestContext encapsulates the context of each test, and provide context managementand caching support for the test instance.By default, the ApplicationContext is not accessible by the test instances.To expose the ApplicationContext to the test instances the test class must implementthe ApplicationContextAware interface.As an alternative to implementing the ApplicationContextAware interface, it is possible toinject the application context through the @Autowired annotation.Once a TestContext loads an ApplicationContext, that context is cached and reused for allsubsequent tests within the same test suite.When and where do you use @Transactional in testing?The @Transactional annotation can be used on the test class ordirectly on the method.By default a rollback occurs after completition of the test.To change that behaviour you can use the @Rollback(false) annotation orthe @Commit annotation.Also, the defaultRollback property of the @TransactionConfiguration annotationcan be used to change that behaviour.How are mock frameworks such as Mockito or EasyMock used?To run a test that uses Mockito you have to:  Declare the mock by using the @Mock annotation  Inject the mock in the tested unit by using the @InjectMocks annotation  Configure the mock by using the when statement  Run test  Assert resultsMockito provides a Junit 5 extension class MockitoExtension that automatically initialize mocks.In alternative, you can use the static method MockitoAnnotations.initMocks.Spring Boot simplifies the mocking process by providing the@MockBean annotation.It adds the mock to the Spring ApplicationContext.Can be used directly on the test class or @Configuration classes and fields.When used on a field of the test class, the mock is injected into the field.How is @ContextConfiguration used?The @ContextConfiguration annotation is used on a test class to specify the components classesto use to configure the ApplicationContext.A component class is:  A class annotated with @Configuration  A component (i.e., a class annotated with @Component, @Service, @Repository, etc.)  A JSR-330 compliant class that is annotated with javax.inject annotations  Any class that contains @Bean-methods  Any other class that is intended to be registered as a Spring component (i.e., a Spring bean in the ApplicationContext), potentially taking advantage of automatic autowiring of a single constructor without the use of Spring annotationsWhen @ContextConfiguration is used without properties it searches for a file namedtestClassName-context.xml in the same location of the test class.How does Spring Boot simplify writing tests?Spring Boot provides all the libraries needed to run a well-rounded test by justusing the spring-boot-starter-test dependency:  JUnit\u2009, the de-facto standard for unit testing Java applications  Spring Test &amp; Spring Boot Test, utilities and integration test support for Spring Boot applications  AssertJ\u2009, a fluent assertion library  Hamcrest\u2009, alibrary of matcher objects (also known as constraints or predicates)  Mockito\u2009, a Java mocking framework  JSONassert\u2009, an assertion library for JSON  JsonPath\u2009, \u2009XPath for JSONWhat does @SpringBootTest do? How does it interact with @SpringBootApplication and @SpringBootConfiguration?Spring Boot exposes the @SpringBootTest annotation to configure an ApplicationContextby using the SpringBootContextLoader loader class.It provides the following features:  automatically searches for any @SpringBootConfiguration-classes when no @ContextConfiguration annotation is defined  allow to customize the Environment properties to be defined using the properties attribute  allow application arguments to be defined using the args attribute  support for different webEnvironment modes, including the ability to start a fully running webserver listening on a defined or random port  register a TestRestTemplate beanSpring Boot BasicsWhat is Spring Boot?Spring Boot is a way to ease create production-ready, stand-alone Spring-based applications.It provides an opinionated starter configuration for code and annotation, to quickly start new Spring projects with no timewith minimal or zero configurations.What are the advantages of using Spring Boot?  Very easy to develop Spring based applications  It reduces development time and increase productivity  It avoids writing boilerplate code, annotations and xml configuration  Easy integration with Spring ecosystem, such as Spring JDBC, Spring ORM, Spring Data, Security\u2026  It reduces developer effort by providing opinionated defaults configuration  It provides embedded HTTP servers, Tomcat by default  It provides a CLI to develop and test Spring applicationsWhat things affect what Spring Boot sets up?Spring Boot attempts to automatically configure the Spring application based on whichlibraries are available in the classpath.The auto-configuration is enabled by adding the @EnableAutoConfiguration or @SpringBootApplication annotationsto one of the @Configuration classes.What is a Spring Boot starter? Why is it useful?A starter is a set of convenient dependency descriptors that can be included in a Spring application.For example, if you want to get started using Spring and JPA for database access, include thespring-boot-starter-data-jpa dependency in the project.All Spring starters follow the naming pattern spring-boot-starter-*.Can you control logging with Spring Boot? How?The default auto-configured logger in Spring Boot is logback. Appropriate logback routing is also included to ensure that dependant libraries that use others providers work correctly. Logging can be configured by using some well-known properties in the logging.* group:  logging.file.name or logging.file.path to indicate the file where to log  logging.logback.rollingpolicy.* to configure the file rotation policy  logging.level.&lt;logger-name&gt;=&lt;level&gt;, where level is one of TRACE, DEBUG, INFO,WARN, ERROR, FATAL or OFF, to configure the log level of a specific logger.  logging.group.&lt;group-name&gt;=&lt;loggers&gt; to group loggers together.Spring Boot provides two pre-defined logging groups, web and sql.Another option to configure logging is to use a dedicated configuration file:  logback-spring.xml  logback.xml  log4j-spring.xml or log4j.xml  logging.propertiesWhere does Spring Boot look for application.properties file by default?Ordered by precedence:  In the classpath root  In the \/config package classpath  In the current directory  In the \/config subdirectory in the current directory  In the immediate child directories of the \/config subdirectoriesHow do you define profile specific property files?Spring Boot attempt to load profile-specific files using the naming conventionapplication-&lt;profile&gt;.For example, if the dev profile is enabled, then both application.yml and application-dev.ymlwill be considered.Profile specific files always override the non-specific ones.If several profiles are specified, a last-wins strategy applies.If no profiles are activated, the properties from application-default are considered.How do you access the properties defined in the property files?You can access to properties values defined in property files by:  retrieving the current Environment  using the @Value annotation withing the ${&lt;property-name&gt;} placeholder  using the @ConfigurationProperties annotationWhat properties do you have to define in order to configure external MySQL?  spring.datasource.url=jdbc:mysql:\/\/mysql-hostname\/db-name  spring.datasource.username=hello  spring.datasource.password=worldHow do you configure default schema and initial data?Spring Boot loads SQL from the standard root classpath locations:  schema.sql  data.sqlIn addition it processes the files:  schema-&lt;platform&gt;.sql, where platform is the value of the spring.datasource.platform property  data-&lt;platform&gt;.sqlSpring Boot automatically creates the schema for embedded data sources.This behaviour can be customized by using the spring.datasource.initialization-mode property.What is a fat jar? How is it different from the original jar?A fat jar is a jar that contains all the application dependencies.What embedded containers does Spring Boot support?  Tomcat  Jetty  Undertow  Reactor NettySpring Boot Auto ConfigurationHow does Spring Boot know what to configure?Spring auto-configuration attempts to automatically configure the Spring applicationbased on the jar dependencies that are present in the classpath.What does @EnableAutoConfiguration do?It enables the Spring auto-configuration of the Spring Application Context.Auto-configuration classes are usually applied based on the classpath content andwhat beans are available.Auto-configuration classes are Spring @Configuration beans located by theSpringFactoriesLoadeer mechanism.Those beans are usually @Conditional beans,most often using @ConditionalOnClass and @ConditionalOnMissingBean annotations.What does @SpringBootApplication do?The @SpringBootApplication annotation is meta-annotated with  @SpringBootConfiguration, an alias for the @Configuration annotation  @EnableAutoConfiguration, to enable the Spring auto-configuration  @ComponentScan to scan the current packages and all its children for component beansDoes Spring Boot do component scanning? Where does it look by default?The @SpringBootApplication enable component scanning for the current packageand all its sub-packages.How are DataSource and JdbcTemplate auto-configured?Spring Boot automatically creates a JdbcTemplate.The DataSource is auto-configured only if in the classpathis present an in-memory database dependency.What is spring.factories file for?The spring.factories file is used by Spring Boot to locate the auto-configuration candidates andto register ApplicationListeners.A sample of the spring.factories file is:org.springframework.boot.autoconfigure.EnableAutoConfiguration=\\com.mycorp.libx.autoconfigure.LibXAutoConfiguration,\\com.mycorp.libx.autoconfigure.LibXWebAutoConfigurationHow do you customize Spring Boot auto configuration?The @EnableAutoConfiguration annotation supports the following properties:  exclude, to specify the list of classes to skip  excludeName, to specify the list of classes names to skipAlternatively:  Use the spring.autoconfigure.exclude property to excludesome auto-configuration classes.  Use the spring.factories file to specify the auto-configuration classes  Override the beans created by the auto-configuration classesWhat are the examples of @Conditional annotations? How are they used?Conditional annotations are used to skip beans definitions under certain conditions.For example the JdbcTemplateAutoConfiguration is annotated with the following annotations: @ConditionalOnClass(value={javax.sql.DataSource.class,org.springframework.jdbc.core.JdbcTemplate.class}) @ConditionalOnSingleCandidate(value=javax.sql.DataSource.class)Meaning that a JdbcTemplate is auto-configured only if:  A single datasource exists  The JdbcTemplate class is on the classpathSpring Boot ActuatorWhat value does Spring Boot Actuator provide?It provides production-ready features to help you monitor and manage the Spring Boot application.What are the two protocols you can use to access actuator endpoints?  JMX  HTTPWhat are the actuator endpoints that are provided out of the box?An actuator endpoint can be enabled and exposed, when both are true then theactuator endpoint is available.By default all endpoints except for shutdown are enabled.To enable\/disable an actuator use the property pattern management.endpoint.&lt;name&gt;.enabled.To disable all endpoints by default use the property management.endpoints.enabled-by-default=false.All jmx endpoints are exposed out-of-the-box.Instead only few http endpoints are exposed by default:  health endpoint  info endpointAdditional endpoints are enabled if the application is a web application:  heapdump, returns an hprof heap dump file  jolokia, when jolokia is in the classpath, it exposes jmx beans over http  logfile, if logging.file.name or logging.file.path properties have been set, returns the content of the logfile  prometheus, exposes metrics in a format that can be scraped by PrometheusTo change which endpoints are exposed, use the the following properties:  management.endpoints.jmx.exposure.exclude  management.endpoints.jmx.exposure.include, default value is *  management.endpoints.web.exposure.exclude  management.endpoints.web.exposure.include, default value is info, healthYou can customize the actuator web endpoints path by using the following properties  management.endpoints.web.base-path  management.endpoints.web.path-mapping.&lt;id&gt;What is info endpoint for? How do you supply data?The info endpoint exposes arbitrary data.Spring Boot by default exposes the following data:  any key from the Environment under the info key  git information if a git.properties file is available  build information if a META-INF\/build-info.properties file is availableYou can create a component class that implements the InfoContributor interfaceto add additional data to the info endpoint.How do you change logging level of a package using loggers endpoint?By doing a POST to the \/actuator\/loggers endpoint with the following body:{  \"configuredLevel\": \"DEBUG\"}Pass null to reset the logger level.How do you access an endpoint using a tag?What is metrics for?Spring Boot provides a metrics endpoint that can be used to gatherthe metrics collected by the application.Navigating to the \/actuator\/metrics display the list of available metrics.Provide the metric name to retrieve the data of a particular metric, e.g. \/actuator\/metrics\/jvm.memory.max.You can use tags to drill down a specific metric data, e.g. \/actuator\/metrics\/jvm.memory.max?tag=area:nonheap.How do you create a custom metric?Inject the MeterRegistry bean in your component and use it to register a custom metric.What is Health Indicator?It provides the health status of the current application.What is the Health Indicator status?The HTTP status code in the response reflects the overall health status.What are the Health Indicator statuses that are provided out of the box?By default it supports the following statuses:  UP, no mapping by default  UNKNOWN, no mapping by default  DOWN, mapped to the code 503  OUT_OF_SERVICE, mapped to the code 503Any unmapped health status map to 200.How do you change the Health Indicator status severity order?Using the management.endpoint.health.status.order propertySpring Boot TestingWhen do you want to use @SpringBootTest annotation?The @SpringBootTest annotation is an alternative to the spring-test @ContextConfigurationannotation when you need Spring Boot features.The annotation works by creating the ApplicationContext used by the tests through SpringApplication.By default it will not start a server.You can use the webEnvironment property to configure the test class web environment:  MOCK, the default value, loads a WebApplicationContext and provides amocked web environment. It can be used with the @AutoConfigureMockMvc for mock-based testing of your web application.  RANDOM_PORT, loads a WebServerApplicationContext and provides a real web environment.The embedded server port can be injected by using the @LocalServerPort annotation or by injecting the local.server.portproperty.  DEFINED_PORT, is like RANDOM_PORT but the listening port is gathered from the Environment.  NONE, Loads an ApplicationContext but does not provide any web environment.What does @SpringBootTest auto-configure?  It uses the SpringBootContextLoader to confiure the ApplicationContext  It registers a TestRestTemplate bean for use in web tests that are using a fully running webserverWhat dependencies does spring-boot-starter-test brings to the classpath?  JUnit\u2009, the de-facto standard for unit testing Java applications  Spring Test &amp; Spring Boot Test, utilities and integration test support for Spring Boot applications  AssertJ\u2009, a fluent assertion library  Hamcrest\u2009, a library of matcher objects  Mockito\u2009, a Java mocking framework  JSONassert\u2009, an assertion library for JSON  JsonPath\u2009,\u2009XPath for JSONHow do you perform integration testing with @SpringBootTest for a web application?  Use @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) to start the embedded server  Use @LocalServerPort annotation on a field to inject the listening port  Use the @TestRestTemplate to perform requests against the webserver  Assert the resultsWhen do you want to use @WebMvcTest? What does it auto-configure?The @WebMvcTest annotation is used for tests that focus only on Spring MVC components.Using this annotation the auto-configuration is disabled and only MVC related configurationsare configured.Tests annotated with @WebMvcTest annotation will also auto-configure Spring Security and MockMvc.Use the @WithMockUser annotation on a test method to mock the Principalused by the mockMvc requests.The @WithMockUser annotation supports the following properties:  username, to set the principal username  roles, to set the principal roles  password, to set the principal passwordWhat are the differences between @MockBean and @Mock?  @Mock can only be applied to fields and parameters  @MockBean can only be applied to classes and fields  @Mock can be used to mock java classes or interfaces  @MockBean only allows for  mocking Spring beans or creation of mock Spring beans  @MockBean can be used only with a running SpringRunner or SpringExtension  @MockBean can be used for meta-annotate other beans  @MockBean and @Mock are both included in spring-boot-starter-testWhen do you want @DataJpaTest for? What does it auto-configure?Like @WebMvcTest it disables the full auto-configuration, and apply only configurationsrelevant to JPA.By default tests annotated with @DataJpaTest are transactional and roll back at the end of each test execution.It forces the use of an embedded in-memory database.","content_html":"<p>The company I work for has a department called <em>The School<\/em> which allows employeesto sign up for internal courses and obtain certifications if necessary.<\/p><p>The courses list is extensive, and this year I have chosen to takethe <em>Spring Professional<\/em> training course to consolidate my knowledge of the framework.<br \/>The course allowed receiving a voucher to attend the <code class=\"language-plaintext highlighter-rouge\">VMware Spring Professional<\/code> exam certification,so I decided to give it a try.<\/p><p><strong>I don\u2019t really believe in certifications<\/strong>, and I even tend toconsider them a contraindication.<br \/>As expected the exam was full of nitty-gritty detail questions on <em>Spring<\/em> internalsand stuff like that, <strong>But<\/strong>, I must admit that I\u2019ve learned a lot while I was preparing for the exam.<\/p><p>Now I am aware of how <code class=\"language-plaintext highlighter-rouge\">Spring<\/code> and <code class=\"language-plaintext highlighter-rouge\">Spring Boot<\/code> work under the hood, and I better understand<code class=\"language-plaintext highlighter-rouge\">Spring<\/code> features that I don\u2019t use daily at work. I am satisfied with the choice.<\/p><h1 id=\"study-material\">Study material<\/h1><p>I spent 30 hours to prepare the exam, ~1 hour per weekday.<br \/>The most useful resource has been the official<a href=\"https:\/\/www.vmware.com\/content\/dam\/digitalmarketing\/vmware\/en\/pdf\/certification\/vmw-spring-professional-certification-study-guide.pdf\">VMware Spring study guide<\/a>.<br \/>It contains a list of questions that lead you through the topics addressed by the exam.Below the answers I wrote while studying.<\/p><h1 id=\"container-dependency-and-ioc\">Container, Dependency and IOC<\/h1><h2 id=\"what-is-dependency-injection-and-what-are-the-advantages-of-using-it\">What is dependency injection and what are the advantages of using it?<\/h2><p>The <em>dependency injection<\/em> is the linking action between dependents and dependencies.The dependent component is not allowed to inject dependencies by itself.The DI is delegated to an external authority called <em>Spring IoC container<\/em> to achieve decoupling between components.This behaviour is commonly called <em>Inversion of Control<\/em>.The injection process happens at runtime, allowing flexibility because the application behaviourcan be modified by an external configuration.<\/p><p>Other advantages are:<\/p><ul>  <li>Loosely coupled components<\/li>  <li>High cohesion<\/li>  <li>Code is more readable<\/li>  <li>Code is cleaner<\/li>  <li>Easy testing by mocking dependencies<\/li><\/ul><h2 id=\"what-is-an-interface-and-what-are-the-advantages\">What is an interface and what are the advantages?<\/h2><p>A java interface specifies behaviour that the implementing class must implementand separates what the caller expects from the implementation.<\/p><ul>  <li>Interfaces allow an object to be referenced by the methods they support without consideringthe implementation.<\/li>  <li>Interfaces allow multi-inheritance of types, which is the ability of a class to implementmore than one interface.<\/li>  <li>Interfaces can help the testing process, by creating implementations mocks<\/li><\/ul><h2 id=\"what-is-an-applicationcontext\">What is an ApplicationContext?<\/h2><p>An application context is an instance of any class that implements the<a href=\"https:\/\/docs.spring.io\/spring-framework\/docs\/current\/javadoc-api\/org\/springframework\/context\/ApplicationContext.html\">ApplicationContext<\/a>interface.<\/p><ul>  <li>It provides configuration for the Spring application.<\/li>  <li>It manages all the beans, created and initialized by the <em>Spring IoC Container<\/em>.<\/li><\/ul><h2 id=\"how-are-you-going-to-create-a-new-instance-of-an-applicationcontext\">How are you going to create a new instance of an ApplicationContext?<\/h2><p>To create a new instance of an application context you have to instantiatea class that implements the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> interface, for examplethe <code class=\"language-plaintext highlighter-rouge\">AnnotationConfigApplicationContext<\/code> class.<br \/><strong>Spring boot<\/strong> expose a convenient method <code class=\"language-plaintext highlighter-rouge\">SpringApplication.run<\/code>to bootstrap the application by creating an <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> instance.<\/p><h2 id=\"can-you-describe-the-lifecycle-of-a-spring-bean-in-an-applicationcontext\">Can you describe the lifecycle of a Spring Bean in an ApplicationContext?<\/h2><p><\/p><ol>  <li>Constructor is called    <ul>      <li>If the bean is created using the constructor DI the 1st and 2nd steps are a single step<\/li>    <\/ul>  <\/li>  <li>Dependencies are injected<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PostConstruct<\/code> (jsr-250) annoted method is called, it must be void with no arguments<\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">setBeanName(name)<\/code> declared in the <em>BeanNameAware<\/em> interface<\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">postProcessBeforeInitialization(bean, beanName)<\/code> declared in the <em>BeanPostProcess<\/em> interface<\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">afterPropertiesSet<\/code> declared in the <em>InitializingBean<\/em> interface<\/li>  <li>Call the init method defined in <code class=\"language-plaintext highlighter-rouge\">@Bean.initMethod<\/code><\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">postProcessAfterInitialization(bean, beanName)<\/code> declared in the <em>BeanPostProcess<\/em> interface<\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">postProcessBeforeDestruction(bean, beanName)<\/code> declared <em>DestructionAwareBeanPostProcessors<\/em> interface<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PreDestroy<\/code> (jsr-250) annoted method is called<\/li>  <li>Call method <code class=\"language-plaintext highlighter-rouge\">destroy()<\/code> declared in the <em>DisposableBean<\/em> interface<\/li>  <li>Call destroy method defined in <code class=\"language-plaintext highlighter-rouge\">@Bean.destroyMethod<\/code><\/li><\/ol><h2 id=\"how-are-you-going-to-create-an-applicationcontext-in-an-integration-test\">How are you going to create an ApplicationContext in an integration test?<\/h2><ol>  <li>Annotate the test class with <code class=\"language-plaintext highlighter-rouge\">@RunWith(SpringJUnit4ClassRunner.class)<\/code> if using JUnit4,or with <code class=\"language-plaintext highlighter-rouge\">@ExtendWith(SpringExtension.class)<\/code> if using JUnit 5<\/li>  <li>Annotate the test class with <code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code> to tell the runner where the definitionsof the beans are    <ul>      <li><code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code> accepts the <em>classes<\/em> and <em>locations<\/em> attributes<\/li>    <\/ul>  <\/li><\/ol><p>In <strong>Spring Boot<\/strong> just annotate the test class with <code class=\"language-plaintext highlighter-rouge\">@SpringBootTest<\/code>.<\/p><h2 id=\"what-is-the-preferred-way-to-close-an-application-context-does-spring-boot-do-this-for-you\">What is the preferred way to close an application context? Does Spring Boot do this for you?<\/h2><ul>  <li>Call the <code class=\"language-plaintext highlighter-rouge\">close()<\/code> or the <code class=\"language-plaintext highlighter-rouge\">registerShutdownHook()<\/code> of the <code class=\"language-plaintext highlighter-rouge\">ConfigurableApplicationContext<\/code> implementation class.<\/li>  <li>Call the <code class=\"language-plaintext highlighter-rouge\">exit(...)<\/code> method of the <code class=\"language-plaintext highlighter-rouge\">SpringApplication<\/code> class<\/li><\/ul><p><strong>Spring Boot<\/strong> registers a shutdown hook with the JVM to make sure the application exits appropriately.Furthermore, it exposes an actuator endpoint to shutdown the context on demand.<\/p><h2 id=\"are-beans-lazily-or-eagerly-instantiated-by-default-how-do-you-alter-this-behaviour\">Are beans lazily or eagerly instantiated by default? How do you alter this behaviour?<\/h2><p><em>Spring<\/em> instantiate beans eagerly by default.<\/p><p>To make a bean lazy:<\/p><ul>  <li>Create it as <code class=\"language-plaintext highlighter-rouge\">Prototype<\/code><\/li>  <li>Enable lazy component scanning by using the <code class=\"language-plaintext highlighter-rouge\">@ComponentScan(lazyInit = true)<\/code> annotation<\/li>  <li>Use the <code class=\"language-plaintext highlighter-rouge\">@Lazy<\/code> annotation    <ul>      <li>The <em>lazy<\/em> annotation can be used on <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> classes, <code class=\"language-plaintext highlighter-rouge\">@Bean<\/code> methods, <code class=\"language-plaintext highlighter-rouge\">@Component<\/code> classes, and along withthe <code class=\"language-plaintext highlighter-rouge\">@Autowired<\/code> annotation.<\/li>    <\/ul>  <\/li><\/ul><h2 id=\"what-is-a-property-source-how-would-you-use-propertysource\">What is a property source? How would you use @PropertySource?<\/h2><p>A property source represents a source of name\/value property pairs.The <code class=\"language-plaintext highlighter-rouge\">@PropertySource<\/code> annotation is used for adding a property source to <em>Spring\u2019s Environment<\/em>.It accepts the list of resource locations of the properties file to be loaded.<\/p><div class=\"language-java highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nd\">@Configuration<\/span><span class=\"nd\">@PropertySource<\/span><span class=\"o\">(<\/span><span class=\"s\">\"classpath:\/it\/me\/my.properties\"<\/span><span class=\"o\">)<\/span><span class=\"kd\">public<\/span> <span class=\"kd\">class<\/span> <span class=\"nc\">AppConfig<\/span> <span class=\"o\">{<\/span>  <span class=\"nd\">@Autowired<\/span>  <span class=\"nc\">Environment<\/span> <span class=\"n\">env<\/span><span class=\"o\">;<\/span>  <span class=\"nd\">@Bean<\/span>  <span class=\"kd\">public<\/span> <span class=\"nc\">TestBean<\/span> <span class=\"nf\">testBean<\/span><span class=\"o\">()<\/span> <span class=\"o\">{<\/span>    <span class=\"nc\">TestBean<\/span> <span class=\"n\">testBean<\/span> <span class=\"o\">=<\/span> <span class=\"k\">new<\/span> <span class=\"nc\">TestBean<\/span><span class=\"o\">();<\/span>    <span class=\"n\">testBean<\/span><span class=\"o\">.<\/span><span class=\"na\">setName<\/span><span class=\"o\">(<\/span><span class=\"n\">env<\/span><span class=\"o\">.<\/span><span class=\"na\">getProperty<\/span><span class=\"o\">(<\/span><span class=\"s\">\"testbean.name\"<\/span><span class=\"o\">));<\/span>    <span class=\"k\">return<\/span> <span class=\"n\">testBean<\/span><span class=\"o\">;<\/span>  <span class=\"o\">}<\/span><span class=\"o\">}<\/span><\/code><\/pre><\/div><\/div><h2 id=\"what-is-a-beanfactorypostprocessor-and-what-is-it-used-forwhen-is-it-invoked\">What is a BeanFactoryPostProcessor and what is it used for?<br \/>When is it invoked?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">BeanFactoryPostProcessor<\/code> is an interface and beans that implement it can alter othersbean <strong>definitions<\/strong>.<br \/>It is invoked during startup of the Spring context, <strong>after<\/strong> all bean definitions are loaded.For example, it is used by the property source mechanism to resolve <code class=\"language-plaintext highlighter-rouge\">@Value(${placeholder})<\/code> placeholdersby injecting values read from the <em>Spring<\/em> environment and its set of property sources, declared using the<code class=\"language-plaintext highlighter-rouge\">@PropertySource<\/code> annotation.The interface define the method <code class=\"language-plaintext highlighter-rouge\">postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory)<\/code>.<\/p><div class=\"language-java highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kd\">public<\/span> <span class=\"kd\">class<\/span> <span class=\"nc\">CustomBeanFactory<\/span> <span class=\"kd\">implements<\/span> <span class=\"nc\">BeanFactoryPostProcessor<\/span> <span class=\"o\">{<\/span>  <span class=\"nd\">@Override<\/span>  <span class=\"kd\">public<\/span> <span class=\"kt\">void<\/span> <span class=\"nf\">postProcessBeanFactory<\/span><span class=\"o\">(<\/span><span class=\"nc\">ConfigurableListableBeanFactory<\/span> <span class=\"n\">beanFactory<\/span><span class=\"o\">)<\/span> <span class=\"kd\">throws<\/span> <span class=\"nc\">BeansException<\/span> <span class=\"o\">{<\/span>    <span class=\"k\">for<\/span> <span class=\"o\">(<\/span><span class=\"nc\">String<\/span> <span class=\"n\">beanName<\/span> <span class=\"o\">:<\/span> <span class=\"n\">beanFactory<\/span><span class=\"o\">.<\/span><span class=\"na\">getBeanDefinitionNames<\/span><span class=\"o\">())<\/span> <span class=\"o\">{<\/span>      <span class=\"nc\">BeanDefinition<\/span> <span class=\"n\">beanDefinition<\/span> <span class=\"o\">=<\/span> <span class=\"n\">beanFactory<\/span><span class=\"o\">.<\/span><span class=\"na\">getBeanDefinition<\/span><span class=\"o\">(<\/span><span class=\"n\">beanName<\/span><span class=\"o\">);<\/span>      <span class=\"c1\">\/\/ manipulate bean definition<\/span>    <span class=\"o\">}<\/span>  <span class=\"o\">}<\/span><span class=\"o\">}<\/span><\/code><\/pre><\/div><\/div><h2 id=\"what-is-a-beanpostprocessor-and-how-is-it-different-to-a-beanfactorypostprocessor-what-do-they-dowhen-are-they-called\">What is a BeanPostProcessor and how is it different to a BeanFactoryPostProcessor? What do they do?<br \/>When are they called?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">BeanPostProcess<\/code> is an interface that adds some processing logic before and after executing the beaninitialization method after the <em>Spring IoC Container<\/em> instantiates the bean.The interface defines the methods <code class=\"language-plaintext highlighter-rouge\">postProcessBeforeInitialization(Object bean, String beanName)<\/code>and <code class=\"language-plaintext highlighter-rouge\">postProcessAfterInitialization(Object bean, String beanName)<\/code>.<\/p><h2 id=\"what-does-component-scanning-do\">What does component scanning do?<\/h2><p>The process of searching the classpath for classes that can contribute to the application context is called component scanning.A class must be annotated with a <em>Stereotype<\/em> annotation in order to be scanned and registered by component scanning.<\/p><h2 id=\"what-is-the-behaviour-of-the-annotation-autowired-with-regards-to-field-injection-constructor-injection-and-method-injection\">What is the behaviour of the annotation @Autowired with regards to field injection, constructor injection and method injection?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">@Autowired<\/code> annotation populates the annotated attribute with the desired bean.The <em>Bean Factory<\/em> first looks for a bean of the annotated attribute type, then it tries using the nameif none or more than one are returned it throws a <code class=\"language-plaintext highlighter-rouge\">NoSuchBeanDefinitionException<\/code> exception.<br \/>If multiple matching it tries to match name and type.<code class=\"language-plaintext highlighter-rouge\">@Qualifier<\/code> or <code class=\"language-plaintext highlighter-rouge\">@Primary<\/code> annotations can be used to alter that behaviour.<br \/>When <code class=\"language-plaintext highlighter-rouge\">@Autowired<\/code> is used on a field, <code class=\"language-plaintext highlighter-rouge\">Reflection<\/code> is used to populate the field.<code class=\"language-plaintext highlighter-rouge\">@Autowired<\/code> cannot be used to autowire primitive types.<\/p><h2 id=\"how-does-the-qualifier-annotation-complement-the-use-of-autowired\">How does the @Qualifier annotation complement the use of @Autowired?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">@Qualifier<\/code> annotation is used to give a name to a bean.When <em>Spring<\/em> cannot decide what to autowire based on type, it looks for a bean named with<code class=\"language-plaintext highlighter-rouge\">@Qualifier<\/code> annotation value.<\/p><h2 id=\"what-is-a-proxy-object-and-what-are-the-two-different-types-of-proxies-spring-can-create\">What is a proxy object and what are the two different types of proxies Spring can create?<\/h2><p>A proxy is a wrapper around the target object, it is used to alter the behaviour of the target object.Spring has two types of proxy, the prefered one is the <em>JDK Dynamic Proxy<\/em>, the other oneis the <em>CGLIB<\/em>.The former can only proxy by interface, the latter is used when the target class doesnot implement an interface, by subclassing it.The disadvantage of these proxies is that they can only intercept public methods, final classes andfinal methods are not supported too.<\/p><h2 id=\"what-does-the-bean-annotation-do\">What does the @Bean annotation do?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">@Bean<\/code> annotation indicates that a method produces a bean to be managed by the <em>Spring container<\/em>.Typically it is used in <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> classes, in this case the bean methods may reference other<code class=\"language-plaintext highlighter-rouge\">@Bean<\/code> methods in the same class by calling them directly, this mechanism is called <em>inter-bean references<\/em>.<\/p><h2 id=\"what-is-the-default-bean-id-if-you-only-use-bean-how-can-you-override-this\">What is the default bean id if you only use @Bean? How can you override this?<\/h2><p>The default bean id is the name of the method.You can override it by using the name\/value attribute of the <code class=\"language-plaintext highlighter-rouge\">@Bean<\/code> annotation.<\/p><h2 id=\"why-are-you-not-allowed-to-annotate-a-final-class-with-configuration\">Why are you not allowed to annotate a final class with @Configuration<\/h2><p>Spring uses CGLIB to proxy the <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> annotated class, and CGLIB proxy does notsupport final classes because they cannot be subclassed.<\/p><h2 id=\"how-do-you-configure-profiles-what-are-possible-use-cases-where-they-might-be-useful\">How do you configure profiles? What are possible use cases where they might be useful?<\/h2><p>A profile can be configured by using the <code class=\"language-plaintext highlighter-rouge\">@Profile<\/code> annotation.The set of profile names for which the annotated component should be registered is indicatedby the value attribute of the annotation.The profile name may contain the NOT operator, !, the annotated component is registered if theprofile is not active.<br \/><code class=\"language-plaintext highlighter-rouge\">@Profile<\/code> can be used:<\/p><ul>  <li>on any class directly or indirectly annotated with @Component<\/li>  <li>on custom annotations<\/li>  <li>on any @Bean method<\/li><\/ul><p>To select the active profiles the property <code class=\"language-plaintext highlighter-rouge\">spring.profiles.active<\/code> can be used.Profiles may also be activated in integration tests via the <code class=\"language-plaintext highlighter-rouge\">@ActiveProfile<\/code> annotation.<br \/>Profiles are used to modify the application behaviour, like loading an implementation insteadof another, by simple modifying the external configuration.<\/p><h2 id=\"can-you-use-bean-together-with-profile\">Can you use @Bean together with @Profile?<\/h2><p>Yes.<\/p><h2 id=\"can-you-use-component-together-with-profile\">Can you use @Component together with @Profile?<\/h2><p>Sure.<\/p><h2 id=\"how-many-profiles-can-you-have\">How many profiles can you have?<\/h2><p>Many as you want.<\/p><h2 id=\"how-do-you-inject-scalarliteral-values-into-spring-beans\">How do you inject scalar\/literal values into Spring beans?<\/h2><p>To inject primitive types you must use the <code class=\"language-plaintext highlighter-rouge\">@Value<\/code> annotation.<br \/>It supports <em>SpEL<\/em> language <code class=\"language-plaintext highlighter-rouge\">#{systemProperties.myProp}<\/code>.Alternatively, values may be injected using <code class=\"language-plaintext highlighter-rouge\">${my.app.myProp}<\/code> style property placeholders.<br \/>The <code class=\"language-plaintext highlighter-rouge\">@Value<\/code> processing is performed by a <em>BeanPostProcessor<\/em>, meaning that <code class=\"language-plaintext highlighter-rouge\">@Value<\/code> annotationcan\u2019t be used within <em>BeanPostProcessor<\/em> or <em>BeanFactoryPostProcessor<\/em> types.<\/p><h2 id=\"what-is-spring-expression-language-spel-for-short\">What is Spring Expression Language (SpEL for short)?<\/h2><p><em>SpEL<\/em> is a powerful expression language that supports manipulating an object at runtime.<br \/>It supports the following functionality:<\/p><ul>  <li>Literal expressions<\/li>  <li>Boolean and relational operators<\/li>  <li>Regular expressions<\/li>  <li>Class expressions<\/li>  <li>Accessing properties, arrays, lists, maps<\/li>  <li>Method invocation<\/li>  <li>Relational operators<\/li>  <li>Assignment<\/li>  <li>Calling constructors<\/li>  <li>Bean references<\/li>  <li>Array construction<\/li>  <li>Inline lists<\/li>  <li>Ternary operator<\/li>  <li>Variables<\/li>  <li>User-defined functions<\/li>  <li>Collection projection<\/li>  <li>Collection selection<\/li>  <li>Templated expressions<\/li><\/ul><h2 id=\"what-is-the-environment-abstraction-in-spring\">What is the Environment abstraction in Spring?<\/h2><p>The <em>Spring Environment<\/em> represents the environment in which the application is running.It unifies access to all types of property sources, such as property files, JVM system properties,system environment variables and servlet context parameters.<\/p><h2 id=\"where-can-properties-in-the-environment-come-from--there-are-many-sources-for-properties--check-the-documentation-if-not-sure-spring-boot-adds-even-more\">Where can properties in the environment come from \u2013 there are many sources for properties \u2013 check the documentation if not sure. Spring Boot adds even more.<\/h2><p>The Spring boot <code class=\"language-plaintext highlighter-rouge\">PropertySource<\/code> order is:<\/p><ul>  <li>Devtools global settings properties on your home directory (~\/.spring-boot-devtools.properties when devtools is active).<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@TestPropertySource<\/code> annotations on your tests.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@SpringBootTest#properties<\/code> annotation attribute on your tests.<\/li>  <li>Command line arguments.<\/li>  <li>Properties from <code class=\"language-plaintext highlighter-rouge\">SPRING_APPLICATION_JSON<\/code> (inline JSON embedded in an environment variable or system property)<\/li>  <li>ServletConfig init parameters.<\/li>  <li>ServletContext init parameters.<\/li>  <li>JNDI attributes from java:comp\/env.<\/li>  <li>Java System properties (System.getProperties()).<\/li>  <li>OS environment variables.<\/li>  <li>A RandomValuePropertySource that only has properties in random.*.<\/li>  <li>Profile-specific application properties outside of your packaged jar (<code class=\"language-plaintext highlighter-rouge\">application-{profile}.properties<\/code> and YAML variants)<\/li>  <li>Profile-specific application properties packaged inside your jar (<code class=\"language-plaintext highlighter-rouge\">application-{profile}.properties<\/code> and YAML variants)<\/li>  <li>Application properties outside of your packaged jar (application.properties and YAML variants).<\/li>  <li>Application properties packaged inside your jar (application.properties and YAML variants).<\/li>  <li>@PropertySource annotations on your <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> classes.<\/li>  <li>Default properties (specified using SpringApplication.setDefaultProperties).<\/li><\/ul><h2 id=\"what-can-you-reference-using-spel\">What can you reference using SpEL?<\/h2><p>Beans.<\/p><h2 id=\"what-is-the-difference-between--and--in-value-expressions\">What is the difference between $ and # in @Value expressions?<\/h2><p>The former is the property placeholder style, used to inject primitives types,the latter indicates that it is a <em>SpEL<\/em> expression.<\/p><h1 id=\"aspect-oriented-programming-aop\">Aspect-Oriented Programming (AOP)<\/h1><h2 id=\"what-is-the-concept-of-aop-which-problem-does-it-solve-what-is-a-cross-cutting-concern\">What is the concept of AOP? Which problem does it solve? What is a cross-cutting concern?<\/h2><p><em>AOP<\/em>, Aspect-Oriented Programming, is a programming paradigm where is possible toalter the behaviour of the existing code without modification of the codeitself.<br \/><em>AOP<\/em> help to avoid <em>tangling<\/em> and <em>scattering<\/em>.<\/p><div class=\"language-java highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"c1\">\/\/ tangling and scattering example, logging is mixed with business-logic<\/span><span class=\"c1\">\/\/ and scattered across the codebase.<\/span><span class=\"kd\">public<\/span> <span class=\"kt\">void<\/span> <span class=\"nf\">aMethodService<\/span><span class=\"o\">()<\/span> <span class=\"o\">{<\/span>  <span class=\"n\">logger<\/span><span class=\"o\">.<\/span><span class=\"na\">entering<\/span><span class=\"o\">(<\/span><span class=\"s\">\"MyService\"<\/span><span class=\"o\">,<\/span><span class=\"s\">\"aMethodService\"<\/span><span class=\"o\">);<\/span>  <span class=\"n\">anotherService<\/span><span class=\"o\">.<\/span><span class=\"na\">myMethod<\/span><span class=\"o\">();<\/span>  <span class=\"n\">logger<\/span><span class=\"o\">.<\/span><span class=\"na\">exiting<\/span><span class=\"o\">(<\/span><span class=\"s\">\"MyService\"<\/span><span class=\"o\">,<\/span><span class=\"s\">\"aMethodService\"<\/span><span class=\"o\">);<\/span><span class=\"o\">}<\/span><span class=\"kd\">public<\/span> <span class=\"kt\">void<\/span> <span class=\"nf\">anotherMethodService<\/span><span class=\"o\">()<\/span> <span class=\"o\">{<\/span>  <span class=\"n\">logger<\/span><span class=\"o\">.<\/span><span class=\"na\">entering<\/span><span class=\"o\">(<\/span><span class=\"s\">\"MyService\"<\/span><span class=\"o\">,<\/span><span class=\"s\">\"anotherMethodService\"<\/span><span class=\"o\">);<\/span>  <span class=\"n\">suchService<\/span><span class=\"o\">.<\/span><span class=\"na\">myMethod<\/span><span class=\"o\">();<\/span>  <span class=\"n\">logger<\/span><span class=\"o\">.<\/span><span class=\"na\">exiting<\/span><span class=\"o\">(<\/span><span class=\"s\">\"MyService\"<\/span><span class=\"o\">,<\/span><span class=\"s\">\"anotherMethodService\"<\/span><span class=\"o\">);<\/span><span class=\"o\">}<\/span><\/code><\/pre><\/div><\/div><p>A cross-cutting concern is a functionality shared across multiple application modules.Such as logging, because is something that can happen across our application.<\/p><h2 id=\"what-is-a-pointcut-a-join-point-an-advice-an-aspect-weaving\">What is a pointcut, a join point, an advice, an aspect, weaving?<\/h2><p>A <strong>pointcut<\/strong> is an expression that matches join points, for example the executionof a method with a certain name.<\/p><p>A <strong>join point<\/strong> is a point during the execution of a program, in <em>Spring AOP<\/em> isalways a method execution.<\/p><p>An <strong>advice<\/strong> is the code to run by an <em>aspect<\/em> at a particular <em>join point<\/em>.<\/p><p>An <strong>aspect<\/strong> is the class that contains the cross-cutting concern logic.It is annotated with the <code class=\"language-plaintext highlighter-rouge\">@Aspect<\/code> annotation.<\/p><p>The <strong>weaving<\/strong> is the mechanism used by <code class=\"language-plaintext highlighter-rouge\">Spring AOP<\/code> to proxy the target object to create an advised object.<code class=\"language-plaintext highlighter-rouge\">Spring AOP<\/code> only supports runtime weaving by using <code class=\"language-plaintext highlighter-rouge\">JDK Dynamic Proxy<\/code> or <code class=\"language-plaintext highlighter-rouge\">CGLIB<\/code>, the former is prefered, butyou can switch to the latter by using <code class=\"language-plaintext highlighter-rouge\">@EnableAspectJAutoProxy(proxyTargetClass=true)<\/code>.<\/p><h2 id=\"how-does-spring-solve-implement-a-cross-cutting-concern\">How does Spring solve (implement) a cross-cutting concern?<\/h2><p>Spring implement a <em>cross-cutting concern<\/em> by weaving the <em>aspect<\/em> to the target object, creating a proxy thatcan intercept the execution of the method as specified by the <em>pointcuts<\/em>.<\/p><h2 id=\"which-are-the-limitations-of-the-two-proxy-types\">Which are the limitations of the two proxy-types?<\/h2><p><em>JDK Dynamic Proxy<\/em> limitations:<\/p><ul>  <li>Target class must implement an interface<\/li>  <li>*-private, protected and final methods are not supported<\/li>  <li>Methods not defined in the interface are not supported<\/li>  <li>Does not support self-referencing, if a matched <em>join point<\/em> method callsanother method of the same class matched by a <em>join point<\/em>, the advice is calledjust once.<\/li><\/ul><p><em>CGLIB<\/em> limitations:<\/p><ul>  <li>Target class can\u2019t be <em>final<\/em>, as <em>CGLIB<\/em> can\u2019t subclass it<\/li>  <li>Private and final methods are not supported<\/li>  <li>Protected methods are supported but not recommended<\/li><\/ul><h2 id=\"how-many-advice-types-does-spring-support-can-you-name-each-one\">How many advice types does Spring support? Can you name each one?<\/h2><p>In <em>Spring AOP<\/em> there are multiple advice types:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">@Before<\/code> advice, will execute before the <em>join point<\/em>, it can\u2019t prevent the executionof the method unless it throws an exception<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@AfterReturning<\/code> advice, will execute after the <em>join point<\/em> completes normally , meaning thatthe target method returns normally without throwing an exception.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@AfterThrowing<\/code> advice, will execute after the <em>join point<\/em> execution that ended by throwingan exception<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@After<\/code> advice, will execute after the <em>join point<\/em> execution, no matter the how execution ended.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@Around<\/code> advice, which is the most powerful advice, will intercept the target method and surroundthe <em>join point<\/em>. It can perform custom behaviour before and after target invocation, it can execute ornot the target method and stop the propagation of a possible exception raised by the target.<\/li><\/ul><h2 id=\"if-shown-pointcut-expressions-would-you-understand-them\">If shown pointcut expressions, would you understand them?<\/h2><p>Possible <em>pointcut<\/em> expressions are:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">execution([Modifiers] [ReturnType] [FullClassName].[MethodName]([Arguments]) throws [ExceptionType])<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">whithin()<\/code>, limits maching to <em>join point<\/em> of certain types, <code class=\"language-plaintext highlighter-rouge\">@Pointcut(\"within(it.manuel..*)\")<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">target()<\/code> and <code class=\"language-plaintext highlighter-rouge\">this()<\/code>, the former is used to match a class that implements the specified interface, andthe latter is used to match a specific class type.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@target<\/code> limits matching to <em>join point<\/em> where the class of the target method has a specific annotation,<code class=\"language-plaintext highlighter-rouge\">@Pointcut(\"@target(org.springframework.stereotype.Service)\")<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@args<\/code> limits matching to <em>join point<\/em> where the argument of the target method has a specific annotation,<code class=\"language-plaintext highlighter-rouge\">@Pointcut(\"@args(it.manuel.annotations.MyArgAnnotation)\")<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@annotation<\/code> limits matching to <em>join point<\/em> where the target method has a specific annotation,<code class=\"language-plaintext highlighter-rouge\">@Pointcut(\"@annotation(org.springframework.beans.factory.annotation.Qualifier)\")<\/code><\/li><\/ul><p>The <em>pointcut<\/em> expression supports wildcards such as <code class=\"language-plaintext highlighter-rouge\">+<\/code> and <code class=\"language-plaintext highlighter-rouge\">*<\/code>, and logical operators like <code class=\"language-plaintext highlighter-rouge\">&amp;&amp;<\/code>, <code class=\"language-plaintext highlighter-rouge\">||<\/code> and <code class=\"language-plaintext highlighter-rouge\">!<\/code>.The <code class=\"language-plaintext highlighter-rouge\">+<\/code> wildcard indicates that the method can also be matched in subclasses <code class=\"language-plaintext highlighter-rouge\">it.manuel.+.helloWorld<\/code>.<\/p><h2 id=\"what-is-the-joinpoint-argument-used-for\">What is the JoinPoint argument used for?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">JoinPoint<\/code> type is the argument type used for all the <em>advice<\/em> annotated methods, except <code class=\"language-plaintext highlighter-rouge\">@Around<\/code>,which argument type is <code class=\"language-plaintext highlighter-rouge\">ProceedingJoinPoint<\/code>.<\/p><h2 id=\"what-is-a-proceedingjoinpoint-which-advice-type-is-it-used-with\">What is a ProceedingJoinPoint? Which advice type is it used with?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">ProceedingJoinPoint<\/code> is the argument type for <code class=\"language-plaintext highlighter-rouge\">@Around<\/code> advice.It exposes the <code class=\"language-plaintext highlighter-rouge\">.proceed<\/code> method used to call the method matched by the <em>pointcut<\/em>.<\/p><h1 id=\"data-management-jdbc-transactions\">Data Management: JDBC, Transactions<\/h1><h2 id=\"what-is-the-difference-between-checked-and-unchecked-exceptions\">What is the difference between checked and unchecked exceptions?<\/h2><p>A checked exception is a compile-time exception that must be managed.An unchecked exception is a runtime exception, it\u2019s not required to manage it.The latter is used by the data access mechanism to report errors, by throwingexceptions that extend the <code class=\"language-plaintext highlighter-rouge\">DataAccessException<\/code> class.<br \/>Rollback is automatic only on unchecked exceptions.<\/p><h2 id=\"how-do-you-configure-a-datasource-in-spring\">How do you configure a DataSource in Spring?<\/h2><p>To access a database using <em>JDBC<\/em>, the following are needed:<\/p><ul>  <li>A DB driver to dialogue with the database<\/li>  <li>The DB connection url<\/li>  <li>The DB credentials<\/li><\/ul><p>A <em>DataSource<\/em> is created using the previous information, and each database librarymust implement it.<\/p><p>In Spring to programmatically create a <code class=\"language-plaintext highlighter-rouge\">DataSource<\/code> you have to define a <code class=\"language-plaintext highlighter-rouge\">DataSource<\/code> beanand return an instance of the implementation provided by the DB vendor you are using.<\/p><p>In <strong>Spring Boot<\/strong>, is possible to define a <em>DataSource<\/em> by setting properties<\/p><div class=\"language-yaml highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"s\">app.datasource.url=jdbc:h2:mem:hello<\/span><span class=\"s\">app.datasource.username=hello<\/span><span class=\"s\">app.datasource.password=world<\/span><\/code><\/pre><\/div><\/div><p><strong>Spring Boot<\/strong> also provide a class builder named <code class=\"language-plaintext highlighter-rouge\">DataSourceBuilder<\/code>, it auto-detect the driverbased on what is available on the classpath or based on the <em>JDBC<\/em> URL.<\/p><h2 id=\"what-is-the-template-design-pattern-and-what-is-the-jdbc-template\">What is the Template design pattern and what is the JDBC template?<\/h2><p>The template design pattern is one of the most popular, itenables the developer to implement a complex functionality by encapsulatingthe logic in a single method, the so-called template method.The effective implementation is defined later or delegated to subclasses.<\/p><div class=\"language-java highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"kd\">public<\/span> <span class=\"kd\">abstract<\/span> <span class=\"kd\">class<\/span> <span class=\"nc\">MyTemplateBuilder<\/span> <span class=\"o\">{<\/span>  <span class=\"kd\">public<\/span> <span class=\"kd\">final<\/span> <span class=\"nc\">JdbcTemplate<\/span> <span class=\"nf\">buildHouse<\/span><span class=\"o\">()<\/span> <span class=\"o\">{<\/span>    <span class=\"n\">setupFoundation<\/span><span class=\"o\">();<\/span>    <span class=\"n\">buildWalls<\/span><span class=\"o\">();<\/span>    <span class=\"n\">buildRoof<\/span><span class=\"o\">();<\/span>    <span class=\"n\">paintWalls<\/span><span class=\"o\">();<\/span>  <span class=\"o\">}<\/span>  <span class=\"kd\">public<\/span> <span class=\"kd\">abstract<\/span> <span class=\"nf\">setupFoundation<\/span><span class=\"o\">();<\/span>  <span class=\"kd\">public<\/span> <span class=\"kd\">abstract<\/span> <span class=\"nf\">buildWalls<\/span><span class=\"o\">();<\/span>  <span class=\"kd\">public<\/span> <span class=\"kd\">abstract<\/span> <span class=\"nf\">buildRoof<\/span><span class=\"o\">();<\/span>  <span class=\"kd\">public<\/span> <span class=\"kd\">abstract<\/span> <span class=\"nf\">paintWalls<\/span><span class=\"o\">();<\/span><span class=\"o\">}<\/span><\/code><\/pre><\/div><\/div><p>The <em>JDBC Template<\/em> is the central class to manage DB operations.Under the hood it manages all the repetitive actions to execute a query,such as opening and closing the connection, handling exceptions and transactions,leaving application code to only provide SQL and extract results.<\/p><h2 id=\"what-is-a-callback-what-are-the-jdbctemplate-callback-interfaces-that-can-be-used-with-queries-what-is-each-used-for-you-would-not-have-to-remember-the-interface-names-in-the-exam-but-you-should-know-what-they-do-if-you-see-them-in-a-code-sample\">What is a callback? What are the JdbcTemplate callback interfaces that can be used with queries? What is each used for? (You would not have to remember the interface names in the exam, but you should know what they do if you see them in a code sample).<\/h2><p>A callback is a method called by another method when this one finished the execution,usually, the callback method argument is the return value of the former.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">JdbcTemplate<\/code> query method accepts 3 callback interface types:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">ResultSetExtractor&lt;T&gt;<\/code> interface expose the <code class=\"language-plaintext highlighter-rouge\">&lt;T&gt; T extractData(ResultSet set)<\/code> method<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">RowMapper&lt;T&gt;<\/code>expose the <code class=\"language-plaintext highlighter-rouge\">&lt;T&gt; List&lt;T&gt; mapRow(ResultSet set, int rowNum)<\/code> method<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">RowCallbackHandler<\/code> expose the <code class=\"language-plaintext highlighter-rouge\">void processRow(ResultSet set)<\/code>  method<\/li><\/ul><p><code class=\"language-plaintext highlighter-rouge\">ResultSetExtractor<\/code> is supposed to extract the whole <code class=\"language-plaintext highlighter-rouge\">ResultSet<\/code>, it may contain multiple rows,while <code class=\"language-plaintext highlighter-rouge\">RowMapper<\/code> is fed with a row at a time.Both are typically <em>stateless<\/em>.<\/p><p><code class=\"language-plaintext highlighter-rouge\">RowCallbackHandler<\/code> is fed with a row at time, and it is tipically <em>stateful<\/em>.<\/p><h2 id=\"can-you-execute-a-plain-sql-statement-with-the-jdbc-template\">Can you execute a plain SQL statement with the JDBC template?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">JdbcTemplate<\/code> execute plain SQL queries.<\/p><h2 id=\"when-does-the-jdbc-template-acquire-and-release-a-connection-for-every-method-called-or-once-per-template-why\">When does the JDBC template acquire (and release) a connection, for every method called or once per template? Why?<\/h2><p><em>JDBC template<\/em> acquire a connection for every method called.<\/p><h2 id=\"how-does-the-jdbctemplate-support-queries-how-does-it-return-objects-and-listsmaps-of-objects\">How does the JdbcTemplate support queries? How does it return objects and lists\/maps of objects?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">JdbcTemplate<\/code> support queries for any type.<br \/>It exposes many overloaded methods:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">&lt;T&gt; List&lt;T&gt; query(...)<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Map&lt;String, Object&gt; queryForMap(...)<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">&lt;T&gt; T queryForObject(...)<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">&lt;T&gt; List&lt;T&gt; queryForList(...)<\/code><\/li><\/ul><h2 id=\"what-is-a-transaction-what-is-the-difference-between-a-local-and-a-global-transaction\">What is a transaction? What is the difference between a local and a global transaction?<\/h2><p>In a DB context, a transaction is an atomic set of operations, when all operations have completed successfullythe changes are persisted, else if one operation fails, the transaction is <em>rolled back<\/em> leaving the database pristine.<\/p><p>A local transaction is a transaction that involves only local operations,whereas a global transaction involves external operations, such as an external database operationor a message on an external queue.<\/p><h2 id=\"what-does-declarative-transaction-management-mean\">What does declarative transaction management mean?<\/h2><p>Declarative transaction management is a non-invasive approach for managing transactions providedby <em>Spring AOP<\/em>.It means that you can enable and tag a class or a method transactional by using annotations.<br \/>To use transactions is sufficient to add <code class=\"language-plaintext highlighter-rouge\">@EnableTransactionManagement<\/code> to your configurationand annotate your classes or methods with the <code class=\"language-plaintext highlighter-rouge\">@Transactional<\/code> annotation.<\/p><h2 id=\"what-is-the-default-rollback-policy-how-can-you-override-it\">What is the default rollback policy? How can you override it?<\/h2><p>By default, a transaction is rolled back when an unchecked exception occurs withinthe transactional code.<br \/>It is possible to override that policy by configuring the <code class=\"language-plaintext highlighter-rouge\">@Transactional<\/code> annotation parameters:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">noRollbackFor<\/code> to define which exception types must not cause a transaction rollback<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">noRollbackForClassName<\/code> to define which exception types must not cause a transaction rollback<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">rollbackFor<\/code> to define which exception types must cause a transaction rollback<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">rollbackForClassName<\/code> to define which exception types must cause a transaction rollback<\/li><\/ul><h2 id=\"what-is-the-default-rollback-policy-in-a-junit-test-when-you-use-the-runwithspringjunit4classrunnerclass-in-junit-4-or-extendwithspringextension-class-in-junit-5-and-annotate-your-test-annotated-method-with-transactional\">What is the default rollback policy in a JUnit test, when you use the @RunWith(SpringJUnit4ClassRunner.class) in JUnit 4 or ExtendWith(SpringExtension. class) in JUnit 5, and annotate your @Test annotated method with @Transactional?<\/h2><p>By default tests do not commit, they always require a rollback.That behaviour can be altered by using the <code class=\"language-plaintext highlighter-rouge\">@Rollback(false)<\/code> annotation orthe <code class=\"language-plaintext highlighter-rouge\">@Commit<\/code> annotation.<\/p><h2 id=\"are-you-able-to-participate-in-a-given-transaction-in-spring-while-working-with-jpa\">Are you able to participate in a given transaction in Spring while working with JPA?<\/h2><p>Yes, by using the <code class=\"language-plaintext highlighter-rouge\">TransactionAwareDataSourceProxy<\/code> class.<\/p><h2 id=\"which-platformtransactionmanagers-can-you-use-with-jpa\">Which PlatformTransactionManager(s) can you use with JPA?<\/h2><ul>  <li><code class=\"language-plaintext highlighter-rouge\">JpaTransactionManager<\/code>, also supports direct <code class=\"language-plaintext highlighter-rouge\">DataSource<\/code> access within transaction<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">JtaTransactionManager<\/code>, necessary for accessing multiple transaction resources within the same transaction<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">HibernateTransactionManager<\/code>, ref<a href=\"https:\/\/docs.spring.io\/spring-framework\/docs\/current\/reference\/html\/data-access.html#orm-jpa-tx\">https:\/\/docs.spring.io\/spring-framework\/docs\/current\/reference\/html\/data-access.html#orm-jpa-tx<\/a><\/li><\/ul><h2 id=\"what-do-you-have-to-configure-to-use-jpa-with-spring-how-does-spring-boot-make-this-easier\">What do you have to configure to use JPA with Spring? How does Spring Boot make this easier?<\/h2><p>To enable <em>JPA<\/em> must be used the <code class=\"language-plaintext highlighter-rouge\">@EnableJpaRepositories<\/code> annotation, it will scan the packageof the annotated configuration class for <em>Spring Data<\/em> repositories by default.To change the package to scan the <code class=\"language-plaintext highlighter-rouge\">basePackages<\/code> or <code class=\"language-plaintext highlighter-rouge\">basePackageClasses<\/code> attributes can be used.<\/p><p>To configure <em>JPA<\/em> an <code class=\"language-plaintext highlighter-rouge\">EntityManagerFactory<\/code> instance must be provided, for example by using the<code class=\"language-plaintext highlighter-rouge\">LocalContainerEntityManagerFactoryBean<\/code> class.The <code class=\"language-plaintext highlighter-rouge\">FactoryBean<\/code> reads configuration settings from the <code class=\"language-plaintext highlighter-rouge\">META-INF\/persistence.xml<\/code> config file.<\/p><p><em>Spring Boot<\/em> automagically setup <em>JPA<\/em> and transactions if the <code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-data-jpa<\/code> module is on theclasspath.<\/p><h1 id=\"spring-data-jpa\">Spring Data JPA<\/h1><h2 id=\"what-is-a-spring-data-repository-interface\">What is a Spring Data Repository interface?<\/h2><h2 id=\"how-do-you-define-a-spring-data-repository-interface-why-is-it-an-interface-not-a-class\">How do you define a Spring Data Repository interface? Why is it an interface not a class?<\/h2><p>A <em>Spring Data Repository<\/em> interface typically extends one of the interfaces in the <code class=\"language-plaintext highlighter-rouge\">Repository&lt;T, ID extends Serializable&gt;<\/code>hierarchy.Such interfaces expose a ready-to-use set of methods to manipulate entities.Beans of this type are called <em>instant repositories<\/em>, because Spring looks for these interfaces types and implementsthem at runtime to create beans.<\/p><p>Alternatively can be used the <code class=\"language-plaintext highlighter-rouge\">@RepositoryDefinition<\/code> annotation to create a repository, passing to the<code class=\"language-plaintext highlighter-rouge\">domainClass<\/code> attribute the type of entity and to the <code class=\"language-plaintext highlighter-rouge\">idClass<\/code> attribute the id type.<\/p><h2 id=\"what-is-the-naming-convention-for-finder-methods-in-a-spring-data-repository-interface\">What is the naming convention for finder methods in a Spring Data Repository interface?<\/h2><p>The supported query keywords are<\/p><ul>  <li>find\u2026By<\/li>  <li>read\u2026By<\/li>  <li>get\u2026By<\/li>  <li>query\u2026By<\/li>  <li>search\u2026By<\/li>  <li>stream\u2026By<\/li><\/ul><p>To specify the number of returned records:<\/p><ul>  <li>\u2026First<number><\/number><\/li>  <li>\u2026Top<number><\/number><\/li><\/ul><p>To return only unique results:<\/p><ul>  <li>\u2026Distinct\u2026<\/li><\/ul><h2 id=\"how-are-spring-data-repositories-implemented-by-spring-at-runtime\">How are Spring Data repositories implemented by Spring at runtime?<\/h2><p>Spring uses a factory bean named <code class=\"language-plaintext highlighter-rouge\">RepositoryFactorySupport<\/code> to create instancesof a given repository interface.It creates a <em>JDK Dynamic<\/em> proxy implementing the repository interface and apply an <em>advice<\/em> handling the control to the<code class=\"language-plaintext highlighter-rouge\">QueryExecutorMethodInterceptor<\/code> class.<\/p><h2 id=\"what-is-query-used-for\">What is @Query used for?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">Query<\/code> annotation is used to declare finder queries directly on repository methods.It accepts both <em>JPQL<\/em> (only <em>JPA<\/em>) and native <em>SQL<\/em> queries.The <code class=\"language-plaintext highlighter-rouge\">@Query<\/code> annotation takes precedence over <code class=\"language-plaintext highlighter-rouge\">@NamedQuery<\/code> annotation.To create update queries add the <code class=\"language-plaintext highlighter-rouge\">@Modifying<\/code> annotation to the <code class=\"language-plaintext highlighter-rouge\">@Query<\/code> annotated method.<\/p><h1 id=\"spring-mvc-basics\">Spring MVC Basics<\/h1><h2 id=\"what-is-the-controller-annotation-used-for\">What is the @Controller annotation used for?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@Controller<\/code> <em>stereotype<\/em> annotation is used to indicate that the annotated class provides handler methodsfor <em>HTTP<\/em> requests.Typically it is used in combination with handler methods annotated with <code class=\"language-plaintext highlighter-rouge\">@RequestMapping<\/code>.<\/p><h2 id=\"how-is-an-incoming-request-mapped-to-a-controller-and-mapped-to-a-method\">How is an incoming request mapped to a controller and mapped to a method?<\/h2><p>The <em>Spring Web MVC<\/em> entry point is the <code class=\"language-plaintext highlighter-rouge\">DispatcherServlet<\/code> class, it dispatches HTTP requeststo handlers that match the request pattern.<\/p><h2 id=\"what-is-the-difference-between-requestmapping-and-getmapping\">What is the difference between @RequestMapping and @GetMapping?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@GetMapping<\/code> is equal to <code class=\"language-plaintext highlighter-rouge\">@RequestMapping(method = RequestMethod.GET)<\/code>.<\/p><h2 id=\"what-is-requestparam-used-for\">What is @RequestParam used for?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@RequestParam<\/code> annotation is used to indicate a query parameter in the request URL.To indicate an optional query parameter set to false the <code class=\"language-plaintext highlighter-rouge\">required<\/code> attribute.A default value can be provided by using the <code class=\"language-plaintext highlighter-rouge\">defaultValue<\/code> attribute.<\/p><h2 id=\"what-are-the-differences-between-requestparam-and-pathvariable\">What are the differences between @RequestParam and @PathVariable?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@PathVariable<\/code> annotation is used to indicate a path parameter in the request URL.<\/p><h2 id=\"what-are-the-ready-to-use-argument-types-you-can-use-in-a-controller-method\">What are the ready-to-use argument types you can use in a controller method?<\/h2><p>The signatures of the handler methods are very flexible, they can receive almost any type-specificto a web application.The following are some parameters types:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">@RequestHeader<\/code> annotated parameters to gather a request header<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">WebRequest<\/code>, <code class=\"language-plaintext highlighter-rouge\">NativeWebRequest<\/code> and <code class=\"language-plaintext highlighter-rouge\">ServletRequest<\/code> provide access to headers, request and session attributes<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">ServletRespose<\/code> to enrich the response<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">HttpSession<\/code> to access the session<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Principal<\/code> to access the credentials<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">HttpMethod<\/code> which represents the request method<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Locale<\/code> to get the current locale<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">BindingResult<\/code> to access the validation reports<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Error<\/code> to access the validation errors<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">InputStream<\/code>, <code class=\"language-plaintext highlighter-rouge\">Reader<\/code> to access the raw body of the request<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">OutputStream<\/code>, <code class=\"language-plaintext highlighter-rouge\">Writer<\/code> to access the raw body of the response<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@RequestParam<\/code> annotated parameter to get the query parameters<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PathVariable<\/code> annotated parameter to access the path variables<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MatrixVariable<\/code> annotated parameter to access the matrix variables, ex. <code class=\"language-plaintext highlighter-rouge\">http:\/\/hello.world\/;g=1;u=3;=9<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@CookieValue<\/code> annotated parameter to access the cookies<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Map<\/code>, <code class=\"language-plaintext highlighter-rouge\">Model<\/code> or <code class=\"language-plaintext highlighter-rouge\">ModelMap<\/code> that contain the data that the view template will render<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">RedirectAttributes<\/code> along with <code class=\"language-plaintext highlighter-rouge\">@ModelAttribute<\/code> to specify data to be used by a view which was redirected to<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@SessionAttribute<\/code> or <code class=\"language-plaintext highlighter-rouge\">@RequestAttribute<\/code> to access to request and session attributes<\/li><\/ul><h2 id=\"what-are-some-of-the-valid-return-types-of-a-controller-method\">What are some of the valid return types of a controller method?<\/h2><ul>  <li><code class=\"language-plaintext highlighter-rouge\">HttpEntity<\/code>, <code class=\"language-plaintext highlighter-rouge\">ResponseEntity<\/code> to return a full response<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">HttpHeaders<\/code> to return a response without a body<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">String<\/code> to specify a logical view name<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">View<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Map<\/code> or  <code class=\"language-plaintext highlighter-rouge\">Model<\/code> to specify attributes for the model<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">ModelAndView<\/code> to specify model and view to use<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">void<\/code> to return an empty response or select the default view<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">DeferredResult<\/code>, <code class=\"language-plaintext highlighter-rouge\">CompletableFuture<\/code>, etc. to produce an async response from any thread<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Callable&lt;V&gt;<\/code> to produce an async response in a <em>Spring MVC<\/em> thread<\/li><\/ul><h1 id=\"spring-mvc-rest\">Spring MVC REST<\/h1><h2 id=\"what-does-rest-stand-for\">What does REST stand for?<\/h2><p>REpresentational<br \/>State<br \/>Transfer<\/p><h2 id=\"what-is-a-resource\">What is a resource?<\/h2><p>A <em>REST<\/em> resource is something referenced by an URL.I can get the apple number 1 by using theurl <code class=\"language-plaintext highlighter-rouge\">\/api\/apple\/1<\/code>.<\/p><h2 id=\"is-rest-secure-what-can-you-do-to-secure-it\">Is REST secure? What can you do to secure it?<\/h2><p>It is secure if used safely.For example by using the <code class=\"language-plaintext highlighter-rouge\">HTTPS<\/code> protocol and require some credentials to accessa protected resource.<\/p><h2 id=\"is-rest-scalable-andor-interoperable\">Is REST scalable and\/or interoperable?<\/h2><p>Yes, it is scalable and interoperable.It does not mandate a specific technology either for the client or server.<\/p><h2 id=\"which-http-methods-does-rest-use\">Which HTTP methods does REST use?<\/h2><p><em>REST<\/em> use the following methods<\/p><ul>  <li><em>GET<\/em>, to get a resource, it is idempotent<\/li>  <li><em>POST<\/em>, to create a resource, it is not idempotent<\/li>  <li><em>PUT<\/em>, to update a resource, it is idempotent<\/li>  <li><em>PATCH<\/em>, to partially update a resource, it is not idempotent<\/li>  <li><em>DELETE<\/em>, to delete a resource, it is idempotent<\/li>  <li><em>OPTIONS<\/em>, to list the supported methods of a resource, it is idempotent<\/li>  <li><em>HEAD<\/em>, to only get the headers related to a resource, it is idempotent<\/li><\/ul><h2 id=\"what-is-an-httpmessageconverter\">What is an HttpMessageConverter?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">HttpMessageConverter<\/code> is the interface used for converting from and to <em>HTTP<\/em> requests and responses.<br \/>By default the following converters are enabled:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">ByteArrayHttpMessageConverter<\/code>, converts byte arrays<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">StringHttpMessageConverter<\/code>, converts Strings<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">ResourceHttpMessageConverter<\/code>, converts org.springframework.core.io.Resource for any type of octet stream<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">SourceHttpMessageConverter<\/code>, converts javax.xml.transform.Source<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">FormHttpMessageConverter<\/code>, converts form data to\/from a MultiValueMap&lt;String, String&gt;.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">Jaxb2RootElementHttpMessageConverter<\/code>, converts Java objects to\/from XML (added only if JAXB2 is present on the classpath)<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">MappingJackson2HttpMessageConverter<\/code>, converts JSON (added only if Jackson 2 is present on the classpath)<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">MappingJacksonHttpMessageConverter<\/code>, converts JSON (added only if Jackson is present on the classpath)<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">AtomFeedHttpMessageConverter<\/code>, converts Atom feeds (added only if Rome is present on the classpath)<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">RssChannelHttpMessageConverter<\/code>, converts RSS feeds (added only if Rome is present on the classpath)<\/li><\/ul><h2 id=\"is-controller-a-stereotype-is-restcontroller-a-stereotype\">Is @Controller a stereotype? Is @RestController a stereotype?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">@Controller<\/code> is a stereotype annotation.<code class=\"language-plaintext highlighter-rouge\">@RestController<\/code> is a specialization of the <code class=\"language-plaintext highlighter-rouge\">@controller<\/code> annotation, but it can\u2019t be considereda stereotype annotation.<\/p><h2 id=\"what-is-the-difference-between-controller-and-restcontroller\">What is the difference between @Controller and @RestController?<\/h2><h2 id=\"when-do-you-need-to-use-responsebody\">When do you need to use @ResponseBody?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">@RestController<\/code> is meta-annotated with <code class=\"language-plaintext highlighter-rouge\">@ResponseBody<\/code> annotation to indicate thatthe value returned by a handler method needs to be bind to the response body, thus serialized.<\/p><h2 id=\"what-are-the-http-status-return-codes-for-a-successful-get-post-put-or-delete-operation\">What are the HTTP status return codes for a successful GET, POST, PUT or DELETE operation?<\/h2><ul>  <li>200 Ok for a <em>GET<\/em><\/li>  <li>201 Created for a <em>POST<\/em><\/li>  <li>200 Ok or 204 No Content for a <em>PUT<\/em><\/li>  <li>200 Ok, 202 Accepted or 204 No Content for a <em>DELETE<\/em><\/li><\/ul><h2 id=\"when-do-you-need-to-use-responsestatus\">When do you need to use @ResponseStatus?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@ResponseStatus<\/code> annotation is used to specify the response status:<\/p><ul>  <li>when a controller method completes successfully<\/li>  <li>along with <code class=\"language-plaintext highlighter-rouge\">@ExceptionHandler<\/code> and <code class=\"language-plaintext highlighter-rouge\">@ControllerAdvice<\/code> for a method that returns an exception<\/li>  <li>for a specific exception by directly annotating it<\/li><\/ul><h2 id=\"where-do-you-need-to-use-responsebody-what-about-requestbody\">Where do you need to use @ResponseBody? What about @RequestBody?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@RequestBody<\/code> annotation is used to indicate a method parameter should be bound to the bodyof the request.The <code class=\"language-plaintext highlighter-rouge\">@ResponseBody<\/code> annotation indicates the return value should be bound to the web response body.<\/p><h2 id=\"what-spring-boot-starter-would-you-use-for-a-spring-rest-application\">What Spring Boot starter would you use for a Spring REST application?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-web<\/code><\/p><h1 id=\"security\">Security<\/h1><h2 id=\"what-are-authentication-and-authorization-which-must-come-first\">What are authentication and authorization? Which must come first?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">Authentication<\/code> is the process used to check if the credentials provided by the userare valid.<code class=\"language-plaintext highlighter-rouge\">Authorization<\/code> is the process used to check if the authenticated user has the permissionsto access the requested resource.<\/p><h2 id=\"is-security-a-cross-cutting-concern-how-is-it-implemented-internally\">Is security a cross-cutting concern? How is it implemented internally?<\/h2><p>Security is a cross-cutting concern, it is applied across multiple modules of ourapplication.Spring uses <em>AOP<\/em> to implement the security mechanism.<\/p><h2 id=\"what-is-the-delegating-filter-proxy\">What is the delegating filter proxy?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">DelegatingFilterProxy<\/code> acts as an interceptor for secured request, delegating the callsto the chained security filter beans, called <code class=\"language-plaintext highlighter-rouge\">SpringSecurityFilterChain<\/code>.<\/p><h2 id=\"what-is-the-security-filter-chain\">What is the security filter chain?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">SecurityFilterChain<\/code> is a list of filters, each one with a particular responsibility.The filters are added or removed from the chain depending on which services are required.The filters ordering is important as there are dependencies between them.It is used by the <code class=\"language-plaintext highlighter-rouge\">FilterChainProxy<\/code> to match the url with the filters to apply.<\/p><h2 id=\"what-is-a-security-context\">What is a security context?<\/h2><p>The security context is where are stored the details of the current security context, such asthe details of the <em>Principal<\/em> that is using the application.<\/p><h2 id=\"what-does-the--pattern-in-an-antmatcher-or-mvcmatcher-do\">What does the ** pattern in an antMatcher or mvcMatcher do?<\/h2><p>Taking the following pattern example <code class=\"language-plaintext highlighter-rouge\">\/admin\/user\/**<\/code>,they both matches any path that starts with <code class=\"language-plaintext highlighter-rouge\">\/admin\/user\/<\/code>,farther the <code class=\"language-plaintext highlighter-rouge\">mvcMatcher<\/code> also matches the <code class=\"language-plaintext highlighter-rouge\">\/admin\/user\/123.json<\/code> path.<\/p><h2 id=\"why-is-the-usage-of-mvcmatcher-recommended-over-antmatcher\">Why is the usage of mvcMatcher recommended over antMatcher?<\/h2><p><code class=\"language-plaintext highlighter-rouge\">mvcMatcher<\/code> is considered more secure than <code class=\"language-plaintext highlighter-rouge\">antMatcher<\/code>.That is because <code class=\"language-plaintext highlighter-rouge\">mvcMatcher<\/code> use the same rules that <em>Spring MVC<\/em> uses for matching.<\/p><h2 id=\"does-spring-security-support-password-encoding\">Does Spring Security support password encoding?<\/h2><p>Yes, just define a bean of the encoder type you want to use and use it.<\/p><h2 id=\"why-do-you-need-method-security-what-type-of-object-is-typically-secured-at-the-method-level-think-of-its-purpose-not-its-java-type\">Why do you need method security? What type of object is typically secured at the method level (think of its purpose not its Java type).<\/h2><p>To protect sensitive resources, for example the resource that represents the users.<\/p><h2 id=\"what-do-preauthorize-and-rolesallowed-do-what-is-the-difference-between-them\">What do @PreAuthorize and @RolesAllowed do? What is the difference between them?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@PreAuthorize<\/code> annotation is used to run a check before a method invocation to decide ifthe user has the permissions to call the method.I can decide to deny a method execution to users which names start with an m.<br \/>The <code class=\"language-plaintext highlighter-rouge\">@RolesAllowed<\/code> annotation is used to check if the <em>Principal<\/em> role is one of the rolesspecified by the annotation.The <code class=\"language-plaintext highlighter-rouge\">@Secured<\/code> and <code class=\"language-plaintext highlighter-rouge\">@RolesAllowed<\/code> annotations are equivalent, the latter is part of theJSR 250 spec.<\/p><p>To enable the security annotations the <code class=\"language-plaintext highlighter-rouge\">@EnableGlobalMethodSecurity<\/code> annotation must be used.It exposes some properties:<\/p><ul>  <li>the <code class=\"language-plaintext highlighter-rouge\">prePostEnabled<\/code> property enables <em>@pre<\/em>\/<em>@post<\/em> annotations<\/li>  <li>the <code class=\"language-plaintext highlighter-rouge\">securedEnabled<\/code> property determines if the <code class=\"language-plaintext highlighter-rouge\">@Secured<\/code> annotation should be enabled<\/li>  <li>the <code class=\"language-plaintext highlighter-rouge\">jsr250Enabled<\/code> property allows us to use the <code class=\"language-plaintext highlighter-rouge\">@RoleAllowed<\/code> annotation<\/li><\/ul><h2 id=\"how-are-these-annotations-implemented\">How are these annotations implemented?<\/h2><p><em>Spring AOP<\/em>.<\/p><h2 id=\"in-which-security-annotation-are-you-allowed-to-use-spel\">In which security annotation, are you allowed to use SpEL?<\/h2><p>Yes, all the <em>@pre<\/em> and <em>@post<\/em> annotations support <em>SpEL<\/em>:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">@PreAuthorize<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PostAuthorize<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PreFilter<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@PostFilter<\/code><\/li><\/ul><h1 id=\"testing\">Testing<\/h1><h2 id=\"what-type-of-tests-typically-use-spring\">What type of tests typically use Spring?<\/h2><p>Tipically, a <em>Spring<\/em> application has unit and integration tests.<strong>Unit<\/strong> tests are used to test the smallest testable parts of anapplication, isolated from any others units.The dependencies are replaced by stubs and mocks to reproduce the expected behaviour.By default <em>Spring Boot<\/em> uses <em>Junit 5<\/em>, but it supports <em>Junit 4<\/em> too.<strong>Integration<\/strong> tests are used to test the multiple units interactions of anapplication.<\/p><h2 id=\"how-can-you-create-a-shared-application-context-in-a-junit-integration-test\">How can you create a shared application context in a JUnit integration test?<\/h2><p>The so-called <code class=\"language-plaintext highlighter-rouge\">TestContext<\/code> encapsulates the context of each test, and provide context managementand caching support for the test instance.<br \/>By default, the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> is not accessible by the test instances.To expose the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> to the test instances the test class must implementthe <code class=\"language-plaintext highlighter-rouge\">ApplicationContextAware<\/code> interface.As an alternative to implementing the <code class=\"language-plaintext highlighter-rouge\">ApplicationContextAware<\/code> interface, it is possible toinject the application context through the <code class=\"language-plaintext highlighter-rouge\">@Autowired<\/code> annotation.<br \/>Once a <code class=\"language-plaintext highlighter-rouge\">TestContext<\/code> loads an <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code>, that context is cached and reused for allsubsequent tests within the same test suite.<\/p><h2 id=\"when-and-where-do-you-use-transactional-in-testing\">When and where do you use @Transactional in testing?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@Transactional<\/code> annotation can be used on the test class ordirectly on the method.By default a rollback occurs after completition of the test.To change that behaviour you can use the <code class=\"language-plaintext highlighter-rouge\">@Rollback(false)<\/code> annotation orthe <code class=\"language-plaintext highlighter-rouge\">@Commit<\/code> annotation.Also, the <code class=\"language-plaintext highlighter-rouge\">defaultRollback<\/code> property of the <code class=\"language-plaintext highlighter-rouge\">@TransactionConfiguration<\/code> annotationcan be used to change that behaviour.<\/p><h2 id=\"how-are-mock-frameworks-such-as-mockito-or-easymock-used\">How are mock frameworks such as Mockito or EasyMock used?<\/h2><p>To run a test that uses <em>Mockito<\/em> you have to:<\/p><ol>  <li>Declare the mock by using the <code class=\"language-plaintext highlighter-rouge\">@Mock<\/code> annotation<\/li>  <li>Inject the mock in the tested unit by using the <code class=\"language-plaintext highlighter-rouge\">@InjectMocks<\/code> annotation<\/li>  <li>Configure the mock by using the <code class=\"language-plaintext highlighter-rouge\">when<\/code> statement<\/li>  <li>Run test<\/li>  <li>Assert results<\/li><\/ol><p><em>Mockito<\/em> provides a <em>Junit 5<\/em> extension class <code class=\"language-plaintext highlighter-rouge\">MockitoExtension<\/code> that automatically initialize mocks.In alternative, you can use the static method <code class=\"language-plaintext highlighter-rouge\">MockitoAnnotations.initMocks<\/code>.<\/p><p><em>Spring Boot<\/em> simplifies the mocking process by providing the<code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> annotation.It adds the mock to the <em>Spring<\/em> <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code>.Can be used directly on the test class or <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> classes and fields.When used on a field of the test class, the mock is injected into the field.<\/p><h2 id=\"how-is-contextconfiguration-used\">How is @ContextConfiguration used?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code> annotation is used on a test class to specify the components classesto use to configure the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code>.<br \/>A component class is:<\/p><ul>  <li>A class annotated with <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code><\/li>  <li>A component (i.e., a class annotated with @Component, @Service, @Repository, etc.)<\/li>  <li>A <code class=\"language-plaintext highlighter-rouge\">JSR-330<\/code> compliant class that is annotated with <em>javax.inject<\/em> annotations<\/li>  <li>Any class that contains <code class=\"language-plaintext highlighter-rouge\">@Bean<\/code>-methods<\/li>  <li>Any other class that is intended to be registered as a Spring component (i.e., a Spring bean in the ApplicationContext), potentially taking advantage of automatic autowiring of a single constructor without the use of Spring annotations<\/li><\/ul><p>When <code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code> is used without properties it searches for a file named<code class=\"language-plaintext highlighter-rouge\">testClassName-context.xml<\/code> in the same location of the test class.<\/p><h2 id=\"how-does-spring-boot-simplify-writing-tests\">How does Spring Boot simplify writing tests?<\/h2><p><em>Spring Boot<\/em> provides all the libraries needed to run a well-rounded test by justusing the <code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-test<\/code> dependency:<\/p><ul>  <li>JUnit\u2009, the de-facto standard for unit testing Java applications<\/li>  <li>Spring Test &amp; Spring Boot Test, utilities and integration test support for Spring Boot applications<\/li>  <li>AssertJ\u2009, a fluent assertion library<\/li>  <li>Hamcrest\u2009, alibrary of matcher objects (also known as constraints or predicates)<\/li>  <li>Mockito\u2009, a Java mocking framework<\/li>  <li>JSONassert\u2009, an assertion library for JSON<\/li>  <li>JsonPath\u2009, \u2009XPath for JSON<\/li><\/ul><h2 id=\"what-does-springboottest-do-how-does-it-interact-with-springbootapplication-and-springbootconfiguration\">What does @SpringBootTest do? How does it interact with @SpringBootApplication and @SpringBootConfiguration?<\/h2><p><em>Spring Boot<\/em> exposes the <code class=\"language-plaintext highlighter-rouge\">@SpringBootTest<\/code> annotation to configure an <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code>by using the <code class=\"language-plaintext highlighter-rouge\">SpringBootContextLoader<\/code> loader class.<br \/>It provides the following features:<\/p><ul>  <li>automatically searches for any <code class=\"language-plaintext highlighter-rouge\">@SpringBootConfiguration<\/code>-classes when no <code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code> annotation is defined<\/li>  <li>allow to customize the <code class=\"language-plaintext highlighter-rouge\">Environment<\/code> properties to be defined using the <code class=\"language-plaintext highlighter-rouge\">properties<\/code> attribute<\/li>  <li>allow application arguments to be defined using the <code class=\"language-plaintext highlighter-rouge\">args<\/code> attribute<\/li>  <li>support for different webEnvironment modes, including the ability to start a fully running webserver listening on a defined or random port<\/li>  <li>register a <code class=\"language-plaintext highlighter-rouge\">TestRestTemplate<\/code> bean<\/li><\/ul><h1 id=\"spring-boot-basics\">Spring Boot Basics<\/h1><h2 id=\"what-is-spring-boot\">What is Spring Boot?<\/h2><p><em>Spring Boot<\/em> is a way to ease create production-ready, stand-alone Spring-based applications.It provides an opinionated starter configuration for code and annotation, to quickly start new Spring projects with no timewith minimal or zero configurations.<\/p><h2 id=\"what-are-the-advantages-of-using-spring-boot\">What are the advantages of using Spring Boot?<\/h2><ul>  <li>Very easy to develop Spring based applications<\/li>  <li>It reduces development time and increase productivity<\/li>  <li>It avoids writing boilerplate code, annotations and xml configuration<\/li>  <li>Easy integration with Spring ecosystem, such as Spring JDBC, Spring ORM, Spring Data, Security\u2026<\/li>  <li>It reduces developer effort by providing opinionated defaults configuration<\/li>  <li>It provides embedded HTTP servers, Tomcat by default<\/li>  <li>It provides a CLI to develop and test Spring applications<\/li><\/ul><h2 id=\"what-things-affect-what-spring-boot-sets-up\">What things affect what Spring Boot sets up?<\/h2><p><em>Spring Boot<\/em> attempts to automatically configure the Spring application based on whichlibraries are available in the classpath.The auto-configuration is enabled by adding the <code class=\"language-plaintext highlighter-rouge\">@EnableAutoConfiguration<\/code> or <code class=\"language-plaintext highlighter-rouge\">@SpringBootApplication<\/code> annotationsto one of the <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> classes.<\/p><h2 id=\"what-is-a-spring-boot-starter-why-is-it-useful\">What is a Spring Boot starter? Why is it useful?<\/h2><p>A starter is a set of convenient dependency descriptors that can be included in a Spring application.For example, if you want to get started using Spring and JPA for database access, include the<code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-data-jpa<\/code> dependency in the project.All Spring starters follow the naming pattern <code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-*<\/code>.<\/p><h2 id=\"can-you-control-logging-with-spring-boot-how\">Can you control logging with Spring Boot? How?<\/h2><p>The default auto-configured logger in <em>Spring Boot<\/em> is logback. Appropriate logback routing is also included to ensure that dependant libraries that use others providers work correctly.<br \/> Logging can be configured by using some well-known properties in the <code class=\"language-plaintext highlighter-rouge\">logging.*<\/code> group:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">logging.file.name<\/code> or <code class=\"language-plaintext highlighter-rouge\">logging.file.path<\/code> to indicate the file where to log<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logging.logback.rollingpolicy.*<\/code> to configure the file rotation policy<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logging.level.&lt;logger-name&gt;=&lt;level&gt;<\/code>, where level is one of <code class=\"language-plaintext highlighter-rouge\">TRACE<\/code>, <code class=\"language-plaintext highlighter-rouge\">DEBUG<\/code>, <code class=\"language-plaintext highlighter-rouge\">INFO<\/code>,<code class=\"language-plaintext highlighter-rouge\">WARN<\/code>, <code class=\"language-plaintext highlighter-rouge\">ERROR<\/code>, <code class=\"language-plaintext highlighter-rouge\">FATAL<\/code> or <code class=\"language-plaintext highlighter-rouge\">OFF<\/code>, to configure the log level of a specific logger.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logging.group.&lt;group-name&gt;=&lt;loggers&gt;<\/code> to group loggers together.<em>Spring Boot<\/em> provides two pre-defined logging groups, <code class=\"language-plaintext highlighter-rouge\">web<\/code> and <code class=\"language-plaintext highlighter-rouge\">sql<\/code>.<\/li><\/ul><p>Another option to configure logging is to use a dedicated configuration file:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">logback-spring.xml<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logback.xml<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">log4j-spring.xml<\/code> or <code class=\"language-plaintext highlighter-rouge\">log4j.xml<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logging.properties<\/code><\/li><\/ul><h2 id=\"where-does-spring-boot-look-for-applicationproperties-file-by-default\">Where does Spring Boot look for application.properties file by default?<\/h2><p>Ordered by precedence:<\/p><ol>  <li>In the classpath root<\/li>  <li>In the <code class=\"language-plaintext highlighter-rouge\">\/config<\/code> package classpath<\/li>  <li>In the current directory<\/li>  <li>In the <code class=\"language-plaintext highlighter-rouge\">\/config<\/code> subdirectory in the current directory<\/li>  <li>In the immediate child directories of the <code class=\"language-plaintext highlighter-rouge\">\/config<\/code> subdirectories<\/li><\/ol><h2 id=\"how-do-you-define-profile-specific-property-files\">How do you define profile specific property files?<\/h2><p><em>Spring Boot<\/em> attempt to load profile-specific files using the naming convention<code class=\"language-plaintext highlighter-rouge\">application-&lt;profile&gt;<\/code>.For example, if the <code class=\"language-plaintext highlighter-rouge\">dev<\/code> profile is enabled, then both <code class=\"language-plaintext highlighter-rouge\">application.yml<\/code> and <code class=\"language-plaintext highlighter-rouge\">application-dev.yml<\/code>will be considered.Profile specific files always override the non-specific ones.If several profiles are specified, a last-wins strategy applies.If no profiles are activated, the properties from <code class=\"language-plaintext highlighter-rouge\">application-default<\/code> are considered.<\/p><h2 id=\"how-do-you-access-the-properties-defined-in-the-property-files\">How do you access the properties defined in the property files?<\/h2><p>You can access to properties values defined in property files by:<\/p><ul>  <li>retrieving the current <code class=\"language-plaintext highlighter-rouge\">Environment<\/code><\/li>  <li>using the <code class=\"language-plaintext highlighter-rouge\">@Value<\/code> annotation withing the <code class=\"language-plaintext highlighter-rouge\">${&lt;property-name&gt;}<\/code> placeholder<\/li>  <li>using the <code class=\"language-plaintext highlighter-rouge\">@ConfigurationProperties<\/code> annotation<\/li><\/ul><h2 id=\"what-properties-do-you-have-to-define-in-order-to-configure-external-mysql\">What properties do you have to define in order to configure external MySQL?<\/h2><ol>  <li><code class=\"language-plaintext highlighter-rouge\">spring.datasource.url=jdbc:mysql:\/\/mysql-hostname\/db-name<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">spring.datasource.username=hello<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">spring.datasource.password=world<\/code><\/li><\/ol><h2 id=\"how-do-you-configure-default-schema-and-initial-data\">How do you configure default schema and initial data?<\/h2><p><em>Spring Boot<\/em> loads <em>SQL<\/em> from the standard root classpath locations:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">schema.sql<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">data.sql<\/code><\/li><\/ul><p>In addition it processes the files:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">schema-&lt;platform&gt;.sql<\/code>, where <em>platform<\/em> is the value of the <code class=\"language-plaintext highlighter-rouge\">spring.datasource.platform<\/code> property<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">data-&lt;platform&gt;.sql<\/code><\/li><\/ul><p><em>Spring Boot<\/em> automatically creates the schema for embedded data sources.This behaviour can be customized by using the <code class=\"language-plaintext highlighter-rouge\">spring.datasource.initialization-mode<\/code> property.<\/p><h2 id=\"what-is-a-fat-jar-how-is-it-different-from-the-original-jar\">What is a fat jar? How is it different from the original jar?<\/h2><p>A fat jar is a jar that contains all the application dependencies.<\/p><h2 id=\"what-embedded-containers-does-spring-boot-support\">What embedded containers does Spring Boot support?<\/h2><ul>  <li>Tomcat<\/li>  <li>Jetty<\/li>  <li>Undertow<\/li>  <li>Reactor Netty<\/li><\/ul><h1 id=\"spring-boot-auto-configuration\">Spring Boot Auto Configuration<\/h1><h2 id=\"how-does-spring-boot-know-what-to-configure\">How does Spring Boot know what to configure?<\/h2><p><em>Spring<\/em> auto-configuration attempts to automatically configure the <em>Spring<\/em> applicationbased on the jar dependencies that are present in the classpath.<\/p><h2 id=\"what-does-enableautoconfiguration-do\">What does @EnableAutoConfiguration do?<\/h2><p>It enables the <em>Spring<\/em> auto-configuration of the <em>Spring Application Context<\/em>.Auto-configuration classes are usually applied based on the classpath content andwhat beans are available.Auto-configuration classes are Spring <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> beans located by the<code class=\"language-plaintext highlighter-rouge\">SpringFactoriesLoadeer<\/code> mechanism.Those beans are usually <code class=\"language-plaintext highlighter-rouge\">@Conditional<\/code> beans,most often using <code class=\"language-plaintext highlighter-rouge\">@ConditionalOnClass<\/code> and <code class=\"language-plaintext highlighter-rouge\">@ConditionalOnMissingBean<\/code> annotations.<\/p><h2 id=\"what-does-springbootapplication-do\">What does @SpringBootApplication do?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@SpringBootApplication<\/code> annotation is meta-annotated with<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">@SpringBootConfiguration<\/code>, an alias for the <code class=\"language-plaintext highlighter-rouge\">@Configuration<\/code> annotation<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@EnableAutoConfiguration<\/code>, to enable the <em>Spring<\/em> auto-configuration<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@ComponentScan<\/code> to scan the current packages and all its children for component beans<\/li><\/ul><h2 id=\"does-spring-boot-do-component-scanning-where-does-it-look-by-default\">Does Spring Boot do component scanning? Where does it look by default?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@SpringBootApplication<\/code> enable component scanning for the current packageand all its sub-packages.<\/p><h2 id=\"how-are-datasource-and-jdbctemplate-auto-configured\">How are DataSource and JdbcTemplate auto-configured?<\/h2><p><em>Spring Boot<\/em> automatically creates a JdbcTemplate.The <code class=\"language-plaintext highlighter-rouge\">DataSource<\/code> is auto-configured only if in the classpathis present an in-memory database dependency.<\/p><h2 id=\"what-is-springfactories-file-for\">What is spring.factories file for?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">spring.factories<\/code> file is used by <em>Spring Boot<\/em> to locate the auto-configuration candidates andto register <code class=\"language-plaintext highlighter-rouge\">ApplicationListener<\/code>s.<\/p><p>A sample of the <code class=\"language-plaintext highlighter-rouge\">spring.factories<\/code> file is:<\/p><div class=\"language-yaml highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"s\">org.springframework.boot.autoconfigure.EnableAutoConfiguration=\\<\/span><span class=\"s\">com.mycorp.libx.autoconfigure.LibXAutoConfiguration,\\<\/span><span class=\"s\">com.mycorp.libx.autoconfigure.LibXWebAutoConfiguration<\/span><\/code><\/pre><\/div><\/div><h2 id=\"how-do-you-customize-spring-boot-auto-configuration\">How do you customize Spring Boot auto configuration?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@EnableAutoConfiguration<\/code> annotation supports the following properties:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">exclude<\/code>, to specify the list of classes to skip<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">excludeName<\/code>, to specify the list of classes names to skip<\/li><\/ul><p>Alternatively:<\/p><ul>  <li>Use the <code class=\"language-plaintext highlighter-rouge\">spring.autoconfigure.exclude<\/code> property to excludesome auto-configuration classes.<\/li>  <li>Use the <code class=\"language-plaintext highlighter-rouge\">spring.factories<\/code> file to specify the auto-configuration classes<\/li>  <li>Override the beans created by the auto-configuration classes<\/li><\/ul><h2 id=\"what-are-the-examples-of-conditional-annotations-how-are-they-used\">What are the examples of @Conditional annotations? How are they used?<\/h2><p>Conditional annotations are used to skip beans definitions under certain conditions.For example the <code class=\"language-plaintext highlighter-rouge\">JdbcTemplateAutoConfiguration<\/code> is annotated with the following annotations:<\/p><div class=\"language-java highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code> <span class=\"nd\">@ConditionalOnClass<\/span><span class=\"o\">(<\/span><span class=\"n\">value<\/span><span class=\"o\">={<\/span><span class=\"n\">javax<\/span><span class=\"o\">.<\/span><span class=\"na\">sql<\/span><span class=\"o\">.<\/span><span class=\"na\">DataSource<\/span><span class=\"o\">.<\/span><span class=\"na\">class<\/span><span class=\"o\">,<\/span><span class=\"n\">org<\/span><span class=\"o\">.<\/span><span class=\"na\">springframework<\/span><span class=\"o\">.<\/span><span class=\"na\">jdbc<\/span><span class=\"o\">.<\/span><span class=\"na\">core<\/span><span class=\"o\">.<\/span><span class=\"na\">JdbcTemplate<\/span><span class=\"o\">.<\/span><span class=\"na\">class<\/span><span class=\"o\">})<\/span> <span class=\"nd\">@ConditionalOnSingleCandidate<\/span><span class=\"o\">(<\/span><span class=\"n\">value<\/span><span class=\"o\">=<\/span><span class=\"n\">javax<\/span><span class=\"o\">.<\/span><span class=\"na\">sql<\/span><span class=\"o\">.<\/span><span class=\"na\">DataSource<\/span><span class=\"o\">.<\/span><span class=\"na\">class<\/span><span class=\"o\">)<\/span><\/code><\/pre><\/div><\/div><p>Meaning that a <code class=\"language-plaintext highlighter-rouge\">JdbcTemplate<\/code> is auto-configured only if:<\/p><ul>  <li>A <strong>single<\/strong> datasource exists<\/li>  <li>The <code class=\"language-plaintext highlighter-rouge\">JdbcTemplate<\/code> class is on the classpath<\/li><\/ul><h1 id=\"spring-boot-actuator\">Spring Boot Actuator<\/h1><h2 id=\"what-value-does-spring-boot-actuator-provide\">What value does Spring Boot Actuator provide?<\/h2><p>It provides production-ready features to help you monitor and manage the <em>Spring Boot<\/em> application.<\/p><h2 id=\"what-are-the-two-protocols-you-can-use-to-access-actuator-endpoints\">What are the two protocols you can use to access actuator endpoints?<\/h2><ul>  <li>JMX<\/li>  <li>HTTP<\/li><\/ul><h2 id=\"what-are-the-actuator-endpoints-that-are-provided-out-of-the-box\">What are the actuator endpoints that are provided out of the box?<\/h2><p>An actuator endpoint can be enabled and exposed, when both are true then theactuator endpoint is available.<\/p><p>By default all endpoints except for <code class=\"language-plaintext highlighter-rouge\">shutdown<\/code> are enabled.To enable\/disable an actuator use the property pattern <code class=\"language-plaintext highlighter-rouge\">management.endpoint.&lt;name&gt;.enabled<\/code>.<br \/>To disable all endpoints by default use the property <code class=\"language-plaintext highlighter-rouge\">management.endpoints.enabled-by-default=false<\/code>.<\/p><p>All jmx endpoints are exposed out-of-the-box.Instead only few http endpoints are exposed by default:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">health<\/code> endpoint<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">info<\/code> endpoint<\/li><\/ul><p>Additional endpoints are enabled if the application is a web application:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">heapdump<\/code>, returns an <code class=\"language-plaintext highlighter-rouge\">hprof<\/code> heap dump file<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">jolokia<\/code>, when jolokia is in the classpath, it exposes jmx beans over http<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">logfile<\/code>, if <code class=\"language-plaintext highlighter-rouge\">logging.file.name<\/code> or <code class=\"language-plaintext highlighter-rouge\">logging.file.path<\/code> properties have been set, returns the content of the logfile<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">prometheus<\/code>, exposes metrics in a format that can be scraped by <em>Prometheus<\/em><\/li><\/ul><p>To change which endpoints are exposed, use the the following properties:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.jmx.exposure.exclude<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.jmx.exposure.include<\/code>, default value is <code class=\"language-plaintext highlighter-rouge\">*<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.web.exposure.exclude<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.web.exposure.include<\/code>, default value is <code class=\"language-plaintext highlighter-rouge\">info, health<\/code><\/li><\/ul><p>You can customize the actuator web endpoints path by using the following properties<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.web.base-path<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">management.endpoints.web.path-mapping.&lt;id&gt;<\/code><\/li><\/ul><h2 id=\"what-is-info-endpoint-for-how-do-you-supply-data\">What is info endpoint for? How do you supply data?<\/h2><p>The info endpoint exposes arbitrary data.<em>Spring Boot<\/em> by default exposes the following data:<\/p><ul>  <li>any key from the <code class=\"language-plaintext highlighter-rouge\">Environment<\/code> under the <code class=\"language-plaintext highlighter-rouge\">info<\/code> key<\/li>  <li>git information if a <code class=\"language-plaintext highlighter-rouge\">git.properties<\/code> file is available<\/li>  <li>build information if a <code class=\"language-plaintext highlighter-rouge\">META-INF\/build-info.properties<\/code> file is available<\/li><\/ul><p>You can create a component class that implements the <code class=\"language-plaintext highlighter-rouge\">InfoContributor<\/code> interfaceto add additional data to the info endpoint.<\/p><h2 id=\"how-do-you-change-logging-level-of-a-package-using-loggers-endpoint\">How do you change logging level of a package using loggers endpoint?<\/h2><p>By doing a <code class=\"language-plaintext highlighter-rouge\">POST<\/code> to the <code class=\"language-plaintext highlighter-rouge\">\/actuator\/loggers<\/code> endpoint with the following body:<\/p><div class=\"language-json highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"p\">{<\/span><span class=\"w\">  <\/span><span class=\"nl\">\"configuredLevel\"<\/span><span class=\"p\">:<\/span><span class=\"w\"> <\/span><span class=\"s2\">\"DEBUG\"<\/span><span class=\"w\"><\/span><span class=\"p\">}<\/span><span class=\"w\"><\/span><\/code><\/pre><\/div><\/div><p>Pass null to reset the logger level.<\/p><h2 id=\"how-do-you-access-an-endpoint-using-a-tag\">How do you access an endpoint using a tag?<\/h2><h2 id=\"what-is-metrics-for\">What is metrics for?<\/h2><p><em>Spring Boot<\/em> provides a <code class=\"language-plaintext highlighter-rouge\">metrics<\/code> endpoint that can be used to gatherthe metrics collected by the application.<br \/>Navigating to the <code class=\"language-plaintext highlighter-rouge\">\/actuator\/metrics<\/code> display the list of available metrics.Provide the metric name to retrieve the data of a particular metric, e.g. <code class=\"language-plaintext highlighter-rouge\">\/actuator\/metrics\/jvm.memory.max<\/code>.You can use tags to drill down a specific metric data, e.g. <code class=\"language-plaintext highlighter-rouge\">\/actuator\/metrics\/jvm.memory.max?tag=area:nonheap<\/code>.<\/p><h2 id=\"how-do-you-create-a-custom-metric\">How do you create a custom metric?<\/h2><p>Inject the <code class=\"language-plaintext highlighter-rouge\">MeterRegistry<\/code> bean in your component and use it to register a custom metric.<\/p><h2 id=\"what-is-health-indicator\">What is Health Indicator?<\/h2><p>It provides the health status of the current application.<\/p><h2 id=\"what-is-the-health-indicator-status\">What is the Health Indicator status?<\/h2><p>The HTTP status code in the response reflects the overall health status.<\/p><h2 id=\"what-are-the-health-indicator-statuses-that-are-provided-out-of-the-box\">What are the Health Indicator statuses that are provided out of the box?<\/h2><p>By default it supports the following statuses:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">UP<\/code>, no mapping by default<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">UNKNOWN<\/code>, no mapping by default<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">DOWN<\/code>, mapped to the code 503<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">OUT_OF_SERVICE<\/code>, mapped to the code 503<\/li><\/ul><p>Any unmapped health status map to 200.<\/p><h2 id=\"how-do-you-change-the-health-indicator-status-severity-order\">How do you change the Health Indicator status severity order?<\/h2><p>Using the <code class=\"language-plaintext highlighter-rouge\">management.endpoint.health.status.order<\/code> property<\/p><h1 id=\"spring-boot-testing\">Spring Boot Testing<\/h1><h2 id=\"when-do-you-want-to-use-springboottest-annotation\">When do you want to use @SpringBootTest annotation?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@SpringBootTest<\/code> annotation is an alternative to the <code class=\"language-plaintext highlighter-rouge\">spring-test<\/code> <code class=\"language-plaintext highlighter-rouge\">@ContextConfiguration<\/code>annotation when you need <em>Spring Boot<\/em> features.The annotation works by creating the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> used by the tests through <code class=\"language-plaintext highlighter-rouge\">SpringApplication<\/code>.<br \/>By default it will not start a server.You can use the <code class=\"language-plaintext highlighter-rouge\">webEnvironment<\/code> property to configure the test class web environment:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">MOCK<\/code>, the default value, loads a <code class=\"language-plaintext highlighter-rouge\">WebApplicationContext<\/code> and provides amocked web environment. It can be used with the <code class=\"language-plaintext highlighter-rouge\">@AutoConfigureMockMvc<\/code> for mock-based testing of your web application.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">RANDOM_PORT<\/code>, loads a <code class=\"language-plaintext highlighter-rouge\">WebServerApplicationContext<\/code> and provides a real web environment.The embedded server port can be injected by using the <code class=\"language-plaintext highlighter-rouge\">@LocalServerPort<\/code> annotation or by injecting the <code class=\"language-plaintext highlighter-rouge\">local.server.port<\/code>property.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">DEFINED_PORT<\/code>, is like <code class=\"language-plaintext highlighter-rouge\">RANDOM_PORT<\/code> but the listening port is gathered from the <code class=\"language-plaintext highlighter-rouge\">Environment<\/code>.<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">NONE<\/code>, Loads an <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code> but does not provide any web environment.<\/li><\/ul><h2 id=\"what-does-springboottest-auto-configure\">What does @SpringBootTest auto-configure?<\/h2><ul>  <li>It uses the <code class=\"language-plaintext highlighter-rouge\">SpringBootContextLoader<\/code> to confiure the <code class=\"language-plaintext highlighter-rouge\">ApplicationContext<\/code><\/li>  <li>It registers a <code class=\"language-plaintext highlighter-rouge\">TestRestTemplate<\/code> bean for use in web tests that are using a fully running webserver<\/li><\/ul><h2 id=\"what-dependencies-does-spring-boot-starter-test-brings-to-the-classpath\">What dependencies does spring-boot-starter-test brings to the classpath?<\/h2><ul>  <li><em>JUnit<\/em>\u2009, the de-facto standard for unit testing Java applications<\/li>  <li><em>Spring Test<\/em> &amp; <em>Spring Boot Test<\/em>, utilities and integration test support for Spring Boot applications<\/li>  <li><em>AssertJ<\/em>\u2009, a fluent assertion library<\/li>  <li><em>Hamcrest<\/em>\u2009, a library of matcher objects<\/li>  <li><em>Mockito\u2009<\/em>, a Java mocking framework<\/li>  <li><em>JSONassert<\/em>\u2009, an assertion library for JSON<\/li>  <li><em>JsonPath<\/em>\u2009,\u2009XPath for JSON<\/li><\/ul><h2 id=\"how-do-you-perform-integration-testing-with-springboottest-for-a-web-application\">How do you perform integration testing with @SpringBootTest for a web application?<\/h2><ul>  <li>Use <code class=\"language-plaintext highlighter-rouge\">@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)<\/code> to start the embedded server<\/li>  <li>Use <code class=\"language-plaintext highlighter-rouge\">@LocalServerPort<\/code> annotation on a field to inject the listening port<\/li>  <li>Use the <code class=\"language-plaintext highlighter-rouge\">@TestRestTemplate<\/code> to perform requests against the webserver<\/li>  <li>Assert the results<\/li><\/ul><h2 id=\"when-do-you-want-to-use-webmvctest-what-does-it-auto-configure\">When do you want to use @WebMvcTest? What does it auto-configure?<\/h2><p>The <code class=\"language-plaintext highlighter-rouge\">@WebMvcTest<\/code> annotation is used for tests that focus <strong>only<\/strong> on <em>Spring MVC<\/em> components.Using this annotation the auto-configuration is disabled and only <em>MVC<\/em> related configurationsare configured.Tests annotated with <code class=\"language-plaintext highlighter-rouge\">@WebMvcTest<\/code> annotation will also auto-configure <em>Spring Security<\/em> and <code class=\"language-plaintext highlighter-rouge\">MockMvc<\/code>.Use the <code class=\"language-plaintext highlighter-rouge\">@WithMockUser<\/code> annotation on a test method to mock the <code class=\"language-plaintext highlighter-rouge\">Principal<\/code>used by the <code class=\"language-plaintext highlighter-rouge\">mockMvc<\/code> requests.The <code class=\"language-plaintext highlighter-rouge\">@WithMockUser<\/code> annotation supports the following properties:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">username<\/code>, to set the principal username<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">roles<\/code>, to set the principal roles<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">password<\/code>, to set the principal password<\/li><\/ul><h2 id=\"what-are-the-differences-between-mockbean-and-mock\">What are the differences between @MockBean and @Mock?<\/h2><ul>  <li><code class=\"language-plaintext highlighter-rouge\">@Mock<\/code> can only be applied to fields and parameters<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> can only be applied to classes and fields<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@Mock<\/code> can be used to mock java classes or interfaces<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> only allows for  mocking <em>Spring<\/em> beans or creation of mock <em>Spring<\/em> beans<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> can be used only with a running <code class=\"language-plaintext highlighter-rouge\">SpringRunner<\/code> or <code class=\"language-plaintext highlighter-rouge\">SpringExtension<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> can be used for meta-annotate other beans<\/li>  <li><code class=\"language-plaintext highlighter-rouge\">@MockBean<\/code> and <code class=\"language-plaintext highlighter-rouge\">@Mock<\/code> are both included in <code class=\"language-plaintext highlighter-rouge\">spring-boot-starter-test<\/code><\/li><\/ul><h2 id=\"when-do-you-want-datajpatest-for-what-does-it-auto-configure\">When do you want @DataJpaTest for? What does it auto-configure?<\/h2><p>Like <code class=\"language-plaintext highlighter-rouge\">@WebMvcTest<\/code> it disables the full auto-configuration, and apply only configurationsrelevant to JPA.By default tests annotated with <code class=\"language-plaintext highlighter-rouge\">@DataJpaTest<\/code> are transactional and roll back at the end of each test execution.It forces the use of an embedded in-memory database.<\/p>","url":"https:\/\/manuelmazzuola.dev\/2021\/04\/26\/spring-certification-study-notes","tags":["spring","certification","java"],"date_published":"2021-04-26T00:00:00+00:00","date_modified":"2021-04-26T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2021\/04\/26\/i-got-spring-certification","title":"I got the Spring Professional Certification","summary":"I got the Spring Professional Certification","content_text":"https:\/\/www.credly.com\/badges\/2683a45a-35e1-47d4-8964-3e2779f9ca47","content_html":"<p><a href=\"https:\/\/www.credly.com\/badges\/2683a45a-35e1-47d4-8964-3e2779f9ca47\">https:\/\/www.credly.com\/badges\/2683a45a-35e1-47d4-8964-3e2779f9ca47<\/a><\/p>","url":"https:\/\/manuelmazzuola.dev\/2021\/04\/26\/i-got-spring-certification","tags":["spring","certification","java"],"date_published":"2021-04-26T00:00:00+00:00","date_modified":"2021-04-26T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2021\/03\/28\/deploy-helm-chart-go","title":"Deploy Helm Charts on a Kubernetes cluster in Go","summary":"How to programmatically deploy Helm Charts on a Kubernetes cluster using Go language","content_text":"  What is Helm  The Helm Go SDK  Sync the Helm repository  Authenticate to the K8S API server  The install intent  Locate and load the chart  Install the chart  ConclusionsWhat is HelmThe basic idea of Helm is to facilitate the deployment and manage complexapplication workloads in Kubernetes.Helm is kinda like a package manager for Kubernetes, and the packages are so-called Helm Charts.A Helm Chart is essentially the combination of a template file, that describes the Kubernetesresources to deploy, and a values file used to valorize the former.The Helm Chart templating system enables the reusability of the packages.The Helm Go SDKHelm comes with a handy CLI, a powerful command-line client for end-users, responsiblefor managing all the lifecycle phases of a Chart.But, to programmatically install Helm Charts on a K8S cluster the CLI is not the solutionif the intention is to build a robust and predictable application, luckily, the Helm developersdesigned the CLI as an interface for the Go SDKthat I\u2019ll reuse for my purpose.Sync the Helm repositoryHelm supports Charts storage in remote repositories, so the first thing to dois to gather from the repository the information about the available charts.I skip logging, error checking, and methods definitions to keep this blog post briefand concise.Let\u2019s start by downloading the repository index file.import (  \"helm.sh\/helm\/v3\/pkg\/repo\")\/\/ repoEntry contains the name and the url of the repository\/\/ settings contains the path to the directory where to store the repository index filer, err := repo.NewChartRepository(repoEntry, getter.All(settings))_, err = r.DownloadIndexFile()Then I fetch the information about the charts stored in the repositoryand cache the results into a dedicated folder.import (  \"helm.sh\/helm\/v3\/pkg\/repo\")var f repo.Filef.Update(repoEntry)err := f.WriteFile(CacheFolderPath, 0644)The before pieces of code replicate the Helm CLI commands:  helm repo add repo_name repo_url  helm repo updateAuthenticate to the K8S API serverThe Helm Go SDK allows building an authenticated client instance by using the path of thekubeconfigfile on the filesystem.In the following code, the kube.GetConfig method returns an implementationof the RESTClientGetter interface, used to obtain a REST client configurationbased on the provided kubeconfig path.The REST client configuration contains everything is needed to dialogwith the Kubernetes server API.restClientGetter, err := kube.GetConfig(KUBECONFIG_PATH_ON_FS)actionConfig := new(action.Configuration)err = actionConfig.Init(restClientGetter, ...)However, I can\u2019t leverage that method because I don\u2019t want to persist on the filesystemthe Kubernetes credentials I got from the outside.After having done some research and opened an issueon Github (I had to first search for similar issues, my bad), I finally ended upwriting my implementation of the RESTClientGetter interface,restclient.go.Then I build a RESTClientGetter by calling the NewRESTClientGetter methodpassing to it the kubeconfig content directly.import (  \"helm.sh\/helm\/v3\/pkg\/action\")restClientGetter, err := NewRESTClientGetter(kubeconfigContent, namespace)actionConfig := new(action.Configuration)err = actionConfig.Init(restClientGetter, ...)Due to an issue with the Helm SDK, I override thenamespacevalue in my rest client builder.The install command, which you\u2019ll see later, wrongly uses the default namespace,so I fixed it by forcing the namespace to the value passed in the NewRESTClientGetter method.The install intentAfter that, I instantiate a new Install action and configure it by populatingsome of its attributes: the namespace where to create the resources and the release name.import (  \"helm.sh\/helm\/v3\/pkg\/action\")client := action.NewInstall(actionConfig)client.Namespace = namespaceclient.ReleaseName = releaseNameLocate and load the chartThe next step is to locate from the repository synced before the chart to deploy and load it.import (  \"helm.sh\/helm\/v3\/pkg\/chart\/loader\")cp, err := client.ChartPathOptions.LocateChart(chart, settings)chartReq, err := loader.Load(cp)Install the chartOnce the chart is loaded, I can execute the deployment by calling the Run method andpassing to it the requested chart.err := client.Run(chartReq, nil)All the above steps are required to imitate the Helm CLI commandhelm install name --namespace namespace chart.ConclusionsI hope I\u2019ve given you an idea of \u200b\u200bhow to use the Helm Go SDK to installHelm Charts on a Kubernetes cluster.More can be done, if the deployed Chart exposes a web service,you can use the K8S Go SDK to create a load balancer Service type to reachthe application from the internet.","content_html":"<ul>  <li><a href=\"#what-is-helm\">What is Helm<\/a><\/li>  <li><a href=\"#the-helm-go-sdk\">The Helm Go SDK<\/a><\/li>  <li><a href=\"#sync-the-helm-repository\">Sync the Helm repository<\/a><\/li>  <li><a href=\"#authenticate-to-the-k8s-api-server\">Authenticate to the K8S API server<\/a><\/li>  <li><a href=\"#the-install-intent\">The install intent<\/a><\/li>  <li><a href=\"#locate-and-load-the-chart\">Locate and load the chart<\/a><\/li>  <li><a href=\"#install-the-chart\">Install the chart<\/a><\/li>  <li><a href=\"#conclusions\">Conclusions<\/a><\/li><\/ul><h1 id=\"what-is-helm\">What is Helm<\/h1><p>The basic idea of <a href=\"https:\/\/helm.sh\/\">Helm<\/a> is to facilitate the deployment and manage complexapplication workloads in <a href=\"https:\/\/kubernetes.io\/\">Kubernetes<\/a>.<\/p><p>Helm is kinda like a package manager for Kubernetes, and the packages are so-called Helm Charts.A Helm Chart is essentially the combination of a template file, that describes the Kubernetesresources to deploy, and a values file used to valorize the former.The Helm Chart templating system enables the reusability of the packages.<\/p><h1 id=\"the-helm-go-sdk\">The Helm Go SDK<\/h1><p>Helm comes with a handy CLI, a powerful command-line client for end-users, responsiblefor managing all the lifecycle phases of a Chart.<br \/>But, to programmatically install Helm Charts on a K8S cluster the CLI is not the solutionif the intention is to build a robust and predictable application, luckily, the Helm developersdesigned the CLI as an interface for the <a href=\"https:\/\/pkg.go.dev\/helm.sh\/helm\/v3\">Go SDK<\/a>that I\u2019ll reuse for my purpose.<\/p><h1 id=\"sync-the-helm-repository\">Sync the Helm repository<\/h1><p>Helm supports Charts storage in remote repositories, so the first thing to dois to gather from the repository the information about the available charts.<\/p><p>I skip logging, error checking, and methods definitions to keep this blog post briefand concise.<\/p><p>Let\u2019s start by downloading the repository index file.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>  <span class=\"s\">\"helm.sh\/helm\/v3\/pkg\/repo\"<\/span><span class=\"p\">)<\/span><span class=\"c\">\/\/ repoEntry contains the name and the url of the repository<\/span><span class=\"c\">\/\/ settings contains the path to the directory where to store the repository index file<\/span><span class=\"n\">r<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">repo<\/span><span class=\"o\">.<\/span><span class=\"n\">NewChartRepository<\/span><span class=\"p\">(<\/span><span class=\"n\">repoEntry<\/span><span class=\"p\">,<\/span> <span class=\"n\">getter<\/span><span class=\"o\">.<\/span><span class=\"n\">All<\/span><span class=\"p\">(<\/span><span class=\"n\">settings<\/span><span class=\"p\">))<\/span><span class=\"n\">_<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">r<\/span><span class=\"o\">.<\/span><span class=\"n\">DownloadIndexFile<\/span><span class=\"p\">()<\/span><\/code><\/pre><\/div><\/div><p>Then I fetch the information about the charts stored in the repositoryand cache the results into a dedicated folder.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>  <span class=\"s\">\"helm.sh\/helm\/v3\/pkg\/repo\"<\/span><span class=\"p\">)<\/span><span class=\"k\">var<\/span> <span class=\"n\">f<\/span> <span class=\"n\">repo<\/span><span class=\"o\">.<\/span><span class=\"n\">File<\/span><span class=\"n\">f<\/span><span class=\"o\">.<\/span><span class=\"n\">Update<\/span><span class=\"p\">(<\/span><span class=\"n\">repoEntry<\/span><span class=\"p\">)<\/span><span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">f<\/span><span class=\"o\">.<\/span><span class=\"n\">WriteFile<\/span><span class=\"p\">(<\/span><span class=\"n\">CacheFolderPath<\/span><span class=\"p\">,<\/span> <span class=\"m\">0644<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>The before pieces of code replicate the Helm CLI commands:<\/p><ul>  <li><code class=\"language-plaintext highlighter-rouge\">helm repo add repo_name repo_url<\/code><\/li>  <li><code class=\"language-plaintext highlighter-rouge\">helm repo update<\/code><\/li><\/ul><h1 id=\"authenticate-to-the-k8s-api-server\">Authenticate to the K8S API server<\/h1><p>The Helm Go SDK allows building an authenticated client instance by using the path of the<a href=\"https:\/\/kubernetes.io\/docs\/concepts\/configuration\/organize-cluster-access-kubeconfig\/\">kubeconfig<\/a>file on the filesystem.<\/p><p>In the following code, the <code class=\"language-plaintext highlighter-rouge\">kube.GetConfig<\/code> method returns an implementationof the <code class=\"language-plaintext highlighter-rouge\">RESTClientGetter<\/code> interface, used to obtain a REST client configurationbased on the provided kubeconfig path.<br \/>The REST client configuration contains everything is needed to dialogwith the Kubernetes server API.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">restClientGetter<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">kube<\/span><span class=\"o\">.<\/span><span class=\"n\">GetConfig<\/span><span class=\"p\">(<\/span><span class=\"n\">KUBECONFIG_PATH_ON_FS<\/span><span class=\"p\">)<\/span><span class=\"n\">actionConfig<\/span> <span class=\"o\">:=<\/span> <span class=\"nb\">new<\/span><span class=\"p\">(<\/span><span class=\"n\">action<\/span><span class=\"o\">.<\/span><span class=\"n\">Configuration<\/span><span class=\"p\">)<\/span><span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">actionConfig<\/span><span class=\"o\">.<\/span><span class=\"n\">Init<\/span><span class=\"p\">(<\/span><span class=\"n\">restClientGetter<\/span><span class=\"p\">,<\/span> <span class=\"o\">...<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>However, I can\u2019t leverage that method because I don\u2019t want to persist on the filesystemthe Kubernetes credentials I got from the outside.<\/p><p>After having done some research and opened an <a href=\"https:\/\/github.com\/helm\/helm\/issues\/9473\">issue<\/a>on Github (I had to first search for similar issues, my bad), I finally ended upwriting my implementation of the <code class=\"language-plaintext highlighter-rouge\">RESTClientGetter<\/code> interface,<a href=\"https:\/\/gist.github.com\/manuelmazzuola\/943db0cda93dfc435912968e7cf34126\">restclient.go<\/a>.<\/p><p>Then I build a <code class=\"language-plaintext highlighter-rouge\">RESTClientGetter<\/code> by calling the <code class=\"language-plaintext highlighter-rouge\">NewRESTClientGetter<\/code> methodpassing to it the <strong>kubeconfig content<\/strong> directly.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>  <span class=\"s\">\"helm.sh\/helm\/v3\/pkg\/action\"<\/span><span class=\"p\">)<\/span><span class=\"n\">restClientGetter<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">NewRESTClientGetter<\/span><span class=\"p\">(<\/span><span class=\"n\">kubeconfigContent<\/span><span class=\"p\">,<\/span> <span class=\"n\">namespace<\/span><span class=\"p\">)<\/span><span class=\"n\">actionConfig<\/span> <span class=\"o\">:=<\/span> <span class=\"nb\">new<\/span><span class=\"p\">(<\/span><span class=\"n\">action<\/span><span class=\"o\">.<\/span><span class=\"n\">Configuration<\/span><span class=\"p\">)<\/span><span class=\"n\">err<\/span> <span class=\"o\">=<\/span> <span class=\"n\">actionConfig<\/span><span class=\"o\">.<\/span><span class=\"n\">Init<\/span><span class=\"p\">(<\/span><span class=\"n\">restClientGetter<\/span><span class=\"p\">,<\/span> <span class=\"o\">...<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>Due to an <a href=\"https:\/\/github.com\/helm\/helm\/issues\/9256\">issue<\/a> with the Helm SDK, I override the<a href=\"https:\/\/kubernetes.io\/docs\/concepts\/overview\/working-with-objects\/namespaces\/\">namespace<\/a>value in my rest client builder.The install command, which you\u2019ll see later, wrongly uses the <code class=\"language-plaintext highlighter-rouge\">default<\/code> namespace,so I fixed it by forcing the namespace to the value passed in the <code class=\"language-plaintext highlighter-rouge\">NewRESTClientGetter<\/code> method.<\/p><h1 id=\"the-install-intent\">The install intent<\/h1><p>After that, I instantiate a new Install action and configure it by populatingsome of its attributes: the namespace where to create the resources and the release name.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>  <span class=\"s\">\"helm.sh\/helm\/v3\/pkg\/action\"<\/span><span class=\"p\">)<\/span><span class=\"n\">client<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">action<\/span><span class=\"o\">.<\/span><span class=\"n\">NewInstall<\/span><span class=\"p\">(<\/span><span class=\"n\">actionConfig<\/span><span class=\"p\">)<\/span><span class=\"n\">client<\/span><span class=\"o\">.<\/span><span class=\"n\">Namespace<\/span> <span class=\"o\">=<\/span> <span class=\"n\">namespace<\/span><span class=\"n\">client<\/span><span class=\"o\">.<\/span><span class=\"n\">ReleaseName<\/span> <span class=\"o\">=<\/span> <span class=\"n\">releaseName<\/span><\/code><\/pre><\/div><\/div><h1 id=\"locate-and-load-the-chart\">Locate and load the chart<\/h1><p>The next step is to locate from the repository synced before the chart to deploy and load it.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">import<\/span> <span class=\"p\">(<\/span>  <span class=\"s\">\"helm.sh\/helm\/v3\/pkg\/chart\/loader\"<\/span><span class=\"p\">)<\/span><span class=\"n\">cp<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">client<\/span><span class=\"o\">.<\/span><span class=\"n\">ChartPathOptions<\/span><span class=\"o\">.<\/span><span class=\"n\">LocateChart<\/span><span class=\"p\">(<\/span><span class=\"n\">chart<\/span><span class=\"p\">,<\/span> <span class=\"n\">settings<\/span><span class=\"p\">)<\/span><span class=\"n\">chartReq<\/span><span class=\"p\">,<\/span> <span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">loader<\/span><span class=\"o\">.<\/span><span class=\"n\">Load<\/span><span class=\"p\">(<\/span><span class=\"n\">cp<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><h1 id=\"install-the-chart\">Install the chart<\/h1><p>Once the chart is loaded, I can execute the deployment by calling the Run method andpassing to it the requested chart.<\/p><div class=\"language-go highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">err<\/span> <span class=\"o\">:=<\/span> <span class=\"n\">client<\/span><span class=\"o\">.<\/span><span class=\"n\">Run<\/span><span class=\"p\">(<\/span><span class=\"n\">chartReq<\/span><span class=\"p\">,<\/span> <span class=\"no\">nil<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>All the above steps are required to imitate the Helm CLI command<code class=\"language-plaintext highlighter-rouge\">helm install name --namespace namespace chart<\/code>.<\/p><h1 id=\"conclusions\">Conclusions<\/h1><p>I hope I\u2019ve given you an idea of \u200b\u200bhow to use the Helm Go SDK to installHelm Charts on a Kubernetes cluster.<br \/>More can be done, if the deployed Chart exposes a web service,you can use the K8S Go SDK to create a load balancer Service type to reachthe application from the internet.<\/p>","url":"https:\/\/manuelmazzuola.dev\/2021\/03\/28\/deploy-helm-chart-go","tags":["kubernetes","k8s","helm","golang","go","helm-chart"],"date_published":"2021-03-28T00:00:00+00:00","date_modified":"2021-03-28T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2021\/03\/09\/softmax-temperature-nan-loss","title":"Use temperature in softmax function to avoid NaN loss","summary":"How to use temperature in softmax function with cross-entropy loss to avoid NaN","content_text":"Table of Contents  Table of Contents  Introduction  Softmax temperature  PyTorch exampleIntroductionThe softmax function isn\u2019t supposed to output zeros or ones, but sometimesit happens due to floating-point precision when the input vector containsnumbers too big or too small for the exponential inside the softmax.  Exponential growth seems slow at the beginning, but it becomes stepper ina very short time.For example, \\(e^{-114}\\approx3.1\\cdot10^{-50}\\)is essentially evaluated as zero.When softmax is used with cross-entropy loss function, a zeroin the former\u2019s output becomes \u00b1\\(\\infin\\) as a result of thelogarithm in latter, which is theoretically correct since the adjustmentsto make the network adapt are infinite, but it is of no use in practiceas the resulting loss could be NaN.A zero or a one in the softmax output means that the model is veryconfident about the prediction, therefore the solution is to decrease thatconfidence to produce a softer probability distribution.Softmax temperature\\[q_i = \\dfrac{e^{(z_i\/T)}}{\\sum_j{e^{(z_j\/T)}}}\\]T is the temperature parameter, usually set to 1.Increasing the temperature parameter will penalize bigger \\(z_i\\) valuesmore than the smaller \\(z_i\\) values, owing to the amplification effectof the exponential, which leads to a decrease in the confidence of the model.T = 1exp(-8\/T) ~ 0.0003exp(8\/T)  ~ 2981exp(3\/T)  ~ 20T = 1.2exp(-8\/T) ~ 0.01exp(8\/T)  ~ 786exp(3\/T)  ~ 3In % terms, the bigger the exponent is, the more it shrinks when atemperature &gt;1 is applied, which implies that the softmax function willassign more probability mass to the smaller samples.  Beware! A high temperature makes the NN \u201ceasily excited\u201d,resulting in more mistakes.PyTorch exampleLet\u2019s write the softmax function and the cross-entropy loss function.def softmax(input, t=1.0):  print(\"input\", input)  ex = torch.exp(input\/t)  print(\"exp\", ex)  sum = torch.sum(ex, axis=0)  return ex \/ sumdef cross_entropy(distribution):  target = torch.tensor([0, 0, 1, 0, 0])  print(\"loss\", -torch.sum(target * torch.log(distribution)))Define a sample containing some large absolute values and applythe softmax function, then the cross-entropy loss.input = torch.tensor([55.8906, -114.5621, 6.3440, -30.2473, -44.1440])cross_entropy(softmax(input))input tensor([  55.8906, -114.5621,    6.3440,  -30.2473,  -44.1440])exp tensor([1.8749e+24, 0.0000e+00, 5.6907e+02, 7.3074e-14, 6.7376e-20])loss tensor(nan)The resulting probability distribution contains a zero, the loss value is NaN.Let\u2019s see what happens by setting the temperature to 10.input = torch.tensor([55.8906, -114.5621, 6.3440, -30.2473, -44.1440])cross_entropy(softmax(input, t=10))input tensor([  55.8906, -114.5621,    6.3440,  -30.2473,  -44.1440])exp tensor([2.6748e+02, 1.0584e-05, 1.8859e+00, 4.8571e-02, 1.2102e-02])loss tensor(4.9619)The probability is more equally distributed, the softmax function has assignedmore probability mass to the smallest sample, from 0 to 1.0584e-05, and lessprobability mass to the largest sample, from 1.8749e+24 to 2.6748e+02.Finally, the loss has changed from NaN to a valid value.","content_html":"<h1 id=\"table-of-contents\">Table of Contents<\/h1><ul>  <li><a href=\"#table-of-contents\">Table of Contents<\/a><\/li>  <li><a href=\"#introduction\">Introduction<\/a><\/li>  <li><a href=\"#softmax-temperature\">Softmax temperature<\/a><\/li>  <li><a href=\"#pytorch-example\">PyTorch example<\/a><\/li><\/ul><h1 id=\"introduction\">Introduction<\/h1><p>The <code class=\"language-plaintext highlighter-rouge\">softmax<\/code> function isn\u2019t supposed to output zeros or ones, but sometimesit happens due to floating-point precision when the input vector containsnumbers too big or too small for the exponential inside the softmax.<\/p><blockquote>  <p>Exponential growth seems slow at the beginning, but it becomes stepper ina very short time.<\/p><\/blockquote><p>For example, \\(e^{-114}\\approx3.1\\cdot10^{-50}\\)is essentially evaluated as zero.<br \/>When softmax is used with cross-entropy loss function, a zeroin the former\u2019s output becomes \u00b1\\(\\infin\\) as a result of thelogarithm in latter, which is theoretically correct since the adjustmentsto make the network adapt are infinite, but it is of no use in practiceas the resulting loss could be <a href=\"\/assets\/images\/computer-says-nan.jpg\">NaN<\/a>.<\/p><p>A zero or a one in the softmax output means that the model is veryconfident about the prediction, therefore <strong>the solution is to decrease thatconfidence to produce a <em>softer<\/em> probability distribution<\/strong>.<\/p><h1 id=\"softmax-temperature\">Softmax temperature<\/h1>\\[q_i = \\dfrac{e^{(z_i\/T)}}{\\sum_j{e^{(z_j\/T)}}}\\]<p>T is the temperature parameter, usually set to 1.<br \/>Increasing the temperature parameter will penalize bigger \\(z_i\\) valuesmore than the smaller \\(z_i\\) values, owing to the amplification effectof the exponential, <strong>which leads to a decrease in the confidence of the model<\/strong>.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">T<\/span> <span class=\"o\">=<\/span> <span class=\"mi\">1<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"o\">-<\/span><span class=\"mi\">8<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"o\">~<\/span> <span class=\"mf\">0.0003<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"mi\">8<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span>  <span class=\"o\">~<\/span> <span class=\"mi\">2981<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"mi\">3<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span>  <span class=\"o\">~<\/span> <span class=\"mi\">20<\/span><span class=\"n\">T<\/span> <span class=\"o\">=<\/span> <span class=\"mf\">1.2<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"o\">-<\/span><span class=\"mi\">8<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span> <span class=\"o\">~<\/span> <span class=\"mf\">0.01<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"mi\">8<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span>  <span class=\"o\">~<\/span> <span class=\"mi\">786<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"mi\">3<\/span><span class=\"o\">\/<\/span><span class=\"n\">T<\/span><span class=\"p\">)<\/span>  <span class=\"o\">~<\/span> <span class=\"mi\">3<\/span><\/code><\/pre><\/div><\/div><p>In % terms, the bigger the exponent is, the more it shrinks when atemperature <code class=\"language-plaintext highlighter-rouge\">&gt;1<\/code> is applied, which implies that the softmax function willassign more probability mass to the smaller samples.<\/p><blockquote>  <p>Beware! A high temperature makes the NN \u201ceasily excited\u201d,resulting in more mistakes.<\/p><\/blockquote><h1 id=\"pytorch-example\">PyTorch example<\/h1><p>Let\u2019s write the softmax function and the cross-entropy loss function.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">def<\/span> <span class=\"nf\">softmax<\/span><span class=\"p\">(<\/span><span class=\"nb\">input<\/span><span class=\"p\">,<\/span> <span class=\"n\">t<\/span><span class=\"o\">=<\/span><span class=\"mf\">1.0<\/span><span class=\"p\">):<\/span>  <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"input\"<\/span><span class=\"p\">,<\/span> <span class=\"nb\">input<\/span><span class=\"p\">)<\/span>  <span class=\"n\">ex<\/span> <span class=\"o\">=<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"n\">exp<\/span><span class=\"p\">(<\/span><span class=\"nb\">input<\/span><span class=\"o\">\/<\/span><span class=\"n\">t<\/span><span class=\"p\">)<\/span>  <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"exp\"<\/span><span class=\"p\">,<\/span> <span class=\"n\">ex<\/span><span class=\"p\">)<\/span>  <span class=\"nb\">sum<\/span> <span class=\"o\">=<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">ex<\/span><span class=\"p\">,<\/span> <span class=\"n\">axis<\/span><span class=\"o\">=<\/span><span class=\"mi\">0<\/span><span class=\"p\">)<\/span>  <span class=\"k\">return<\/span> <span class=\"n\">ex<\/span> <span class=\"o\">\/<\/span> <span class=\"nb\">sum<\/span><span class=\"k\">def<\/span> <span class=\"nf\">cross_entropy<\/span><span class=\"p\">(<\/span><span class=\"n\">distribution<\/span><span class=\"p\">):<\/span>  <span class=\"n\">target<\/span> <span class=\"o\">=<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"n\">tensor<\/span><span class=\"p\">([<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">1<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"mi\">0<\/span><span class=\"p\">])<\/span>  <span class=\"k\">print<\/span><span class=\"p\">(<\/span><span class=\"s\">\"loss\"<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"nb\">sum<\/span><span class=\"p\">(<\/span><span class=\"n\">target<\/span> <span class=\"o\">*<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"n\">log<\/span><span class=\"p\">(<\/span><span class=\"n\">distribution<\/span><span class=\"p\">)))<\/span><\/code><\/pre><\/div><\/div><p>Define a sample containing some large absolute values and applythe softmax function, then the cross-entropy loss.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nb\">input<\/span> <span class=\"o\">=<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"n\">tensor<\/span><span class=\"p\">([<\/span><span class=\"mf\">55.8906<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">114.5621<\/span><span class=\"p\">,<\/span> <span class=\"mf\">6.3440<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">30.2473<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">44.1440<\/span><span class=\"p\">])<\/span><span class=\"n\">cross_entropy<\/span><span class=\"p\">(<\/span><span class=\"n\">softmax<\/span><span class=\"p\">(<\/span><span class=\"nb\">input<\/span><span class=\"p\">))<\/span><\/code><\/pre><\/div><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>input tensor([  55.8906, -114.5621,    6.3440,  -30.2473,  -44.1440])exp tensor([1.8749e+24, 0.0000e+00, 5.6907e+02, 7.3074e-14, 6.7376e-20])loss tensor(nan)<\/code><\/pre><\/div><\/div><p>The resulting probability distribution contains a zero, the loss value is NaN.<br \/>Let\u2019s see what happens by setting the temperature to 10.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"nb\">input<\/span> <span class=\"o\">=<\/span> <span class=\"n\">torch<\/span><span class=\"p\">.<\/span><span class=\"n\">tensor<\/span><span class=\"p\">([<\/span><span class=\"mf\">55.8906<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">114.5621<\/span><span class=\"p\">,<\/span> <span class=\"mf\">6.3440<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">30.2473<\/span><span class=\"p\">,<\/span> <span class=\"o\">-<\/span><span class=\"mf\">44.1440<\/span><span class=\"p\">])<\/span><span class=\"n\">cross_entropy<\/span><span class=\"p\">(<\/span><span class=\"n\">softmax<\/span><span class=\"p\">(<\/span><span class=\"nb\">input<\/span><span class=\"p\">,<\/span> <span class=\"n\">t<\/span><span class=\"o\">=<\/span><span class=\"mi\">10<\/span><span class=\"p\">))<\/span><\/code><\/pre><\/div><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>input tensor([  55.8906, -114.5621,    6.3440,  -30.2473,  -44.1440])exp tensor([2.6748e+02, 1.0584e-05, 1.8859e+00, 4.8571e-02, 1.2102e-02])loss tensor(4.9619)<\/code><\/pre><\/div><\/div><p>The probability is more equally distributed, the softmax function has assignedmore probability mass to the smallest sample, <strong>from 0 to 1.0584e-05<\/strong>, and lessprobability mass to the largest sample, from 1.8749e+24 to 2.6748e+02.<br \/>Finally, <strong>the loss has changed from NaN to a valid value<\/strong>.<\/p>","url":"https:\/\/manuelmazzuola.dev\/2021\/03\/09\/softmax-temperature-nan-loss","tags":["deep learning","neural network","softmax","cross-entropy"],"date_published":"2021-03-09T00:00:00+00:00","date_modified":"2021-03-09T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}},{"id":"https:\/\/manuelmazzuola.dev\/2021\/02\/17\/my-first-neural-network","title":"My first neural network classifier, statue or painting?","summary":"How I trained a neural network to recognize paintings and statues","content_text":"Recently I decided to learn deep learning and the math behind the training process and I\u2019d like to summarizewhat I\u2019ve learned so far by writing a simple but efficient classifier, a neural network capable totell if an image contains a painting or a statue.The code uses FastAI deep learning library, mainly because of their motto FASTAI: Making neural nets uncool again and I used Colab, a cloud Python editor with GPU support, to write the code.Table of Contents  Table of Contents  Prepare the dataset  Install the libraries  Configure the datasets  Train the architecture into a model  Testing the neural network  ConclusionsPrepare the datasetLast week I went to the Uffizi Museum and I toke some pictures ofpaintings and statues so I decided to use them as my training data.I removed the blurred photos and split them into two folders, named paint and statue, the folder name is very important because the training code uses it to label the content.Install the librariesThe first lines of code are for installing libraries and importing the necessary modules.!pip install fastai==2.2.5!pip install exiffrom fastai.data.all import *from fastai.vision.all import *from exif import Image as ExifImageAn issue I had is that fastai by default doesn\u2019t rotate the images according to their EXIF metadata so I searched and found a piece of code to rotate the photos in the correct orientation before feeding the neural network with them.def exif_type_tfms(fn, cls, **kwargs):       def get_orientation(fn: (Path, str)):    with open(fn, 'rb') as image_file:      exif_img = ExifImage(image_file)      try:        return exif_img.orientation.value      except AttributeError:        # ignore orientation unset        return 1  def f(img, rotate=0, transpose=False):    img = img.rotate(rotate, expand=True)    if transpose:      img = img.transpose(Image.FLIP_LEFT_RIGHT)    return img  # Image.rotate will do shorcuts on these magic angles, so no need for any   # specific resampling strategy  trafo_fns = {    1: partial(f, rotate=0),    6: partial(f, rotate=270),    8: partial(f, rotate=90),    3: partial(f, rotate=180),    2: partial(f, rotate=0, transpose=True),    5: partial(f, rotate=270, transpose=True),    7: partial(f, rotate=90, transpose=True),    4: partial(f, rotate=180, transpose=True),  }  img = cls.create(fn, **kwargs)  orientation = get_orientation(fn)  img = trafo_fns[orientation](img)  return cls(img)def ExifImageBlock(cls=PILImage):  \"\"\"  if images are rotated with the EXIF orientation flag  it must be respected when loading the images  ExifImageBlock can be pickled (which is important to dump learners)  &gt;&gt;&gt; pickle.dumps(ExifImageBlock())  b'...  \"\"\"  return TransformBlock(type_tfms=partial(exif_type_tfms, cls=cls), batch_tfms=IntToFloatTensor)Configure the datasetsdatablock = DataBlock(    get_items=get_image_files, # how to gather images    item_tfms=Resize(224), # how to transform images before use them    blocks=(ExifImageBlock, CategoryBlock), # (input transforms, label transforms)    get_y=parent_label, # how to label the images    splitter=RandomSplitter()) # how to get the training datasetA DataBlock represents a list of instructions used by the network to gather the dataset, label it, transform it, and how to split it into a training dataset and a validation dataset.The get_image_files function is a utility that grabs all the files in a folder, recursively.The Resize function resizes the image to 224px x 224px.Why 224px? It\u2019s a standard size, you can use the value you\u2019d like but remember that a larger image requires more resources to be processed.The blocks attribute is a Tuple that contains the block of the x, our input dataset, and the block of the y, our labels.A block represents a set of transformations to apply.The ExifImageBlock will rotate the images according to the EXIF metadata, and the CategoryBlock indicates that images are labeled with only two categories, in my case statue or paint.The parent_label function receives the image path as input and returns the name of the containing folder, my label.So if the photo of a statue is inside the statue folder, the neural network will learn that that photo is a statue.The RandomSplitter() without any parameters will create a validation dataset using 20% of the input dataset.dls = datablock.dataloaders('\/content\/drive\/MyDrive\/uffizi', bs=5)dls.show_batch()The bs attribute is the batch size, how many training examples the neural network will process at a time.The larger the batch size the more resources you need, the smaller the batch size the more updates you need.The show_batch method shows the first batch that the neural network will process.All the images are labeled with their parent folder name.Train the architecture into a modellearn = cnn_learner(dls, resnet34, metrics=error_rate)learn.fine_tune(4)Here we use a convolutional neural network, inspired by biological processes, is the state-of-the-art in image recognition, NPL, and bioinformatics.resnet34 is a Residual Network, a model pre-trained with the ImageNet dataset, it consists of 34 layers.The fine_tune(4) call will train the neural network with our dataset 4 times, each time is called epoch.For each epoch, it tests the goodness of the model using the validation datasetby calculating the loss value.Looking at the output of the training process we notice an issue, from the 2nd epoch to the 4th epoch the loss is increasing meaning that the model is getting worse, the learning process is not working correctly, and the reason for that is the batch size and how it interferes with the SGD (Stochastic Gradient Descent), that is not completely clear to me why it happens but I\u2019ll investigate more and I\u2019ll write a post about it.            epoch      train_loss      valid_loss      error_rate      time                  0      0.774661      0.121791      0.071429      00:38                  epoch      train_loss      valid_loss      error_rate      time                  0      0.187633      0.038042      0.000000      00:49              1      0.109573      0.007019      0.000000      00:49              2      0.356412      0.003473      0.000000      00:49              3      0.373253      0.004066      0.000000      00:50      Let\u2019s increase the batch size to 10 and re-launch the training.datablock = DataBlock(    get_items = get_image_files,    item_tfms=Resize(224),    blocks=(ExifImageBlock, CategoryBlock),    get_y=parent_label,    splitter=RandomSplitter())dls = datablock.dataloaders('\/content\/drive\/MyDrive\/uffizi', bs=10)dls.show_batch()learn = cnn_learner(dls, resnet34, metrics=error_rate)learn.fine_tune(4)With a batch size value of 10 the loss decreases for each epoch as we expected,our network is learning!!!            epoch      train_loss      valid_loss      error_rate      time                  0      0.852532      0.558232      0.214286      00:37                  epoch      train_loss      valid_loss      error_rate      time                  0      0.359626      0.068505      0.000000      00:46              1      0.293746      0.003416      0.000000      00:46              2      0.260396      0.004584      0.000000      00:46              3      0.192967      0.005092      0.000000      00:46      A confusion matrix is helpful to understand where our neural network is wrong, we can print it using the following code.interp = ClassificationInterpretation.from_learner(learn)interp.plot_confusion_matrix()Nothing wrong so far \ud83d\ude80Testing the neural networkNow we create an image object using a painting that the model has never seen.Using an image that the network has never seen is very important, you can\u2019t test the network with the images contained in your input dataset, it\u2019s cheating if you do that.paint = PILImage.create('\/content\/dechirico1.jpg')paint = PILImage(paint.resize((430, 224)))paintlearn.predict(paint)('paint', tensor(0), tensor([0.9841, 0.0159]))Hurray!!! The network has confidently classified the image as a painting,it\u2019s 98.41% sure about that.Let\u2019s try with another painting that contains a statue \ud83d\ude44paint = PILImage.create('\/content\/dechirico2.jpg')paint = PILImage(paint.resize((224, 224)))paintlearn.predict(paint)('paint', tensor(0), tensor([0.9972, 0.0028]))It\u2019s 99% sure about paint category \ud83c\udf89And finally lady liberty.paint = PILImage.create('\/content\/ladyliberty.jpg')paint = PILImage(paint.resize((224, 224)))paintlearn.predict(paint)('statue', tensor(1), tensor([0.0070, 0.9930]))It\u2019s 99% sure that it is a statue \ud83d\uddfdConclusionsI am amazed how with so few images it\u2019s possible to create a neural network to classify images that have never been seen by the training process.One of the most important things to get a good result is to have a good training dataset and a good dataset for validation, the organization of the input dataset is crucial to have a good model capable of making predictions.Deep learning is not just for image recognition, it can be applied to almost all fields, including natural language processing, bioinformatics, recommendation systems, etc.I\u2019m still just scratching the surface of deep learning, there are so many things to learn, I\u2019ll continue to study it and to write about it.","content_html":"<p>Recently I decided to learn deep learning and the math behind the training process and I\u2019d like to summarizewhat I\u2019ve learned so far by writing a simple but efficient classifier, a neural network capable totell if an image contains a painting or a statue.<\/p><p>The code uses <a href=\"https:\/\/github.com\/fastai\/fastai\">FastAI<\/a> deep learning library, mainly because of their motto <code class=\"language-plaintext highlighter-rouge\">FASTAI: Making neural nets uncool again<\/code> and I used <a href=\"https:\/\/colab.research.google.com\/\">Colab<\/a>, a cloud Python editor with GPU support, to write the code.<\/p><h1 id=\"table-of-contents\">Table of Contents<\/h1><ul>  <li><a href=\"#table-of-contents\">Table of Contents<\/a><\/li>  <li><a href=\"#prepare-the-dataset\">Prepare the dataset<\/a><\/li>  <li><a href=\"#install-the-libraries\">Install the libraries<\/a><\/li>  <li><a href=\"#configure-the-datasets\">Configure the datasets<\/a><\/li>  <li><a href=\"#train-the-architecture-into-a-model\">Train the architecture into a model<\/a><\/li>  <li><a href=\"#testing-the-neural-network\">Testing the neural network<\/a><\/li>  <li><a href=\"#conclusions\">Conclusions<\/a><\/li><\/ul><h1 id=\"prepare-the-dataset\">Prepare the dataset<\/h1><p>Last week I went to the <a href=\"https:\/\/www.uffizi.it\/\">Uffizi Museum<\/a> and I toke some pictures ofpaintings and statues so I decided to use them as my training data.<\/p><p>I removed the blurred photos and split them into two folders, named <code class=\"language-plaintext highlighter-rouge\">paint<\/code> and <code class=\"language-plaintext highlighter-rouge\">statue<\/code>, the folder name is very important because the training code uses it to label the content.<\/p><h1 id=\"install-the-libraries\">Install the libraries<\/h1><p>The first lines of code are for installing libraries and importing the necessary modules.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"err\">!<\/span><span class=\"n\">pip<\/span> <span class=\"n\">install<\/span> <span class=\"n\">fastai<\/span><span class=\"o\">==<\/span><span class=\"mf\">2.2<\/span><span class=\"p\">.<\/span><span class=\"mi\">5<\/span><span class=\"err\">!<\/span><span class=\"n\">pip<\/span> <span class=\"n\">install<\/span> <span class=\"n\">exif<\/span><span class=\"kn\">from<\/span> <span class=\"nn\">fastai.data.all<\/span> <span class=\"kn\">import<\/span> <span class=\"o\">*<\/span><span class=\"kn\">from<\/span> <span class=\"nn\">fastai.vision.all<\/span> <span class=\"kn\">import<\/span> <span class=\"o\">*<\/span><span class=\"kn\">from<\/span> <span class=\"nn\">exif<\/span> <span class=\"kn\">import<\/span> <span class=\"n\">Image<\/span> <span class=\"k\">as<\/span> <span class=\"n\">ExifImage<\/span><\/code><\/pre><\/div><\/div><p>An issue I had is that <code class=\"language-plaintext highlighter-rouge\">fastai<\/code> by default doesn\u2019t rotate the images according to their <code class=\"language-plaintext highlighter-rouge\">EXIF<\/code> metadata so I searched and found a piece of code to rotate the photos in the correct orientation before feeding the neural network with them.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"k\">def<\/span> <span class=\"nf\">exif_type_tfms<\/span><span class=\"p\">(<\/span><span class=\"n\">fn<\/span><span class=\"p\">,<\/span> <span class=\"n\">cls<\/span><span class=\"p\">,<\/span> <span class=\"o\">**<\/span><span class=\"n\">kwargs<\/span><span class=\"p\">):<\/span>       <span class=\"k\">def<\/span> <span class=\"nf\">get_orientation<\/span><span class=\"p\">(<\/span><span class=\"n\">fn<\/span><span class=\"p\">:<\/span> <span class=\"p\">(<\/span><span class=\"n\">Path<\/span><span class=\"p\">,<\/span> <span class=\"nb\">str<\/span><span class=\"p\">)):<\/span>    <span class=\"k\">with<\/span> <span class=\"nb\">open<\/span><span class=\"p\">(<\/span><span class=\"n\">fn<\/span><span class=\"p\">,<\/span> <span class=\"s\">'rb'<\/span><span class=\"p\">)<\/span> <span class=\"k\">as<\/span> <span class=\"n\">image_file<\/span><span class=\"p\">:<\/span>      <span class=\"n\">exif_img<\/span> <span class=\"o\">=<\/span> <span class=\"n\">ExifImage<\/span><span class=\"p\">(<\/span><span class=\"n\">image_file<\/span><span class=\"p\">)<\/span>      <span class=\"k\">try<\/span><span class=\"p\">:<\/span>        <span class=\"k\">return<\/span> <span class=\"n\">exif_img<\/span><span class=\"p\">.<\/span><span class=\"n\">orientation<\/span><span class=\"p\">.<\/span><span class=\"n\">value<\/span>      <span class=\"k\">except<\/span> <span class=\"nb\">AttributeError<\/span><span class=\"p\">:<\/span>        <span class=\"c1\"># ignore orientation unset<\/span>        <span class=\"k\">return<\/span> <span class=\"mi\">1<\/span>  <span class=\"k\">def<\/span> <span class=\"nf\">f<\/span><span class=\"p\">(<\/span><span class=\"n\">img<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"n\">transpose<\/span><span class=\"o\">=<\/span><span class=\"bp\">False<\/span><span class=\"p\">):<\/span>    <span class=\"n\">img<\/span> <span class=\"o\">=<\/span> <span class=\"n\">img<\/span><span class=\"p\">.<\/span><span class=\"n\">rotate<\/span><span class=\"p\">(<\/span><span class=\"n\">rotate<\/span><span class=\"p\">,<\/span> <span class=\"n\">expand<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">)<\/span>    <span class=\"k\">if<\/span> <span class=\"n\">transpose<\/span><span class=\"p\">:<\/span>      <span class=\"n\">img<\/span> <span class=\"o\">=<\/span> <span class=\"n\">img<\/span><span class=\"p\">.<\/span><span class=\"n\">transpose<\/span><span class=\"p\">(<\/span><span class=\"n\">Image<\/span><span class=\"p\">.<\/span><span class=\"n\">FLIP_LEFT_RIGHT<\/span><span class=\"p\">)<\/span>    <span class=\"k\">return<\/span> <span class=\"n\">img<\/span>  <span class=\"c1\"># Image.rotate will do shorcuts on these magic angles, so no need for any <\/span>  <span class=\"c1\"># specific resampling strategy<\/span>  <span class=\"n\">trafo_fns<\/span> <span class=\"o\">=<\/span> <span class=\"p\">{<\/span>    <span class=\"mi\">1<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">0<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">6<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">270<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">8<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">90<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">3<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">180<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">2<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">0<\/span><span class=\"p\">,<\/span> <span class=\"n\">transpose<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">5<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">270<\/span><span class=\"p\">,<\/span> <span class=\"n\">transpose<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">7<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">90<\/span><span class=\"p\">,<\/span> <span class=\"n\">transpose<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">),<\/span>    <span class=\"mi\">4<\/span><span class=\"p\">:<\/span> <span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">f<\/span><span class=\"p\">,<\/span> <span class=\"n\">rotate<\/span><span class=\"o\">=<\/span><span class=\"mi\">180<\/span><span class=\"p\">,<\/span> <span class=\"n\">transpose<\/span><span class=\"o\">=<\/span><span class=\"bp\">True<\/span><span class=\"p\">),<\/span>  <span class=\"p\">}<\/span>  <span class=\"n\">img<\/span> <span class=\"o\">=<\/span> <span class=\"n\">cls<\/span><span class=\"p\">.<\/span><span class=\"n\">create<\/span><span class=\"p\">(<\/span><span class=\"n\">fn<\/span><span class=\"p\">,<\/span> <span class=\"o\">**<\/span><span class=\"n\">kwargs<\/span><span class=\"p\">)<\/span>  <span class=\"n\">orientation<\/span> <span class=\"o\">=<\/span> <span class=\"n\">get_orientation<\/span><span class=\"p\">(<\/span><span class=\"n\">fn<\/span><span class=\"p\">)<\/span>  <span class=\"n\">img<\/span> <span class=\"o\">=<\/span> <span class=\"n\">trafo_fns<\/span><span class=\"p\">[<\/span><span class=\"n\">orientation<\/span><span class=\"p\">](<\/span><span class=\"n\">img<\/span><span class=\"p\">)<\/span>  <span class=\"k\">return<\/span> <span class=\"n\">cls<\/span><span class=\"p\">(<\/span><span class=\"n\">img<\/span><span class=\"p\">)<\/span><span class=\"k\">def<\/span> <span class=\"nf\">ExifImageBlock<\/span><span class=\"p\">(<\/span><span class=\"n\">cls<\/span><span class=\"o\">=<\/span><span class=\"n\">PILImage<\/span><span class=\"p\">):<\/span>  <span class=\"s\">\"\"\"  if images are rotated with the EXIF orientation flag  it must be respected when loading the images  ExifImageBlock can be pickled (which is important to dump learners)  &gt;&gt;&gt; pickle.dumps(ExifImageBlock())  b'...  \"\"\"<\/span>  <span class=\"k\">return<\/span> <span class=\"n\">TransformBlock<\/span><span class=\"p\">(<\/span><span class=\"n\">type_tfms<\/span><span class=\"o\">=<\/span><span class=\"n\">partial<\/span><span class=\"p\">(<\/span><span class=\"n\">exif_type_tfms<\/span><span class=\"p\">,<\/span> <span class=\"n\">cls<\/span><span class=\"o\">=<\/span><span class=\"n\">cls<\/span><span class=\"p\">),<\/span> <span class=\"n\">batch_tfms<\/span><span class=\"o\">=<\/span><span class=\"n\">IntToFloatTensor<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><h1 id=\"configure-the-datasets\">Configure the datasets<\/h1><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">datablock<\/span> <span class=\"o\">=<\/span> <span class=\"n\">DataBlock<\/span><span class=\"p\">(<\/span>    <span class=\"n\">get_items<\/span><span class=\"o\">=<\/span><span class=\"n\">get_image_files<\/span><span class=\"p\">,<\/span> <span class=\"c1\"># how to gather images<\/span>    <span class=\"n\">item_tfms<\/span><span class=\"o\">=<\/span><span class=\"n\">Resize<\/span><span class=\"p\">(<\/span><span class=\"mi\">224<\/span><span class=\"p\">),<\/span> <span class=\"c1\"># how to transform images before use them<\/span>    <span class=\"n\">blocks<\/span><span class=\"o\">=<\/span><span class=\"p\">(<\/span><span class=\"n\">ExifImageBlock<\/span><span class=\"p\">,<\/span> <span class=\"n\">CategoryBlock<\/span><span class=\"p\">),<\/span> <span class=\"c1\"># (input transforms, label transforms)<\/span>    <span class=\"n\">get_y<\/span><span class=\"o\">=<\/span><span class=\"n\">parent_label<\/span><span class=\"p\">,<\/span> <span class=\"c1\"># how to label the images<\/span>    <span class=\"n\">splitter<\/span><span class=\"o\">=<\/span><span class=\"n\">RandomSplitter<\/span><span class=\"p\">())<\/span> <span class=\"c1\"># how to get the training dataset<\/span><\/code><\/pre><\/div><\/div><p>A DataBlock represents a list of instructions used by the network to gather the dataset, label it, transform it, and how to split it into a training dataset and a validation dataset.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">get_image_files<\/code> function is a utility that grabs all the files in a folder, recursively.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">Resize<\/code> function resizes the image to 224px x 224px.Why 224px? It\u2019s a standard size, you can use the value you\u2019d like but remember that a larger image requires more resources to be processed.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">blocks<\/code> attribute is a Tuple that contains the <code class=\"language-plaintext highlighter-rouge\">block<\/code> of the x, our input dataset, and the <code class=\"language-plaintext highlighter-rouge\">block<\/code> of the y, our labels.<br \/>A <code class=\"language-plaintext highlighter-rouge\">block<\/code> represents a set of transformations to apply.<br \/>The <code class=\"language-plaintext highlighter-rouge\">ExifImageBlock<\/code> will rotate the images according to the <code class=\"language-plaintext highlighter-rouge\">EXIF<\/code> metadata, and the <code class=\"language-plaintext highlighter-rouge\">CategoryBlock<\/code> indicates that images are labeled with only two categories, in my case <code class=\"language-plaintext highlighter-rouge\">statue<\/code> or <code class=\"language-plaintext highlighter-rouge\">paint<\/code>.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">parent_label<\/code> function receives the image path as input and returns the name of the containing folder, my label.<br \/>So if the photo of a statue is inside the <code class=\"language-plaintext highlighter-rouge\">statue<\/code> folder, the neural network will learn that that photo is a <code class=\"language-plaintext highlighter-rouge\">statue<\/code>.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">RandomSplitter()<\/code> without any parameters will create a validation dataset using 20% of the input dataset.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">dls<\/span> <span class=\"o\">=<\/span> <span class=\"n\">datablock<\/span><span class=\"p\">.<\/span><span class=\"n\">dataloaders<\/span><span class=\"p\">(<\/span><span class=\"s\">'\/content\/drive\/MyDrive\/uffizi'<\/span><span class=\"p\">,<\/span> <span class=\"n\">bs<\/span><span class=\"o\">=<\/span><span class=\"mi\">5<\/span><span class=\"p\">)<\/span><span class=\"n\">dls<\/span><span class=\"p\">.<\/span><span class=\"n\">show_batch<\/span><span class=\"p\">()<\/span><\/code><\/pre><\/div><\/div><p>The <code class=\"language-plaintext highlighter-rouge\">bs<\/code> attribute is the batch size, how many training examples the neural network will process at a time.The larger the batch size the more resources you need, the smaller the batch size the more updates you need.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">show_batch<\/code> method shows the first batch that the neural network will process.<\/p><p>All the images are labeled with their parent folder name.<\/p><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_2_0.png\" \/><\/div><h1 id=\"train-the-architecture-into-a-model\">Train the architecture into a model<\/h1><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">learn<\/span> <span class=\"o\">=<\/span> <span class=\"n\">cnn_learner<\/span><span class=\"p\">(<\/span><span class=\"n\">dls<\/span><span class=\"p\">,<\/span> <span class=\"n\">resnet34<\/span><span class=\"p\">,<\/span> <span class=\"n\">metrics<\/span><span class=\"o\">=<\/span><span class=\"n\">error_rate<\/span><span class=\"p\">)<\/span><span class=\"n\">learn<\/span><span class=\"p\">.<\/span><span class=\"n\">fine_tune<\/span><span class=\"p\">(<\/span><span class=\"mi\">4<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>Here we use a convolutional neural network, inspired by biological processes, is the state-of-the-art in image recognition, NPL, and bioinformatics.<\/p><p><code class=\"language-plaintext highlighter-rouge\">resnet34<\/code> is a Residual Network, a model pre-trained with the <a href=\"http:\/\/image-net.org\/challenges\/LSVRC\/2013\/\">ImageNet<\/a> dataset, it consists of 34 layers.<\/p><p>The <code class=\"language-plaintext highlighter-rouge\">fine_tune(4)<\/code> call will train the neural network with our dataset 4 times, each time is called <code class=\"language-plaintext highlighter-rouge\">epoch<\/code>.<br \/>For each epoch, it tests the goodness of the model using the validation datasetby calculating the loss value.<\/p><p>Looking at the output of the training process we notice an issue, from the 2nd epoch to the 4th epoch the loss is increasing meaning that the model is getting worse, the learning process is not working correctly, and the reason for that is the <code class=\"language-plaintext highlighter-rouge\">batch size<\/code> and how it interferes with the <code class=\"language-plaintext highlighter-rouge\">SGD<\/code> (Stochastic Gradient Descent), that is not completely clear to me why it happens but I\u2019ll investigate more and I\u2019ll write a post about it.<\/p><table border=\"1\" class=\"dataframe\">  <thead>    <tr style=\"text-align: left;\">      <th>epoch<\/th>      <th>train_loss<\/th>      <th>valid_loss<\/th>      <th>error_rate<\/th>      <th>time<\/th>    <\/tr>  <\/thead>  <tbody>    <tr>      <td>0<\/td>      <td>0.774661<\/td>      <td>0.121791<\/td>      <td>0.071429<\/td>      <td>00:38<\/td>    <\/tr>  <\/tbody><\/table><table border=\"1\" class=\"dataframe\">  <thead>    <tr style=\"text-align: left;\">      <th>epoch<\/th>      <th>train_loss<\/th>      <th>valid_loss<\/th>      <th>error_rate<\/th>      <th>time<\/th>    <\/tr>  <\/thead>  <tbody>    <tr>      <td>0<\/td>      <td>0.187633<\/td>      <td>0.038042<\/td>      <td>0.000000<\/td>      <td>00:49<\/td>    <\/tr>    <tr>      <td>1<\/td>      <td>0.109573<\/td>      <td>0.007019<\/td>      <td>0.000000<\/td>      <td>00:49<\/td>    <\/tr>    <tr>      <td>2<\/td>      <td>0.356412<\/td>      <td>0.003473<\/td>      <td>0.000000<\/td>      <td>00:49<\/td>    <\/tr>    <tr>      <td>3<\/td>      <td>0.373253<\/td>      <td>0.004066<\/td>      <td>0.000000<\/td>      <td>00:50<\/td>    <\/tr>  <\/tbody><\/table><p>Let\u2019s increase the batch size to 10 and re-launch the training.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">datablock<\/span> <span class=\"o\">=<\/span> <span class=\"n\">DataBlock<\/span><span class=\"p\">(<\/span>    <span class=\"n\">get_items<\/span> <span class=\"o\">=<\/span> <span class=\"n\">get_image_files<\/span><span class=\"p\">,<\/span>    <span class=\"n\">item_tfms<\/span><span class=\"o\">=<\/span><span class=\"n\">Resize<\/span><span class=\"p\">(<\/span><span class=\"mi\">224<\/span><span class=\"p\">),<\/span>    <span class=\"n\">blocks<\/span><span class=\"o\">=<\/span><span class=\"p\">(<\/span><span class=\"n\">ExifImageBlock<\/span><span class=\"p\">,<\/span> <span class=\"n\">CategoryBlock<\/span><span class=\"p\">),<\/span>    <span class=\"n\">get_y<\/span><span class=\"o\">=<\/span><span class=\"n\">parent_label<\/span><span class=\"p\">,<\/span>    <span class=\"n\">splitter<\/span><span class=\"o\">=<\/span><span class=\"n\">RandomSplitter<\/span><span class=\"p\">())<\/span><span class=\"n\">dls<\/span> <span class=\"o\">=<\/span> <span class=\"n\">datablock<\/span><span class=\"p\">.<\/span><span class=\"n\">dataloaders<\/span><span class=\"p\">(<\/span><span class=\"s\">'\/content\/drive\/MyDrive\/uffizi'<\/span><span class=\"p\">,<\/span> <span class=\"n\">bs<\/span><span class=\"o\">=<\/span><span class=\"mi\">10<\/span><span class=\"p\">)<\/span><span class=\"n\">dls<\/span><span class=\"p\">.<\/span><span class=\"n\">show_batch<\/span><span class=\"p\">()<\/span><\/code><\/pre><\/div><\/div><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_4_0.png\" \/><\/div><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">learn<\/span> <span class=\"o\">=<\/span> <span class=\"n\">cnn_learner<\/span><span class=\"p\">(<\/span><span class=\"n\">dls<\/span><span class=\"p\">,<\/span> <span class=\"n\">resnet34<\/span><span class=\"p\">,<\/span> <span class=\"n\">metrics<\/span><span class=\"o\">=<\/span><span class=\"n\">error_rate<\/span><span class=\"p\">)<\/span><span class=\"n\">learn<\/span><span class=\"p\">.<\/span><span class=\"n\">fine_tune<\/span><span class=\"p\">(<\/span><span class=\"mi\">4<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><p>With a batch size value of 10 the loss decreases for each epoch as we expected,our network is learning!!!<\/p><table border=\"1\" class=\"dataframe\">  <thead>    <tr style=\"text-align: left;\">      <th>epoch<\/th>      <th>train_loss<\/th>      <th>valid_loss<\/th>      <th>error_rate<\/th>      <th>time<\/th>    <\/tr>  <\/thead>  <tbody>    <tr>      <td>0<\/td>      <td>0.852532<\/td>      <td>0.558232<\/td>      <td>0.214286<\/td>      <td>00:37<\/td>    <\/tr>  <\/tbody><\/table><table border=\"1\" class=\"dataframe\">  <thead>    <tr style=\"text-align: left;\">      <th>epoch<\/th>      <th>train_loss<\/th>      <th>valid_loss<\/th>      <th>error_rate<\/th>      <th>time<\/th>    <\/tr>  <\/thead>  <tbody>    <tr>      <td>0<\/td>      <td>0.359626<\/td>      <td>0.068505<\/td>      <td>0.000000<\/td>      <td>00:46<\/td>    <\/tr>    <tr>      <td>1<\/td>      <td>0.293746<\/td>      <td>0.003416<\/td>      <td>0.000000<\/td>      <td>00:46<\/td>    <\/tr>    <tr>      <td>2<\/td>      <td>0.260396<\/td>      <td>0.004584<\/td>      <td>0.000000<\/td>      <td>00:46<\/td>    <\/tr>    <tr>      <td>3<\/td>      <td>0.192967<\/td>      <td>0.005092<\/td>      <td>0.000000<\/td>      <td>00:46<\/td>    <\/tr>  <\/tbody><\/table><p>A confusion matrix is helpful to understand where our neural network is wrong, we can print it using the following code.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">interp<\/span> <span class=\"o\">=<\/span> <span class=\"n\">ClassificationInterpretation<\/span><span class=\"p\">.<\/span><span class=\"n\">from_learner<\/span><span class=\"p\">(<\/span><span class=\"n\">learn<\/span><span class=\"p\">)<\/span><span class=\"n\">interp<\/span><span class=\"p\">.<\/span><span class=\"n\">plot_confusion_matrix<\/span><span class=\"p\">()<\/span><\/code><\/pre><\/div><\/div><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_6_1.png\" \/><\/div><p>Nothing wrong so far \ud83d\ude80<\/p><h1 id=\"testing-the-neural-network\">Testing the neural network<\/h1><p>Now we create an image object using a painting that the model has never seen.<\/p><p>Using an image that the network has never seen is very important, you can\u2019t test the network with the images contained in your input dataset, it\u2019s cheating if you do that.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">.<\/span><span class=\"n\">create<\/span><span class=\"p\">(<\/span><span class=\"s\">'\/content\/dechirico1.jpg'<\/span><span class=\"p\">)<\/span><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">(<\/span><span class=\"n\">paint<\/span><span class=\"p\">.<\/span><span class=\"n\">resize<\/span><span class=\"p\">((<\/span><span class=\"mi\">430<\/span><span class=\"p\">,<\/span> <span class=\"mi\">224<\/span><span class=\"p\">)))<\/span><span class=\"n\">paint<\/span><\/code><\/pre><\/div><\/div><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_7_0.png\" \/><\/div><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">learn<\/span><span class=\"p\">.<\/span><span class=\"n\">predict<\/span><span class=\"p\">(<\/span><span class=\"n\">paint<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>('paint', tensor(0), tensor([0.9841, 0.0159]))<\/code><\/pre><\/div><\/div><p>Hurray!!! The network has confidently classified the image as a painting,it\u2019s 98.41% sure about that.Let\u2019s try with another painting that contains a statue \ud83d\ude44<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">.<\/span><span class=\"n\">create<\/span><span class=\"p\">(<\/span><span class=\"s\">'\/content\/dechirico2.jpg'<\/span><span class=\"p\">)<\/span><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">(<\/span><span class=\"n\">paint<\/span><span class=\"p\">.<\/span><span class=\"n\">resize<\/span><span class=\"p\">((<\/span><span class=\"mi\">224<\/span><span class=\"p\">,<\/span> <span class=\"mi\">224<\/span><span class=\"p\">)))<\/span><span class=\"n\">paint<\/span><\/code><\/pre><\/div><\/div><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_9_0.png\" \/><\/div><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">learn<\/span><span class=\"p\">.<\/span><span class=\"n\">predict<\/span><span class=\"p\">(<\/span><span class=\"n\">paint<\/span><span class=\"p\">)<\/span><\/code><\/pre><\/div><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>('paint', tensor(0), tensor([0.9972, 0.0028]))<\/code><\/pre><\/div><\/div><p>It\u2019s 99% sure about paint category \ud83c\udf89<br \/>And finally lady liberty.<\/p><div class=\"language-python highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">.<\/span><span class=\"n\">create<\/span><span class=\"p\">(<\/span><span class=\"s\">'\/content\/ladyliberty.jpg'<\/span><span class=\"p\">)<\/span><span class=\"n\">paint<\/span> <span class=\"o\">=<\/span> <span class=\"n\">PILImage<\/span><span class=\"p\">(<\/span><span class=\"n\">paint<\/span><span class=\"p\">.<\/span><span class=\"n\">resize<\/span><span class=\"p\">((<\/span><span class=\"mi\">224<\/span><span class=\"p\">,<\/span> <span class=\"mi\">224<\/span><span class=\"p\">)))<\/span><span class=\"n\">paint<\/span><\/code><\/pre><\/div><\/div><div class=\"image\"><img src=\"\/assets\/images\/uffizi%20%281%29_11_0.png\" \/><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>learn.predict(paint)<\/code><\/pre><\/div><\/div><div class=\"language-plaintext highlighter-rouge\"><div class=\"highlight\"><pre class=\"highlight\"><code>('statue', tensor(1), tensor([0.0070, 0.9930]))<\/code><\/pre><\/div><\/div><p>It\u2019s 99% sure that it is a statue \ud83d\uddfd<\/p><h1 id=\"conclusions\">Conclusions<\/h1><p>I am amazed how with so few images it\u2019s possible to create a neural network to classify images that have never been seen by the training process.<\/p><p>One of the most important things to get a good result is to have a good training dataset and a good dataset for validation, the organization of the input dataset is crucial to have a good model capable of making predictions.<\/p><p>Deep learning is not just for image recognition, it can be applied to almost all fields, including natural language processing, bioinformatics, recommendation systems, etc.<br \/>I\u2019m still just scratching the surface of deep learning, there are so many things to learn, I\u2019ll continue to study it and to write about it.<\/p>","url":"https:\/\/manuelmazzuola.dev\/2021\/02\/17\/my-first-neural-network","tags":["deep learning","neural network","classifier","colab","fastai"],"date_published":"2021-02-17T00:00:00+00:00","date_modified":"2021-02-17T00:00:00+00:00","author":{"name":"manuelmazzuola","url":"https:\/\/manuelmazzuola.dev","avatar":null}}]}