Log and Debug GORM

Sometimes its necessary to see the sql queries executed by GORM, so its possible to troubleshoot an issue. In this post, we cover different ways doing just that

1. One way is to enable SQL logging by adding following line in conf/DataSource.groovy:

...
   development {
        dataSource {
            dbCreate = "" // one of 'create', 'create-drop', 'update', 'validate', ''
            ...
            logSql = true
        }
 ...

This enables SQL logging to stdout

2. Second way, is to to enable Hibernate’s logging to log the SQL along with bind variables (so you can see the values passed into your calls, and easily replicate the SQL in your editor or otherwise). Add the following lines conf/Config.groovy:

// log4j configuration
log4j = {
...
    // Enable Hibernate SQL logging with param values
    trace 'org.hibernate.type'
    debug 'org.hibernate.SQL'
...

3. Third way, is to profile GORM with some utility such as P6Spy
…coming in future

Adding External Jar to Grails

One option for object type of Dates is LocalDate type (org.joda.time package),however , it  doesn’t come part of Grails stack. In order to use it in Grails projects, it’s necessary to install and configure the external dependency package – org.joda.time.LocalDate. This post covers steps how to add an external Jar( i.e. org.joda.time) in your Grails project

Step 1. Retrieve Dependency

Grails is using Ivy as its dependency manager to organize dependencies in single location locally while documenting. The Ivy dependency manager will load the dependency locally with following command:

>grails install-dependency nameOfDependency

Before calling this command, lets find out the name of dependency for our new data type ‘LocalDate’. To do so, we look in shared centra repo such as maven2 repository for instance.. It hosts nearly of 90% of Java’s open source libraries, so chances are high it will host the dependency – org.joda.time.LocalDate

The maven2 repository search locates the dependency of interest if using keyword ‘joda time’. To locate name, click on the version of the dependency and under ‘Dependency Information’ there is grails name for this dependency as shown here:

As you can see, it specifies Grails framework(green line) along the actual dependency name (red line)

With the name, its time to download the dependency as follows:

>grails install-dependency joda-time:joda-time:2.1

This will make ivy manager go grab the dependency from shared central repo such as maven2 repo and install it in local Ivy repo that way caching it. If you like to check what’s in the Ivy local repo cached, look into .ivy2/cache folder (by default in home directory)

Step 2. Add Dependency to Grails Project Class Path

Once in local Ivy repo, it necessary to tell the particular grails project to use it or, in another words, add it to the class path of the particular Grails project. In order to do that, update  the ‘dependencies’ section in the BuildConfig.groovy as following:

dependencies {
   ...
   compile 'joda-time:joda-time:2.1'
   ...
}

With this, the particular Grails project will know to include your new dependency

Step 3. Refresh and Compile

First, refresh the dependencies as following:

>grails refresh-dependencies

This takes the dependency(.jar) from the local Ivy repo(.ivy2/cache) and moves into grails Ivy repo(.grails/ivy-cache by default home directory) making it in class path for grails to find at compile time.
Next, compile the dependency with command

>grails compile

This grabes the configured dependency name in BuildConfig.groovy(Step2), locates the jar in local Ivy grails repo(.grails/ivy-cache) and compiles

Step 4. Verify

Afterwards, to verify that the particular grails project includes the dependency and the right version, look it up in grails dependency report. To generate report run the following:

>grails dependency-report

This will generate report that you open in any browser. In the report, in the ‘compile’ section you should see your new dependency listed as follows:

To Summarize

A single picture worth more than 1000 words, so graphical summary:

Other

1. Note on Configurations
Besides BuildConfig.groovy also ‘application.properties’ contains configuration of dependencies such as plugins. If in both places there are configuration for the some plugin for instance than the ‘application.properties’ config supersedes the Buildingconfig.groovy configurations
2. Hibernate Support
To ensure that the LocalDate is persistant in the Database accordingly, lets add the hibernate support (in BuildConfig.groovy)

dependencies {
...
compile('joda-time:joda-time-hibernate:1.3') {
            excludes 'hibernate', 'joda-time'
        }
...
}

Troubleshooting

Sometimes the ‘compile’ doesn’t seem to compile the new dependency right the way. To see what is being compiled, turn on the logging in BuildConfig.groovy:

grails.project.dependency.resolution = {
 ...
  log 'info'
...

By setting the log level to ‘info’, the process of compile, build, etc is logged in the console output.

Useful Links

Externalizing Configurations per Grails Application

You may have multiple grails applications and each is using different datasource, mail, etc configurations. It may be benefit to externalize those configurations for maintainability as well as portability reasons. In this post, i demonstrate one way to accomplish it.

Make Grails Externalize

In order to externalize configurations, you have to add following line in the conf/Config.groovy:


grails.config.locations = ["file:${userHome}/.grails/apps-config/${appName}/${appName}-config.groovy"]

This way you specify the path as well as the name of the configuration file. We choose to keep our configuration in Groovy file, however, it works as well if .properties file:


grails.config.locations = ["file:${userHome}/.grails/apps-config/${appName}/${appName}-config.properties"]

However, its preferable in .groovy file because that way you are able to script. Please see section ‘.Groovy vs .Properties’ below

Naming Application

In the above, you can see we using variable – appName. This variable is set by Grails at the startup. It is defined in the ‘application.properties’ file in root directory of the application. Here is example:


#Grails Metadata file
#Sat Oct 06 09:17:17 CDT 2012
app.grails.version=2.1.1
app.name=minnehaha
app.servlet.version=2.5

Grails reads the value of ‘app.name’ and initializes variable ‘appName’ that you can use in any of Grails configuration file. In our case, we use to set proper directory as well as the externalized configurations file name

External Configuration File

Once externalized in ‘conf/Config.groovy’ and application name set as described above, at last, create groovy script with externalized configurations and place it in directory specified above. Here is an example of mail server configuration externalized:

grails{
    mail{
        host="smtp.gmail.com"
        port=465
        username="website@ms.com"
...
    }

}

This will configure mail server with the email account for the specific application

.Groovy vs .Properties file

Externalizing in .groovy file,  does not necessary mean you are unable to accomplish the same with .properties file. For the fact it is doable, however, let me demonstrate how, so its clear, going with .groovy file, is the preferred way.

For example, to externalize in .groovy configuration file something per different environment such as datasource is as easy as adding:

environments {
    test {
       dataSource {
          driverClassName = "com.mysql.jdbc.Driver"
         ...
         }
    }
    development {
       dataSource {
          driverClassName = "com.mysql.jdbc.Driver"
         ...
         }
    }
}

It doesn’t necessary mean you are unable to accomplish the same with .properties file. For the fact it is doable, however, let me demonstrate how, so its clear, going with .groovy file, is the preferred way.

To accomplish the same – different datasource per environment, its necessary to update the externalization variable ‘grails.config.locations’ in the .conf/Config.groovy:

grails.config.locations = ["file:${userHome}/.grails/${appName}-config-${grails.util.Environment.current.name}.properties"]

From the code snippet you can see, we append the environment to the .properties file name. So, that different .properties file is read for each environment that way different configurations applied:

dataSource.dbCreate =
dataSource.driverClassName = com.mysql.jdbc.Driver
dataSource.url = jdbc:mysql://localhost/minnehaha_test
dataSource.username = root
dataSource.password = mysql

Above is an example for test environment .properties file to use appropriate database, however. There needs to be .properties file per environment:

 >/home/me/.grails/myapp/myapp-config-development.properties
  >/home/me/.grails/myapp/myapp-config-test.properties
  >/home/me/.grails/myapp/myapp-config-production.properties

In short, it required many more .properties files to accomplish the same in one .groovy due to scripting capabilities

NOTE: For .properties config files, make sure quotes are removed,otherwise, errors like such ‘ Cannot load JDBC driver class ‘”com.mysql.jdbc.Driver”‘ is received

Its important to note that externalized configurations supersede the configurations in any of the Grails configuration files(BuildConfig.groovy, Config.groovy, DataSource.groovy,etc.)

Issues

1. If the externalized .groovy script file has syntax errors then none of the configurations are read as well as Grails continues the startup without any error notification.

Domain Object Many-To-One relationship Grails GORM and SQL Relationship

This is following post series from ‘Domain Object Relationships in Grails GORM and SQL Relational Databases‘ where we look at the details how the relationships are build in GORM with how physically structrured in DB while cascading and direction addressed.

From Grails Documentation, it appears that the relationship One-To-One is the same as Many-to-One in Grails GORM. In this post, we look at One-To-One relationship used as Many-to-One one and whether it is true that One-to-One equals Many-To-One relationship?

In this post, we use the same example – Book<->Author as in the post on Domain Object One-to-One relationship. In fact, this post builds on it since we are not  going to cover db structure, coding or direction/cascading of the One-to-One relationship. We are building Many-To-One relationship with One-to-One relationship implementation:

 def authorToShare = Author.build().save(flush: true)
 def book1 = Book.build(author: authorToShare).save(flush: true)
 def book2 = Book.build(author: authorToShare).save(flush: true)

As you see, a single Author object – authorToShare is shared  by adding to two different Book objects – book1 & book2 turning the Book and Author  relationship from One-to-One to Many-to-One relationship

Many-To-One with Unidirectional One-To-One Relationship

First, we consider unidirectional One-to-One Book to Author relationship as described in the post ‘Domain Object One-to-One relationship Grails GORM and SQL Relationship‘ to accomplish Many-to-One book and author relationship. If we adding shared author  to two different book objects, it indirectly builds Many-to-One Relationship. To validate, lets have test:

def "one-to-one relationship between Book and Author"() {
 given:
 def authorToShare = Author.build().save(flush: true)

when:
 def book1 = Book.build(author: authorToShare).save(flush: true)
 def book2 = Book.build(author: authorToShare).save(flush: true)
 book1.refresh()

then:
 1 == Author.count()
 book1.author.id == book2.author.id

This test pass, so GORM permits adding more books(i.e. book1 & book2) with the same author(i.e.authorToShare) making it also Many-to-One relationship. From ER diagram of One-To-One relationship, it is possible to add more books that contains the same id for author(i.e. ‘author_id’ column) . In another words, in the ‘book’ table there is entry for each book that contains author_id column holding the same ‘id’ of Author.

Many-To-One with Bidirectional Relationship

The same test as above is run, however, the relationship between Book and Author this time is bidirectional instead of unidirectional one. When running test, the following exceptions is rised:

 More than one row with the given identifier was found: 1, for class: Book; nested exception is org.hibernate.HibernateException:

To better understand what is going on we update the test as following:

def "one-to-one relationship between Book and Author"() {
 given:
 def authorToShare = Author.build().save(flush: true)

when:
 def book1 = Book.build(author: authorToShare).save(flush: true)
 def book2 = Book.build(author: authorToShare).save(flush: true)
 book1.author = null
 book1.refresh()

then:
 1 == Author.count()
 book1.author.id == book2.author.id

By adding ‘book1.author=null'(highlighted line), the test passes. This is because, we adjust GORM session, so that there is no two objects(book1 & book2 in our case) pointing to the same object(authorToShare). Apparently, while it is fine with Database table structure (as we see with Unidirection relationship above), the GORM session mechanism prohibit a shared entity (i.e. author) to be reference more than one object (i.e. books). In short, with bidirectional One-to-One relationship, we are unable to turn it into Many-to-One relationship

To summarize, it is true that GORM One-To-One relationship is equal to Many-To-One but ONLY if Unidirectional One.  For Bidirectional One-to-One relationship, while db structure permits Many-to-One relationship, the GORM session it does not. Thus, if you need true one-to-one relationship, make sure it is bidirectional instead unidirectional. Otherwise, by accident its is possible to create Many-to-one that may cause strange behavior when deleting due to extra references present.

Domain Object One-To-Many Relationship in Grails GORM and SQL Relational DB

This is following post series about ‘Domain Object Relationships in Grails GORM and SQL Relational Databases‘ where we look at the details how the relationships build in GORM with how physically structrured in DB while cascading and direction addressed.

We are going to use RetnalUnit – Review example where one RentalUnit may have multiple reviews and each review can be assigned to only one rental unit. In another words RentalUnit and Review has One-To-Many relationship .

This post is divided into four sections as following:

In each section, there is ER diagram, Class diagram and code snippet to better cover the details

Database Structure for Unidirectional One-To-Many Relationship of GORM

To better see the structure of the relationship in Database, the ER Diagram and a view of tables is presented here:

From the diagram, the unidirectional one-to-many relationship is constructed by having addition table (i.e. Retnal_Unit_Review) containing two columns each Id fields from tables of the entities forming the relationship(i.e. RentalUnit and Review). The new added table is child to parent tables – Review and RentalUnit.

Section 1 – Unidirectional Non-Cascading One-To-Many

(a)Class Diagram

(b)Code Snippet

class Review {
    String submittedBy
    String content
    String dateReceived
    boolean isApproved

    static constraints = {
        submittedBy blank: false, size: 3..50
        content blank: false, size: 5..2500
    ....
    }
...
}
class RentalUnit {
    String name
    String nickname
    Address address
    static hasMany = [reviews:Review]

    static constraints = {
        name blank: false, unique: true, size: 4..10
        nickname blank: false, size: 5..60
    }
}

No association defined  in the Review class which makes it unidirectional. There is association ‘reviews’ defined in the RentalUnit class(line 5 highlighted).

Section 2 – Unidirectional Cascading One-to-Many Relationship

(a)Class Diagram

(b)Code Snippet

class Review {
    String submittedBy
    String content
    String dateReceived
    boolean isApproved

    static belongsTo = RentalUnit

    static constraints = {
        submittedBy blank: false, size: 3..50
        content blank: false, size: 5..2500
    ....
    }
...
}

By adding ‘belongTo’ (highlighted line) in the Review class, makes this unidirectional relationship cascade (you inform GORM that the Review belongs to Rental Unit, so it knows when Rental Unit is deleted to cascade onto particular Review). No change needed in Rental Unit

But who cares for orphans!?

The cascading will work for all those reviews pointed by rentalUnit you are deleting:

given:
   def rev1 = Review.build().save(flush:true)
   def rev2 = Review.build().save(flush:true)
   def rentalUnit = RentalUnit.build().addToReviews(rev1).addToReviews(rev2).save(flush:true)
when:
   assert 2 == Reviews.count()
   rentalUnit.delete()
then:
   0 == Reviews.count()

In line 7, the rental unit is deleted and it deletes  all reviews associated with this rental unit from database(validated on line 9), however. It will not cascade on delete if the review is orphan:

given:
   def rev1 = Review.build().save(flush:true)
   def rev2 = Review.build().save(flush:true)
   def rentalUnit = RentalUnit.build().addToReviews(rev1).addToReviews(rev2).save(flush:true)
when:
   assert 2 == Reviews.count()
   rentalUnit.removeFromReviews(rev1)
then:
   2 == Reviews.count()

In line 7, the review – rev1 becomes an orphan since removing it from rental unit, the review is no more referencing any rental unit (the entry in review table  pointing to none or rental_unit_in is null). The line 9 validates that cascading on delete has not applied to orphans( the removed review ‘rev1’ is still in the database). If you wish to apply cascading also to orphans then you have to add following lines:

class RentalUnit {
    String name
    String nickname
    Address address
    static hasMany = [reviews:Review]

    static mapping = {
        reviews cascade: "all-delete-orphan"
    }
...
}

By specifying ‘all-delete-orphan’, the cascade on delete is applied to orphans as well. In our situation, the review – rev1, thus, will be deleted from database when removed from Rental Unit (i.e. rentalUnit.removeFromReviews(rev1))

Don’t be Tricked

For a while, i was puzzled because i saw in my follow developers code that  there was missing line ‘static belongtsTo = SOMEClass’ (line 7 in our Review class) on the right side of the association(i.e. Review) that  enables cascading but it was still cascading fine on deletes. It turns out that the cascading can be enabled from the left side of the association(i.e. Renta Unit)  with the following:

static mapping = {
        reviews cascade: "all-delete-orphan"
}

So, please, don’t be puzzled if cascading works if you don’t see it enabled from the right side of the association. Check the left side of the association. The same applies vice versa. In short, its enough enable cascading on only one side to make it work

Database Structure for Bidirectional One-To-Many Relationship of GORM

To better see the structure of the relationship in Database, the ER Diagram and  view of tables is presented here:

From the diagram, the bidirectional one-to-many relationship is constructed by having addition field ‘rental_unit_it’ in the Review table that is also a foreign key. This makes a child-parent relationship between rental unit and review in  bidirectional one-to-many relationship.

Section 3 – Bidirectional Cascading One-to-Many Relationship

(a)Class Diagram

(b)Code Snippet

class Review {
    String submittedBy
    String content
    String dateReceived
    boolean isApproved

    static belongsTo = [rentalUnit: RentalUnit]

    static constraints = {
        submittedBy blank: false, size: 3..50
        content blank: false, size: 5..2500
    ....
    }
...
}
class RentalUnit {
    String name
    String nickname
    Address address
    static hasMany = [reviews:Review]

    static mapping = {
        reviews cascade: "all-delete-orphan"
    }

    static constraints = {
        name blank: false, unique: true, size: 4..10
        nickname blank: false, size: 5..60
    }
}

The highlighted line in Review sets the association to Rental Unit. It also enables cascading on delete (save/update is by default). The highlighted line 5 in the Rental Unit sets the associations to many Reviews.

Section 4 – Bidirectional Non-Cascading One-to-Many

(a)Class Diagram

(b)Code Snippet

class Review {
    String submittedBy
    String content
    String dateReceived
    boolean isApproved

    static belongsTo = [rentalUnit: RentalUnit]

    static constraints = {
        submittedBy blank: false, size: 3..50
        content blank: false, size: 5..2500
    ....
    }
...
}
class RentalUnit {
    String name
    String nickname
    Address address
    static hasMany = [reviews:Review]

    static mapping = {
       reviews cascade: "none"
    }

    static constraints = {
        name blank: false, unique: true, size: 4..10
        nickname blank: false, size: 5..60
    }
}

The highlighted lines remove cascading for association between rental unit and review. In another words, if you, for example, removing some review in rentalUnit1.reviews collection, it does not physically remove the review from the Review table in database

Step Further

If you remove the left side association (the reference into Review from the Rental Unit class)  from the Bidirectional Rental Unit to Review relationship described above, guess what it will become? Let’s look at Class Diagram:

From the class diagram, we see it is now unidirectional relationship, however different than one above(make sure you compare diagrams). This Unidirectional One-To-Many relationship  is pointing oposite direction. The database structure is the same as the Bidirectional one-to-many relationship described above. In another words, you have two versions of Unidirectional one-to-many relationship separated from each other by database structure as well as the direction of the single association.

The cascading is inherited unless you explicitly disable with ‘static mapping’ statement as described  above(highlighted lines). With that, i would stay away from this relationship because it quickly becomes impossible to delete  either some instance of ‘review’ or ‘rentalUnit’ with associations present and no ‘removeTo’ grails method available to help.

Other

java.util.NoSuchElementException – Challenging One-To-Many

How would GORM handle situation where we pass the same Review to two different Rental Units making it not true One-To-Many relationship?

In another words, the relationship One-to-Many from RentalUnit to Review == Many-to-One from Review to RentalUnit. Now, if we are able to add the same shared Review to two different RentalUnits, then the Review, instead of one, is pointing to more than one RentalUnit.  Thus, this is no more Many-to-one from Review to RentalUnit. We run following Spock test to analyze ( with bidirectional cascading one-to-many implementation as described above):

when:"making true one-to-many"
        def review3 = Review.build().save(flush: true)
        def rentalUnit1 = RentalUnit.build().addToReviews(review3).save(flush: true)
        def rentalUnit2 = RentalUnit.build().addToReviews(review3).save(flush: true)
        review3.submittedBy = "ziggy"
        review3.save(flush: true)

then:
        rentalUnit1.reviews.first().id == rentalUnit2.reviews.first().id
        review3.rentalUnit.id == rentalUnit2.id
        rentalUnit1.reviews.first().submittedBy == "ziggy"
        rentalUnit2.reviews.first().submittedBy == "ziggy"

For our surprise, this test pass making it look like the review ‘review3’ is pointing to two rental units ‘rentalUnit1’ and ‘rentalUnit2’, however. From database table structure of one-to-many described above, this is not possible.

In another words, in order this test pass, the GORM would have to clone review3 in the Review table referencing it to the another rental_unit-id, but then still line 9 (highlighted) would fail since all IDs are unique in the Review table. So, how is it possible this test pass? Thank you Victor Sergienko at stackflow.com for being patient helping me understand. The test will fail if adding one line as follows:

when:"making true one-to-many"
        def review3 = Review.build().save(flush: true)
        def rentalUnit1 = RentalUnit.build().addToReviews(review3).save(flush: true)
        def rentalUnit2 = RentalUnit.build().addToReviews(review3).save(flush: true)
        review3.submittedBy = "ziggy"
        review3.save(flush: true)
        rentalUnit1.refresh()

then:
        rentalUnit1.reviews.first().id == rentalUnit2.reviews.first().id
        review3.rentalUnit.id == rentalUnit2.id
        rentalUnit1.reviews.first().submittedBy == "ziggy"
        rentalUnit2.reviews.first().submittedBy == "ziggy"

By adding line to refresh() of rentalUnit1, the test fails with exception – ‘java.util.NoSuchElementException’.

In line 4, when adding ‘review3’ to the ‘rentalUnit2’, two things happen. First, the review3 entry in the database is updated with different rental_unit_id to point to rentalUnit2. Second, at the same time, the session for rentalUnit2 is updated to point to review3, however!!!! The session is not updated in respect to rentalUnit1. In the session, the rentalUnit1 still pointing to review3 even thou in database it is not. This was why our test was passing. By refreshing the rentalUnit1, the session was reloaded with what is actually in database making it in sync. Then test was failing as expected.

Important to note, that refreshing either the ‘review3’ or the ‘rentalUnit2’ would not make the test fail. It has to be the rentalUnit1 since it is the one out of sync.

In summary, updating associations for domain objects in GORM, may cause session be out of sync with database. This is because the association overwritten by new association, does not get update in session making out of sync till reloaded from database.

Things Different For Unidirectional

For the Unidirectional One-To-Many relationship the above ‘refresh()’ doesn’t remove the reference to the shared Reveiw. As result the test fails in Unidirectional but not if Bidirectional relationship:

def "one-to-many relationship between RentalUnit and Review"() {
        when:"adding shared review will result referenced by last rental unit"
        def review3 = Review.build().save(flush: true)
        def rentalUnit1 = RentalUnit.build().addToReviews(review3).save(flush: true)
        def rentalUnit2 = RentalUnit.build().addToReviews(review3).save(flush: true)
        rentalUnit1.refresh()

        then:
        !rentalUnit1.reviews.contains(review3)
    }

This test fails, perhaps, because the data structure is different. As you saw from the post, there is extra table in Unidirectional relationship. And, I am guessing, that the ‘addToReviews’ method does not update that extra table(line 5), so the reloading session (highlighted line) still contains reference to the shared review. This is different in Bidirectional relationship where ‘addToReviews’ handles the reference in dB and then by reloading session the reference to shared Review was cleared/sync with database accordingly.

Running Multiple Versions of Grails

There may be times when you have to run multiple versions of Grails in your local development environment. It may be because you supporting multiple Grails applications each running different version, or you would like to test your prime Grails app in different version before shipping to customer, or you just grabbed some Grails application from internet( ‘into wild’) and like try out. For whatever reason, in this post i describe one setup that auto detects version of the application and runs it in appropriate Grails version.

Pre-requisite: Unix based machine

General Idea

We set up so that when ‘grails’ command is called, it runs a script. This script will read the version from Grails application(application.properties) you want to run and forwards all of the commands to the appropriate grails version to run.

All of the multiple Grails framework versions are installed in ‘/DevTools/grails’ folder in a format grails-x.x.x.

From the multiple Grails versions, we have chosen one version as default one  and defined env variable GRAILS_HOME that points to it.

Step 1 – Create Script

Create file ‘grails’ in some folder you choose. Make this file executable by running ‘chmod u+x filename’

Next, copy past the following:

#!/bin/bash grails
GRAILS_CMD=$1
shift

GRAILS_DEFAULT=$GRAILS_HOME

if [ -f application.properties ];then
 export GRAILS_VERSION=`grep app.grails.version application.properties | sed -E 's/.*=(.*)/\1/'`
 export GRAILS_HOME="$DEV_TOOLS/grails/grails-$GRAILS_VERSION"
 echo "application.properties found, using \$GRAILS_HOME of $GRAILS_HOME"
else
 echo "application.properties NOT found, leaving \$GRAILS_HOME as $GRAILS_HOME"
fi

if [ ! -d $GRAILS_HOME ]; then
 if [ ! $GRAILS_HOME = $GRAILS_DEFAULT ] && [ -d $GRAILS_DEFAULT ]; then
        echo "Unable to find $GRAILS_HOME. Switching back to default $GRAILS_DEFAULT"
        export GRAILS_HOME=$GRAILS_DEFAULT
 else
        echo "ERROR: Unable to find \$GRAILS_HOME directory at $GRAILS_HOME"
        exit 1
 fi
fi

echo $GRAILS_HOME/bin/$GRAILS_CMD $*
$GRAILS_HOME/bin/$GRAILS_CMD $*

Make sure to adjust the highlighted line 8 to point to your folder where all of the different Grails versions are located for you.

Step 2 – Setup ‘grails’ Command to Run Script

Update your PATH variable to include the folder containing the script – grails you created in Step 1.

Note: make sure you restart command line terminal to ensure your new updated env variable PATH is loaded

Running Different Versions

Afterwards, when command ‘grails something’ is called, then it goes to your ‘grails’ script. The scripts searches through the application.properties file for the version. Once found, it checks if this version is stored in your machine. If it is not, it runs the default version you have set in ‘GRAILS_HOME’ variable.

There may be times you need to run ‘grails-debug’ command. If that is the case you then change your setup as follows:
1. Rename ‘grails’ script to ‘grails-version’
2. Create new file named ‘grails’ and add the following:

grails-version grails $*

3. Create new file named ‘grails-debug’ and add the following:

grails-version grails-debug $*

Note:make sure you make new scripts executable like step 2

So this will forward ‘grails’ and ‘grails-debug’ commands to versioning script before running either of the commands

Useful Links

Unix Shell Scripting Tutorial

Grails Snippets

This is ongoing post to record different useful codes snippets related to Grails framework

1. Default and Shared Constraints.
It turns out you can specify default constraints that can be applied and shared in any Domain Class. For example, by default the ‘nullable’ constraint is set to ‘false’. If you wish change that,then updated conf/Config.groovy as follows:

grails.gorm.default.constraints = {
    '*'(nullable: false)
    noBlankSized(blank: fralse, size: 4..25)
}

The ‘*’ attaches the constraint to all property’ validations. The constraint ‘myShared’ can be references in any validation you choose as follows:

class User{
...
 static constraints = {
        favoriteDish shared: "noBlankSized"
    }
}

This way you able to share your own default validations while ability to reuse as you like

2. Suppor Handling Request params

A request parameter value can be converted to a specific type with method invocation:

params.long('book.id')

This takes book.id and converts it to long type. Grails supports other methods like byte(), long(), boolean(). Since Grails 2.0 also support for dates.

3. Access Domain class from Plugin
Say you developing plugin that needs to access domain class in the application.

String className = "com.foo.bar.MyDomainClass"
Class clazz = grailsApplication.getDomainClass(className).clazz
def findAnything = clazz.findByStuff(params.stuff)

If Domain class without package and name unique across all packages:

String className = "MyDomainClass"
Class clazz = grailsApplication.domainClasses.find { it.clazz.simpleName == className }.clazz
def findAThing = clazz.findByStuff(params.stuff)

This may not be best practices as we are coupling the plugin with particular applications, however,perhaps in some circumstance it necessary

4. Mocking Upload File feature with multipart/form-data

void testFileUpload(){
        def imgContentType = 'image/jpeg'
        def imgContentBytes = '123' as byte[]
...
        controller.metaClass.request = new MockMultipartHttpServletRequest()
        controller.request.addFile(
            new MockMultipartFile('image', 'myImage.jpg', imgContentType, imgContentBytes)
        )
...
        controller.save()
...
        assertEquals HttpServletResponse.SC_OK, controller.response.status
..
    }

Current Path from Request
To get current path from the current controller and action:

request.forwardURI

Grails Environment Aware Framework

One of the awesome features Grails provides is its ability to configure multiple environments to run your application. It comes with three initial environment setups – Development, Test, Production. Besides these environments,  you are able to setup your own custom environment.

The advantage of having multiple environments is the ability to execute your application with different settings, different data sources and other specifics. For example, you may wish to run tests against different database than one used by actual application. Likewise, you may wish not load application data (User, etc) when testing to avoid tests to break. Despite the need, it is accomplished by having different settings in DataSource, Config, BootStrap, BuildConfig and Searchable configurations(grails-app/conf/) each executed per specified environment.

In this post, first I will briefly  cover how to run different environments. Afterwards, I will demonstrate how to declare and configure environments, all of which in relation to practical problem/solution examples from my development process

Run Apps Different Environment

To run app or execute any command(i.e grails war, etc.) in different environment it is needed to specify the environment after ‘grails’ command (i.e. ‘grails [env] run-app’). Here is some examples:

>grails dev run-app
>grails -Dgrails.env=mycustomenv war
>grails test dbm-status

As you can see, the enviroment is specified right after ‘grails’ command. If you don’t specify, it is using the default which is ‘dev’ or Development environment. For your custom environments, you add ‘Dgrails.enf=’ with the name of your custom environment you wish to run

Practical Usages

1. Loading App Data in Bootstrap

At some point, your application will need to have some data like users, roles, etc. One of the ways to import data is to BootStrap(specify in BootStrap.groovy file) the data into your application, however. At the time of running tests, that is in test environment, you probably don’t want to have any data bootstraped, otherwise, it will break your tests(i.e. assert User.count() == 0 fails). So, the solution is to look up environment before importing or not importing data in BootStrap.groovy. Here is example:

class BootStrap {

    def loadDomainDataService

    def init = { servletContext ->

        environments {
            test {
                log.info("starting in Test env.....")
            }
            dbgorm {
                log.info("starting in dbgorm env.....")
            }
            development {
                log.info("starting in dev env.....")
                (User.findAll().empty) ? loadDomainDataService.loadSecurityData() : log.info("Security Data already loaded. Skipped loading...")
                (RentalUnit.findAll().empty) ? loadDomainDataService.loadAppData() : log.info("App Data already loaded. Skipped loading...")

                //don't start app if data not loaded
                assert User.count()
                assert RentalUnit.count()
            }
        }
...

Or Another way:

import grails.util.GrailsUtil
class BootStrap {

    def loadDomainDataService

    def init = { servletContext ->

        switch (GrailsUtil.environment) {
            case "test":
                log.info("starting in Test env.....")
                break
            case "dbgorm":
                log.info("starting in dbgorm env.....")
                break
            default:
                (User.findAll().empty) ? loadDomainDataService.loadSecurityData() : log.info("Security Data already loaded. Skipped loading...")
                (RentalUnit.findAll().empty) ? loadDomainDataService.loadAppData() : log.info("App Data already loaded. Skipped loading...")
                //don't start app if data not loaded
                assert User.count()
                assert RentalUnit.count()
                break
        }
...

In the above code snippets, the different environments configured is highlighted. There are the predefined env such as ‘test’, ‘dev’ and there is our custom env – ‘dbgorm’. For the custom env, there is no need declare it separately prior using them. Just use them in any configuration file like any of the predefined environments. The custom env will be referenced by the name you choose to use in the configuration files. It is case sensitive.

From the above you can see how we prevent the app data be loaded in ‘test’ or ‘dbgorm’ environment but do load data if application run in ‘dev’ and other env . On the side note, to load data itself we use Grails Fixture plugin about which another post

2. Data Source Dependent on Env
Once your application is loaded with data needed for application like Users, Roles, etc than this info is stored in Database. Afterwards, you would like run tests, however, many of the tests will fail because your database contains application data that most likely conflicts with assertions in your tests (i.e. Users.count()==0).

One of the solutions is to use a separate database for running tests. You have your ‘dev’ environment configured for one database and then your ‘test’ environment with another database. The database used by each environment is configure in conf/DataSource.groovy file. Here is an example:

// environment specific settings
environments {
    development {
        dataSource {
            dbCreate = "" // one of 'create', 'create-drop', 'update', 'validate', ''
            driverClassName = "com.mysql.jdbc.Driver"
            url = "jdbc:mysql://localhost/minnehaha"
            username = "root"
            password = "mysql"
        }
    }
test {
        dataSource {
            dbCreate = "" // one of 'create', 'create-drop', 'update', 'validate', ''
            driverClassName = "com.mysql.jdbc.Driver"
            url = "jdbc:mysql://localhost/minnehaha_test"
            username = "root"
            password = "mysql"

        }
    }
...

The highlighted lines shows the each different database configured. So, in our situation, if application is started in ‘dev’ it will be using ‘minnehaha’ database. If  tests run(i.e. grails test-app) then it uses by default the ‘test’ environment, thus, the tests are run against database – ‘minnehaha_test’. This way we are sharing the same application to develop and test while separating the data into two different databases to avoid conflicting between process of testing and other

Interestingly to note, both 'test' and 'dev' databases is managed by Grails Database Migration Plugin, so how would you run migration commands against each different database? Make sure you specify environment when calling any of the Database migration plugin commands. Here is an example:
 >grails dbm-update
 >grails test dbm-update
 The first command will run migrations on default environment which is 'dev' updating database - 'minnehaha' in our case. The second command will run migration script in 'test' env that will update database - 'minnehaha_test' only

3. Migration Script Run at Start Up?… Not Always
We use Grails Database Migration Plugin to manage the database updates, new table creation and database roll backs when needed. Every time the branch is changed or  I received a new code changeset, I was finding myself repetitively running ‘dbm-update’ manually to get dev database in sync. It turns out, this step can be automized, so that when the grails application start, it runs ‘dbm-update’ if there is new migration scripts. For more, see section ‘Run Migration Scripts at Start’ in  the post – Grails Database Migration Plugin

In times adding/updating domain objects, this was a problem. This is because, after making changes in GORM, it was in conflict with current DB structure built by migration scripts at the start up. To solve the problem we created custom environment specifically for GORM development as follows:

Step 1. In the custom env – dbgorm the database will not use migration plugin. Here is conf/DataSource.groovy:

environments {
..
    dbgorm {
        dataSource {
            dbCreate = "create-drop" // one of 'create', 'create-drop', 'update', 'validate', ''
//          url = "jdbc:h2:mem:devDb;MVCC=TRUE"
            driverClassName = "com.mysql.jdbc.Driver"
            url = "jdbc:mysql://localhost/minnehaha_gorm"
            username = "root"
            password = "mysql"

        }
    }
...
}

We accomplish this by configuring our custom environment create and drop database, instead of database migration plugin does it(change ‘dbCreate’ attribute to ‘create-drop’ data source as highlighted in above code).

Step 2. The step 1 takes care of building the database structure, but we still need to prevent migration scripts to run at start up in our custom env – dbgorm. To accomplish, we update conf/Config.groovy as follows:

//To dbm-update On Start
grails.plugin.databasemigration.updateOnStart = true
grails.plugin.databasemigration.updateOnStartFileNames = ["changelog.groovy"]
grails.plugin.databasemigration.changelogLocation = 'grails-app/migrations'

environments {
    dbgorm {
        grails.plugin.databasemigration.updateOnStart = false
    }
}

So, if the application is run in env – dbgorm, we set the updateOnStart to false that way preventing migrations scripts to be run at start up.

With this setup, we change/update domain class or GORM in dbgorm environment. Afterwards, when we satisfied with the changes then we run:

 grails dbm-gorm-diff --add filename.groovy

in dev environment for capturing the changes. This generates migration scripts added in the changeset log(for more, see post ‘Grails Database Migration Plugin‘) and that is run next time app starts in environment other than dbgorm.

How To Know What’s default?

You may wonder what is the default settings if none is specified for your custom environment. For example, if you have custom environment but you don’t specify Data Source for it, what Data Source is it going to use?

Grails configuration files – DataSurce, Config, BuildConfig,etc are using Groovy utility – ConfigSlurper. The ConfigSlurper “allows a default setting to exist in the property file that can be superceded by a setting in the appropriate environments closure”(from ConfigSlurper documentation). In another words, following example of Data Source, there is the default property – ‘datasource’  following with declarations of ‘datasource’ for each environment that supercedes the default one.

//default
dataSource {
 ...
}

// environment specific settings
environments {

	development {
		dataSource {
...

Useful Links

Domain Object Many-To-Many Relationship in Grails GORM and SQL Relational DB

This is following post series about ‘Domain Object Relationships in Grails GORM and SQL Relational Databases‘ where we look at the details how the relationships build in GORM with how physically structrured in DB while cascading and direction addressed.

In this post, i have a sample of Many-To-Many relationship between entities ‘RentalUnit’ and ‘Review’. A particular RentalUnit may have many Reviews. One guest my have stayed in multiple RentalUnits, so a review written by this guest is linked to many RentalUnits

In our example, there is ‘Save, Update and Delete’ cascading from RentalUnit to Review and ‘none’ cascading from Review to RentalUnit

Class Diagram

From class diagram you can see there is ‘has to’ relationship between RentalUnit and Review entities

Code Snippet

In the above code snippet, the red colored code sets the relationship Many-To-Many. The green colored code sets the cascading

Database Structure

From the ER Diagram above, we see that GORM creates a separated table – RentalUnit_Review that is a child to parent ‘RentalUnit’ and another parent ‘Review’ tables. The IDs of entities ‘Review’ and ‘RentalUnit’ combines the primary key for the new table

Summarized

Our conclusion that GORM creates additional table every single time there are Many-To-Many or One-To-Many relationship.

Other

1. addTo* And removeTo* methods.

Whenever there are Many-To-Many or One-To-Many relationship, GORM provides you with methods – addTo* and removeTo*. These methods does important part – it sets the relationships in both ways. Consider the following:

def review2 = new Review(rentalUnits: [rentalUnit], ...., isApproved: false).save(flush: true) 
review2.addToRentalUnits(rentalUnit2)

In the above code we initialize object ‘review2’ that points to object ‘rentalUnit’ and ‘rentalUnit2’, however.  While object ‘rentalUnit2’ is pointing back to object ‘review2’, the object ‘rentalUnit’ does not. This is because adding object ‘rentalUnit2’ we used method ‘addTo*’ where as for object ‘rentalUnit’ we did not. As result we have incomplete session.

2. “deleted object would be re-saved by cascade”

Entities like ‘Review’ that have references from multiple ‘RentalUnit’ objects while no cascading applied, will cause errors if not handled with care. For example, if you have to delete a review object R1 that is contained by two rentalUnit objects A and B, then you have to first delete the references from A->R1 and B->R1 before deleting the review – R1 itself. Here is example

def r = []
 r += review1.rentalUnits
 r.each {RentalUnit renUnit ->
     renUnit.removeFromReviews(review1)
 }
 review1.delete(flush: true)

In another words, we are doing cascading manually and should not be needed if we have specified cascading from Review to RentalUnit.

If we don’t handle these RentalUnit references as shown above before deleting the review R1, we will receive an error – deleted object would be re-saved by cascade – and the review R1 is not removed. This because GORM deletes the review R1 in session and before flushing, it detects that there exists other references to this review R1, thus , it cannot be remove, instead restores (re-saved) the review R1 back in session.

3. Cascading gets tricky in Many-To-Many relationship

Consider a RentalUnits U1 and U2 with reviews R1 and R2 with associations as following:

U1 -> R1,R2
U2 ->R2

Now, if you wish to remove rentalUnit U1, it will fail even with cascading on ‘delete’ present from RentalUnit->Review as described above. It fails because GORM is executing cascading on all children of rentalUnit that includes all reviews, so it attempts to delete the review R2, however. The review R2 is also referenced by another rentalUnit U2 resulting GORM unable to remove R2, and also U1 as the result.

Grails Database Migration Plugin

Grails Database Migration Plugin is very useful for managing SQL database for your grails application. The most useful aspect of it is that it can generate SQL  scripts from current GORM. In my previous job, we used MyBatis to manage Database, however. With MyBatis we had to create SQL scripts manually and to make it according to GORM was very time consuming, error prone and not efficient process.

In this space, i will cover the installation, setup and everyday use of Grails Database Migration Plugin.

Pre-Condition: SQL Db setup and installed

Install & Setup DataBase Migration Plugin

1. Installation.

To install Grails Database Migration Plugin, run the following command:

grails install-plugin database-migration
-Or-

Add the line ‘runtime “:database-migration:1.1″‘ in the ‘plugin’ code section in grails-app/conf/BuildConfig.groovy just like this:

To verify, open Dependency report (‘grails open dep-report’) and see if there DB migration plugin listed just like this:

2. Generate Initial Script

Next, we are going to generate intial migration script:

grails dbm-generate-changelog changelog.groovy

This will generate ‘changelog.groovy’ containing SQL scripts to create current GORM state and save it in ‘grails-app/migrations’ folder

Note:You may choose to run ‘grails dbm-changelog-sync’ that will record that the changes have already been applied if you don’t wish to dump tables and test this script.

3. Update Datasource

Last, we need to change the app DataSource to inform GORM that it does not need to ‘create’, ‘update’, or do anything with the Database at the application startup since we are going to use DB migration plugin for ‘create’, ‘update’ the app database. So, update variable ‘dbCreate’ in ‘grails-app/conf/DataSource.groovy’ to nothing just like the following:

Using Database Migration Plugin

Test Run

At any given time, to test Database migration plugin and scripts do the following:

1. Drop all tables in the Database.

You can do that manually or utilize Migration plugin by running the following command

grails dbm-rollback-to-date  1900-01-01

2. Roll back the record. ONLY If you drop tables manually you need to update migration record log, so that it appears no migrations scripts have run so far. To do that run the following command:

grails dbm-drop-all

If you have run ‘dbm-rollback-to-date’ as specified above, it would have done this step for you already

3. Run the Scripts and Check Database

To run all of the migration scripts call the following command:

grails dbm-update

This will run any script that is registered in the migration log as hasn’t been run. In our case, we drop all the records, so it will run all migration scripts

Note: At any time, you can check the migration plugin status of what migrations scripts have and have not been run by calling ‘grails dbm-status’. It will display only scripts that haven’t been run

Generating Migrations Scripts

After making changes in GORM all tested then run the following command:

grails dbm-gorm-diff --add filename.groovy

This will generation and register the script as well as include for the filename in the main changelog for you. You are done. Other developer will get your new script and run ‘dbm-update’ to update their database accordingly

Other Useful Migration Commands

1. dbm-clear-checksum.  You can change any Migrations scripts manually but if you do, then don’t forget to run ‘grails dbm-clear-checksum’ afterwards to make the changes valid

2.  dbm-gorm-diff without ‘–add’. If you run only ‘grails dbm-gorm-diff’ without the ‘–add’ tag, it will display what are the current difference between GORM and your database

Run Migration Scripts at Start

Its all good running ‘dbm-update’ every single time you receive new migration script in your workspace , however. You are able to make it automatic at next time your grails application start by adding  the following in the conf/Config.groovy file:

grails.plugin.databasemigration.updateOnStart = true
grails.plugin.databasemigration.updateOnStartFileNames = ["changelog.groovy"]
grails.plugin.databasemigration.changelogLocation = 'grails-app/migrations'

This will detect what changes hasn’t been run from the change log and run appropriate migration scripts all automatically at grails app startup.

Running the migration script at startup including times you making changes in GORM (domain classes) will cause conflicts between GORM and database. Learn, how to solve the problem in post ‘Grails Environment Aware Framework

Issues Encountered

1. ‘Change Set changelog.groovy::[…] (generated) failed.  Error: Error executing SQL ALTER TABLE[…]

We got this error after cloning the project and running the migrations scripts.

It turned out that this error was caused because only on Windows when you generate initial migration script it includes references to specific database name.

To solve it, we manually deleted these references in the initial migration script. Afterwards don’t forget to run ‘dbm-clear-checksum’ to make changes valid

2. ‘Error: Error executing SQL CREATE INDEX `[…]: Incorrect index name’

This error seems to be specific to Database installation,setup or version. Either way, the solution was to manually updated migration script so that this index is created after the foreign key is added

As you can see it is the same index name as marked in above screenshot. So moving the changeSet 5 in front of the changeSet 4 and then updating the numbering(red color) solved the problem for us

Afterwards, don’t forget to run ‘dbm-clear-checksum’ to make the changes valid

3. Grails Migration plugin fails silently.

Most of the time this happens when there is syntax error in one of the migration scripts. I find myself often times deleting a migrations script and then forgetting to update the initial changelog. groovy. At the bottom, the changelog.groovy contains ‘include’ for each migrations script. If you delete some migration script but fail to update (remove the includes) the changelog.groovy, then migration plugin fails silently for ‘update’, ‘status’ and other cmd.

Debugging Grails Database Migration plugin

1. Enable logging. It may help troubleshoot your issue if you enable the Database Migration Plugin by adding following line in Config.groovy:

log4j = {
           ....
           debug 'grails.plugin.databasemigration', 'liquibase'
    }

Useful Sources:

COUNTDOWN TO GRAILS 2.0: DATABASE MIGRATIONS

Working with the Grails Database Migration Plugin

Database Migration Plugin