my workflow to deploy an application to appengine using git

I've been using git for nearly 2 years. I love it. It's so extensible that you can easily map it to your own workflow.

I've recently been working on a contextual gadget for Gmail. The problem with contextual gadget is that it's not easy to test it outside of Gmail. So you need to have an easy and quick way to deploy your app. I wanted also to have a way to easily manage my staging and production environment. For this purpose I wrote a script using fabric that is making it easy to manage the different versions (staging, prod and their history). I'm now using it for all my appengine deployment.

For the staging environment, I'm just using a staging version of the application with App Engine Mulitenancy. In some case, you might want to use a totally different application, It's easy to change.

so here is a typical use :

$ git commit -m "blah blah"
$ fab deploy # this is tagging the version and deploying to staging
$ fab last_tag # this give me the last tag deployed on staging
$ fab production last_tag # this give me the last tag deployed on production
$ fab production deploy:staging-2010-11-15.015 # this deploy the version tagged staging-2010-11-15.015 to production and tag this version with a production tag

If you want to play with it as well, check it out on github and fork it :

By default GateIn come with an implementation of JAX-RS(JSR-311). You can write your REST services in java using this spec, and deploy your jar in GateIn. But at eXo, system admins don't let us access to the filesystem of the GateIn servers in the cloud, so it's not an option. But we can write our services in groovy and deploy them in the portal without having to restart the portal or access to the filesystem, and this, admin sys let us do it.

UPDATE: This is no more needed since GateIn 3.0-GA

In it's Beta5 version, GateIn doesn't come with the needed configuration to be able to deploy groovy REST services.

• Download GateIn (I'm going to use the tomcat version on this article)
• Start it a first time to decompress the war and stop it (./bin/gatein.sh run)
• Add the jar exo.core.component.script.groovy-2.3.0-Beta05.jar to the lib directory of your tomcat
• Edit repository-configuration.xml (webapps/portal/WEB-INF/conf/jcr/repository-configuration.xml) to add a JCR Workspace to store the files and name it "groovy-rest" (if you don't want to do it yourself, copy this file: repository-configuration.xml)
• In the file "webapps/portal/WEB-INF/conf/jcr/jcr-configuration.xml", at the end of the file after the tag "" insert(if you don't want to do it yourself, copy this file: jcr-configuration.xml):

  <component>
    <type>org.exoplatform.services.jcr.ext.script.groovy.GroovyScript2RestLoader</type>
    <init-params>
      <object-param>
        <name>observation.config</name>
        <object type="org.exoplatform.services.jcr.ext.script.groovy.ObservationListenerConfiguration">
          <field name="repository">
            <string>repository</string>
          </field>
          <field name="workspaces">
            <collection type="java.util.ArrayList">
              <value>
                <string>groovy-rest</string>
              </value>
            </collection>
          </field>
        </object>
      </object-param>
    </init-params>
  </component>

• Remove the directory temp to force the creation of the new workspace: rm -rf temp/

Your GateIn is now ready to deploy groovy REST service.

Let's write our first REST service called test.groovy:

// simple groovy script
import javax.ws.rs.Path
import javax.ws.rs.GET
import javax.ws.rs.PathParam

@Path("pub")
public class HelloWorld {
  @GET
  @Path("helloworld/{name}")
  public String hello(@PathParam("name") String name) {
    return "Hello $name\n"
  }
}

To push this script, we simply have to do a POST on our server. We want our script to be loaded automatically so we add "autoload=true". Here is how to do with CURL:

$ curl -u root:gtn -X POST -F "file=@test.groovy;name=test" -F "autoload=true" http://localhost:8080/rest/private/script/groovy/add/repository/groovy-rest/test.groovy
$

the URL is composed as :

• http://localhost:8080: The url of our server
• /rest/private/: The private access of the REST API
• script/groovy/add: The name of the webservice we acces
• repository/groovy-rest/: our repository name followed by the workspace name
• test.groovy: The path to our groovy service

You can test your service:

$ curl http://localhost:8080/rest/pub/helloworld/jeremiHello jeremi
$

the URL is composed as :

• http://localhost:8080: The url of our server
• rest: The public access of the REST API
• pub: The @Path of our class
• helloworld: the @Path of our function
• jeremi: the parameter of our function

To undeploy your service:

$ curl -u root:gtn -X POST http://localhost:8080/rest/private/script/groovy/load/repository/groovy-rest/test.groovy?state=false

If you want to make it easier to manage your groovy files I wrote a simple wrapper around the REST API used here: gatein-groovy-tools. To update your code:

$ exo.groovy update
PUSHING: test.groovy
$

Resources:

• Download GateIn
• gatein-groovy-tools (Download)
• Groovy Scripts as REST Services
• Download CURL

I needed a simple way to proxy an HTTP request and add an authorization header to it, so I created a REST service using JAX-RS and using groovy to call the server. I put the code on our eXo server thanks to eXo IDE, and click deploy and that's it :)

If you need something similar, here is the code:

import javax.ws.rs.Path
import javax.ws.rs.POST
import javax.ws.rs.FormParam

@Path("proxy")
public class Proxy {
  @POST
  @Path("basic_auth")
  public String basic_auth(@FormParam("url") String url, @FormParam("login") String login,
                      @FormParam("password") String password) {
    def encoded = "$login:$password".getBytes().encodeBase64().toString()
    def c= new URL(url).openConnection()
    c.setRequestProperty("Authorization", "Basic $encoded")
    return c.content.text
  }
}

To test it:

curl -d "url=http://TheUrl.com/test&login=my_login&password=my_pwd" \
http://YourDomain.com/rest/proxy/basic_auth/

I published a new set of gadgets for Sonar on Github. You can add them to your GateIn or Jira Dashboard. This gadget does not need you to deploy anything else to work. You can directly install them from Google AppSpot.

Technically, nothing complex so it's a good example to look at the code to learn how to integrate third party application. Sonar has a simple webservice API that made it easy to reuse the code between the gadgets. The only thing I did not find how to do is how to know when the tendency of a metric should be red or green.

I created a file called sonar.js that is reused among all the gadgets. The sonar object simplifies the call to the API. It sends a request to the API of sonar through the gadget proxy to get the data:

sonar.GetMetrics = function(metrics, callback, extras) {
    var prefs = new gadgets.Prefs(), params = {}, url, key;
    url = prefs.getString("sonar_url") + "api/resources?format=json&resource=" + prefs.getString("sonar_project") + "&metrics=" + metrics + "&time=" + new Date().getTime();

    //if there is any extra parameters, we add them to the URL
    for (key in extras) {
        url = url + "&" + key + "=" + extras[key];
    }

    //We want to get the result as a JSON object
    params[gadgets.io.RequestParameters.CONTENT_TYPE] = gadgets.io.ContentType.JSON;
    //We send the request
    // It will call the function callback when the data will be loaded with the result in parameter
    gadgets.io.makeRequest(url, callback, params);
};

After, the callback fill-in an html template with the data:

function update_metrics() {
    sonar.GetMetrics("ncloc,lines,classes,packages,functions,accessors", function(res) {
        var resource = res.data[0];

        //Add the information about the project in the template
        $("#resource_name").text(resource.name);
        $("#resource_description").text(resource.description);

        for (var i = 0; i < resource.msr.length; i++) {
            var metric = resource.msr[i];
            //update the values in the template
            $("#m_" + metric.key).text(metric.frmt_val);
        }
        gadgets.window.adjustHeight($(document).height());
    }, {includetrends: "true"});
}

I did not implement all the possible gadgets for sonar, I invite you to take the code, and create your own. You can reuse sonar.js that handle the request to the sonar server and the preferences. Have a look at the gadget of the code overview, it's the easiest one.

Sonar Gadget in GateIn Dashboard:

Sonar Gadget in Jira4:

Links:

• test the gadgets
• Download
• Fork the code on github (and send me pull request to integrate your updates)
• Gadget specs

I's been nearly a year since the last time I build eXo Portal (now GateIn). The Building process is way easier now!

Get the sources:

$ svn co http://anonsvn.jboss.org/repos/gatein/portal/trunk/ portal

Follow the instructions of the README.txt.

You will need to :

• download tomcat
• install maven 2.2.1

I had error during the build:

 Embedded error: Error while executing the compiler.
 Java heap space
 

Just increase the memory to 512 (or 1024):

$ set MAVEN_OPTS=-Xmx512m
 

Go grab a coffee (or multiple), the build is long, very long (specially with my Chinese connection).

when the build is finish, the tomcat is in "packaging/pkg/target/tomcat/"

just type :

$ cd packaging/pkg/target/tomcat/
$ ./bin/gatein.sh run
 

and go to http://localhost:8080/portal/.