Building SPOKEnPHOTO

Over the last year or so I consulted for a Canadian startup called Spoke Technologies. We built the free SPOKEnPHOTO service which includes native iOS apps SPOKEnPHOTO for iPhone and SPOKEnPHOTO Album for iPad that allow you to ask friends and family to talk about your photos, and for you to then share those photos and the commentary with others in various ways.

I was originally asked to help out with the server development, with an initial prototype already developed in Grails. The product was functional but very much in its initial stages at this point. We started with a small team with few processes or tools.

Over the course of the coming year I formalised the REST API with the team (of which I was the only truly remote member), and we made several refinements to the product. We introduced better processes, moved code to Github, went through Pivotal tracker to JIRA, used TestFlight for ad-hoc iOS build distribution and used Basecamp to communicate. Not all of these tools were universally popular choices, but I think in the end everyone was happy enough.

A professional design agency was taken on to develop the UI and user interactions in the client and web. At the time these skills were simply not available in-house, and if you are making mobile applications you cannot overstate the value of high quality design. The non-mobile web UI remained important to the project because of the nature of the application. If you invite people to talk about photos, they invariably do not have your application yet and may not even have an iOS device on which to install it. I also used iOS 6's neat new app banner mechanism to show direct links into the app store if you view photos on an iOS device in Safari. You can install and open the app right at that photo... although I don't believe the latter part of the URL scheme implementation is complete in the public builds of the apps in the app store.

My work on the implementation of the API was interesting because developing APIs for mobile clients is quite challenging. The biggest issue of concern is that you cannot force customers to update their installed application. You may ship a fix for a critical bug, but they do not have to or may not be able to install it. This means that you have to strictly version your API and ensure that you continue to support older API clients for those older mobile app installs out there, and you have to make sure that your new API changes do not leak through to old API client response payloads. Yet you also want to reuse your code rather than cloning the logic for each API revision and tweaking it.

The implementation of REST style protocols themselves in Grails is simple. You don't need any plugins to do this, and you don't even need built-in REST support features. The versioning is the tricky part. The solution that I arrived at consisted of:

• Having a filter extract the requested API version from /api/1.1/ style URIs and putting this into the request attributes as e.g. appApiVersion
• Making controllers pass this API version information to service calls that implement the API request, so that they can tailor their activity and response accordingly
• Having a UrlMappings.groovy for the current API URL mappings, and moving older mappings over to a LegacyUrlMappings.groovy for hygiene purposes
• Having per-API-version functional tests for all the API requests. This means tests ensure that all versions that the server should support concurrently continue to be tested

This provides for changes over time in request and response formats supported by each API version, changes to binding from requests, changes to response payload formats, changes to URL mappings, and even changes to HTTP headers and status codes used. Build robust versioning infrastructure in from the start.

It isn't foolproof, and in some ways not even that elegant, but it gets the job done well. We had several client and API revisions and no problems that I am aware of with API incompatibility. I always maintained during development that having a full suite of functional tests for the API was absolutely critical to prevent the iOS client and Grails server developers chasing round in circles trying to isolate whether the client or server was at fault. This actually necessitated me making some improvements to the Functional Test plugin 2.0 RCs to support API testing better - something for which I have not found any better solution.

Good logging at the server end was also very helpful on this front. If a client seems to misbehave and you see some dodgy data in the staging server logs, you have just saved yourself a lot of debugging pain. So add lots of DEBUG logging that includes all headers and payloads where possible, and run your staging environment with DEBUG logging all the time.

As far as REST goes, I would prefer a high level API definition DSL as I mentioned recently on the Grails mailing list, but that is something somebody else will have to work on I expect.

On the user-facing side of the website, we had to be able to show photos and albums and allow playback of the audio associated with these. We also had to allow recording of audio directly in the browser. If you've never done that before, all I can say is don't. The state of current browsers is such that the only way to do this in 2013 is still to use a Flash app to record the audio and stream it up to a media server which converts from FLV format to something like MPEG-4 audio. Our app then gets the audio file saved from by the media server and post-processes it to hook it up to the relevant database data.

User signup is handled via the REST API although there is web UI for the password change screen and admin logins. For these we use Grails Platform UI and my Fresh Security plugin which together put a friendly skin over Spring Security for the basic signup, log in and log out requirements of most sites.

Using a custom UI Set and Theme for Platform UI and GSP template overrides for forms provided by Fresh Security, we have a nicely skinned UI with minimal fuss. Fresh Security's UI elements are rendered in a manner and using CSS/JS libraries defined by our application, without us having to explicitly code this or hack styling into the Fresh Security plugin. Platform UI demonstrating its flexibility.

This was a fun project and the Spoke Technologies team are continuing to add new features and services to the product. It was a great experience to work with iOS developers and have some input on the direction of the application and service.

They are now expanding their in-house development teams and I am looking for new clients to help out.

Discussion about the future of Grails

Peter Ledbrook has a good post over on his blog on which I have also made a comment.

I think this is a critical juncture in Grails' history. If you care about the future of Grails you should write about it and tell the world, either on that post or on your own blog.

The ultimate question is: why should 100,000s of developers globally choose to use Grails over the other frameworks out there today and in the future? Yes it needs to be 100,000s for the plugin ecosystem and contributions to be viable because a tiny percentage of people actually contribute.

No longer developing Grails plugins for free

I'm afraid I've had to stop developing Grails plugins for free.

See this post for full details.

If you need some work done on some Grails plugins or other consultancy and development, please do contact me to discuss your requirements.

Email Confirmation 2.0.7 released with docs and e-book guide!

I'm pleased to announce the release of the Email Confirmation Plugin version 2.0.7 for Grails. This release fixes a series of important issues, giving much better confirmation token uniqueness and event handling.

Important possibly breaking changes:

1. The default eventNamespace is 'plugin.emailConfirmation' and no longer 'app'
2. The confirmationToken column in the database is now defined with a unique constraint. You may need to write a migration.

It also now comes with reference groovydocs and a cheap pay-for guide e-book. I spent some time writing some high quality in-depth guide documentation that I hope you will find worth paying for.

Register your interest in the e-book Mastering the Grails Resources Plugin

I've started planning a comprehensive e-book on the Grails Resources plugin. The plan is that if there is enough interest for this to be financially viable, I can begin bashing this out in January 2013 and it shouldn't take me too long to complete.

I'm using Leanpub.com to do this, so that I can gauge interest before I actually start writing.

To get more details and register your interest and how much you would consider paying, please visit the Leanpub page.

If there is insufficient interest I am unlikely to write the book, so this will be another intriguing experiment into the viability of ongoing unpaid open source development.

Let's hope it goes better than support subscriptions!

Free cheat sheets for Platform Core & UI

As part of my recent talks at GGX 2012 on the new Platform plugins, I created a couple of cheat sheet PDFs to help people get for the size and shape of the plugins, as well as being a quick reference.

You can download these from these shortened links:

bit.ly/platform-core-cheat

bit.ly/platform-ui-cheat

I hope you find them useful.

Platform UI Theme Challenge Extended with new GR8conf Ticket prize

I've decided to extend the Platform UI Theme challenge, because I want to make sure that we get at least as many entries as there are prizes, with an earliest closing date now of 25th December 2012.

This is announcement is particularly important because after being impressed by Platform UI during my [talk at GGX 2012, Søren of GR8conf has very generously added a free two-day GR8conf Europe 2013 ticket (excluding university day) to the list of prizes!

Please read the full original post for details of what is required for entries, and link to docs & tools required. However the prizes will now be as follows:

A total of five users will be be chosen.

• The top prize is now the two-day GR8conf ticket.
• The second prize is now the complete A Book Apart library.
• All five winners will receive a Grailsrocks t-shirt, upgraded to a Grailsrocks Hoodie if their entries meet the criteria for this.

We look forward to seeing what you can come up with!

Updated Platform Related Plugins Released

I've released a slew of plugin updates in the last 24 hours to support fixes in the new Platform UI 1.0.RC2.

In summary:

• Platform UI 1.0.RC2 - Improvements to theme/ui set previewers, debug view of discovered themes + ui sets + their GSP templates
• Platform Core 1.0.RC2 - Fixes for Navigation API with > 1 navigation DSL in app, improved Navigation debug UI
• Bootstrap UI 1.0.RC3 - Navigation rendering improvements, Carousel fixes
• Bootstrap Theme 1.0.RC3 - Carousel related fixes
• Resources 1.2.RC3 - Fix for NPE at startup when there are dependencies on undeclared modules

I've also updated the Platform UI Showcase app on github, so it now has the Grails Wrapper built in - just clone it and ./grailsw run-app to try it, follow the README on how to then add Bootstrap theme etc.

Platform UI docs are here The UI Showcase/test app source is here

We're in the last week for the Theme competition with real prizes. You have until December 10th to submit entries. Please do spend a lunch break or two working on a Theme to enter if you possibly can. I need the support of the community for this endeavour to take off. For example, a port of Rob Fletcher's existing HTML5 bootstrap scaffolding styling to a Theme and UI Set would be a great starting point.

I've had no indication of any entries so far, so it looks like I'll be sitting on the prizes :)

The Platform UI Theme Challenge begins today!

CLOSING DATE: DECEMBER 10th 2012

Extended and updated see competition prize update

As you may have seen, I finally managed to release Platform UI plugin 1.0 RC1.

I believe this plugin holds massive potential to take the Grails plugin ecosystem to an entirely new level. This exciting opportunity rests on the availability of high quality Themes and UI Sets. Grails application developers will be able to use these to get amazing looking apps with close to no effort. This in turn will make Grails an even more attractive platform for development.

A great parallel for this is the Wordpress theming community. If you use Wordpress you can find a free or premium Theme (typically under $50) that instantly makes your site look great - and if you think this just means Blog styling you are very out of date. Wordpress Themes adopted ad-hoc conventions shared across Themes by the same author, and sometimes across authors, for tags to render buttons and other content elements. The bottom line is you can hit the ground running without hiring a designer for two weeks, or you can have a designer focus on only the brand-specific tweaks that you need.

Themes just define the layout, right? Wrong. As Wordpress themes have shown, there is huge potential for themes to include content-related functionality also. This might include social sharing buttons, responsive and/or mobile rendering, smart image rendering using light boxes, Tweet panels etc.

Good examples can be found here.

The challenge

So what is the challenge?

It's simple: create a high quality Platform UI Theme and release it as a plugin before December 10th 2012.

I will pick five winners. These will be the most professional and most feature rich entries.

One Theme will be singled out as the supreme champion and will win the entire A Book Apart Library in either ebook or paperback form. For those not aware, these excellent books are by web luminaries such as Mike Monteiro, Jeremy Keith and Karen McGrane.

All winners will receive an extremely rare Grailsrocks t-shirt and the results will be announced in my Platform UI talk at GGX 2012, and some of the winning themes may be featured.

There's more! All winners can upgrade their prize by: creating their theme using a new UI Set created for a CSS framework other than Bootstrap and/or including a fully themed set of scaffolding templates.

These winners will receive a rare Grailsrocks hoodie in addition to the t-shirt. Such a win will obviously benefit two-person teams who might otherwise have to timeshare the t-shirt and hence have to normalise their body shapes.

Why the upgrade? Themes alone are pretty easy, but UI Sets and scaffolding are more involved and the more choice we have in the ecosystem the better.

How to enter

Step one: Start a plugin project using Platform UI 1.0 RC1

Step two: Make an awesome theme

Step three: Test it in a real app

Step four: Release the plugin to the Grails plugin portal

Step five: mail marc@grailrocks.com the links to the plugin portal page and include your screenshot (see rules)

By its nature this competition is about high quality design. So you if you want to enter you may consider teaming up with a designer and splitting the rewards.

The rules

• You can submit as many entries as you like
• All entries must be free open source in the Grails portal tagged "platform-ui"
• All CSS and JS and related code must be open source licensed (ASL 2 or compatible)
• All entries must supply a 1024px wide screenshot minus browser chrome for each theme included
• Theme plugins must be called xxxxx-theme and ditto UI Sets must be xxxxx-uiset
• Themes must support all required Platform UI Theme layouts as per the documentation, use of Platform Core Navigation API for menus, and an implementation of/compatibility with all the UI tags in the UI Set they depend on

What we'll be looking for

Awesome design. Great typography. Contemporary look. Design appropriate for real world applications.

In addition I will be looking out for:

• static and responsive variants
• mobile themes
• multiple colour schemes
• an extra "user" menu for e.g. Sign in/out and profile
• retina support

Tips

Please use grails-user list to get support from me and others.

Look at ThemeForest and similar off-the-shelf Theme sites for inspiration.

Perhaps look at converting high quality free themes that already exist (all credits must be retained and open source licensing apply).

Consider bundling multiple theme variants in the same plugin to maximise your chances, as well as gaining through reuse of shared theme templates and UI Set.

Use Platform Core Configuration API to parameterise themes,

Platform UI 1.0 RC1 Released

I'm excited to announce that, finally, Platform UI 1.0 RC1 for Grails is available as a public release.

Platform UI is an ambitious project that provides a simple API for UI elements independent of CSS and JS frameworks, as well as a Theme API for off-the-shelf great looking app UIs. This is not a monolithic "Grails UI" style UI plugin. Its the framework for the basic UI elements used in most web apps, without any specific implementation other than the tags themselves which remove a lot of the boilerplate for people who actually implement the elements. Ever looked at the code behind g:paginate to consider changing the way it writes out the markup? Tip: don't. Platform UI abstracts the rendering for elements like this.

As you might imagine this is quite a big project and I can't do it all alone. The current specimen Bootstrap Theme available is merely a hacked together proof of concept, but the building blocks in Platform UI are pretty much there. There's still some polishing to do in terms of UI element contracts, but that's why this is an RC1.

What we need now is for people to play with this and build some real high quality Themes and UI Sets (the UI elements). I'm trying to make this as easy as possible for people, but there will be more improvements to be made in future for certain. I have written some reasonably comprehensive docs which are available now.

Right now you can also see the "showcase" application source for details of usage and as a harness for testing your own themes on github.

If you clone that code and run it you'll get an ugly UI. If you then install the bootstrap-theme plugin, you'll get a less ugly UI:

grails install-plugin bootstrap-theme

Just run the app again and as if by magic you have a better UI and Theme on the same app. Imagine if that was a professional quality theme and not something hacked together quickly by me as a proof of concept!

The bootstrap-theme and its accompanying bootstrap-ui (UI Set) plugin are intended to be the starting point for future Themes. Bootstrap UI should be reusable for all Bootstrap based themes, but the theme itself is just for demo at the moment.

I'll be launching a little challenge to motivate people to get involved in this very shortly.

If you are interested in Platform UI but cannot play with it at this time, you may be interested in my talk on this subject at GGX 2012

Platform Core 1.0 RC1 Released

We're pleased to announce that Platform Core plugin 1.0.RC1 has been released.

This release marks the lead up to 1.0 final, with no new features being added until after 1.0 final. The main changes in this release are miscellaneous bug fixes and consolidation of i18n resolution strategies in the UI extension tags.

Sadly due to time constraints with our day jobs we've had to postpone the work on the APIs for Injection and Conventions.

However with GGX 2012 looming large we need to get stable final code out there and to prepare our talks.

There will be at least two talks at GGX 2012 on Platform Core and one on Platform UI.

Support subscriptions currently unavailable

I've suspended support subscriptions for the moment. The costs of taking subscriptions are currently too high for it to be viable to continue offering them.

If you would like commercial support for my Grails plugins or consultancy/development please contact me.

Grails Retina Plugin 1.0 released

I have just released version 1.0 of the Retina plugin - a simple Grails plugin to help with rendering inline images with Retina support. This is a non-trivial area in which no single clean solution exists yet.

Site decoration, sprites, textures etc. that are currently rendered with CSS background-image are easy to handle without a tool like this plugin - just use media queries or any of the other retina CSS background-image techniques. (Don't get me started on how this is a total fail for web standards)

The inline image solutions are more tricky. You have concerns over whether you end up loading low and high resolution versions, flash of page with no inline images, and problems like URLs not being known until page build time i.e. supporting third party Avatars with retina resolution, or showing high resolution photo thumbnails in a gallery. Do you really want your contact sheet to download 100 low resolution images first, then another 100 high resolution?

Currently available techniques include but are not limited to:

1. CSS background images using WebKit image-set. Requires rendering inline styles in your page and using <span> instead of <img> so it breaks semantics. However it auto-adapts to the display in use in multi-screen setups, and loads instantly as the page is parsed. Best user experience (except for accessibility)
2. <img> tags that load the low resolution image (or none) at first, then have their src replaced by JavaScript code after the page loads. Semantic but means page takes longer to look complete. Best semantic HTML
3. Serving images automatically using the correct resolution, depending on a cookie set in the client by JS to indicate whether retina is supported. Requires server-side smarts and arguably has issues with violating HTTP specification by returning different documents for the same URI. Best developer experience

This retina plugin currently supports approaches (1) and (2). You choose which is most appropriate for you for each image you render. Both involve compromise.

However it works, today. We're using it in the imminent next release of NoticeLocal.

Usage is simple. Instead of <img> or <r:img> you simply use the new <img:set> tag:

<!-- Using default mode, CSS background image -->
<img:set uri="/images/products/main.jpg" 2x="/images/products/main@2x.jpg"/>

<!-- Using JS mode -->
<img:set mode="js" uri="/images/products/main.jpg" 2x="/images/products/main@2x.jpg"/>

This simplicity belies some cool if slightly ugly things going on behind the scenes.

The CSS background-image mode (mode="bg", the default) requires Resources plugin 1.2.RC2 as a minimum. This is because it uses a new feature, <r:stash> to store the small chunk of CSS required to set up the background-image, so that this can be rendered cleanly in a single chunk in the <head> section of your page. This is like using <r:script> but supports JS or CSS (and maybe more in future).

The JS mode is simpler but still leverages the power of Resources plugin in order to pull in the small chunk of JS required, and the dependency on jQuery, in such a way that it loads at the right time and can coexist with your own resources.

Yes, the plugin requires jQuery and HTML5 currently. Perhaps we can rewrite the JS later to not require jQuery. However if you don't use jQuery you could always override the resource dependency on jQuery or override the imagesets.js resource to provide your own implementation. Or just fork, or whatever. Its not a huge amount of code.

Grails Resources Plugin 1.2.RC2 Released

I'm pleased to announce the release of Resources plugin 1.2.RC2. It is strongly recommended that you update your apps to this release.

This second release candidate for RC2 fixes a slew of important issues, in particular:

• More fixes for correct module dependency ordering for cases where dependsOn is called with modules in different orderings throughout your definitions
• Fix for the leak of ad hoc resource dependencies across requests (r:script fragments)
• Improvement to CSS processing to support resource that have absolute URLs e.g. after processing with "baseurl" mapper

There are a bunch of other tweaks, decisions and merged pull requests

I would like to thank everybody who has raised issues, and especially those who provided example apps and/or pull requests. This is much appreciated.

Thanks everybody for your patience with these important fixes. I only wish I had more time to work on these plugins.

Platform-Core 1.0.M6 released

Hot on the heels of the Grails platform-core 1.0.M3 release, we have had no less than 3 emergency releases. Due to painting ourselves into a corner with the milestone versioning, these sadly have resulted in M4, M5 and M6 releases. We'll be trying to avoid this kind of thing in the future. So the new platform-core 1.0.M6 release incorporates the following improvements since 1.0.M3:

• Added the missing bootstrap JS file needed for the /platform/ UI to work
• Event DSL artefacts in plugins automatically set the event namespace to pluginNameInBeanCase if no namespace is provided.
• Events sent to undeclared namespace/topic pairs will result in an exception (so no more "why the hell isn't my listener being triggered" when you use wrong namespace)
• Improvements to p:text tag - support for overriding the i18n prefix using scope/plugin attributes just like p:textScope, but one-off.
• Improvements to p:label, p:button and p:smartLink tags to accept textScope and textPlugin attributes to do the same as above, and for body to be used as default value if supplied
• Bug fixes in p:button relating to submit buttons
• Improved /platform/ dev view for showing plugin config - shows all plugin related config now.
• Fixed namespace of i18n strings for security button tags
• Improved output in /platform/showUiExtensions, added more examples of p:button, p:label, p:smartLink and p:text

This of course means that hopefully the next true milestone will be M7 and will include the promised Injection API and events/config improvements.

Install dependency: compile ":platform-core:1.0.M6"

Source: http://github.com/grailsrocks/grails-platform-core

Docs: http://grailsrocks.github.com/grails-platform-core

Platform Core 1.0.M3 Released

We just released the Platform Core plugin version 1.0.M3.

This release was to be called M2, but due to a slip of the fingers there was a previous non-final M2 release so we had to roll it over to M3.

In this release we finally have the public release of the fantastic Events API from Stéphane Maldini, as well as a public release of the Navigation API as well as a slew of of great improvements and new features relating to UI and i18n.

Best of all, we have a lot more docs.

Some highlights: async and sync event bus and DSL, new automatically namespaced plugin session, request and flash attributes, and smart i18n message resolution.

All of these improvements are available right now in the Grails plugin repository and the docs are available now at http://grailsrocks.github.com/grails-platform-core/.

We're really excited about this release as all the key APIs are there to facilitate a whole load of other releases in the pipeline; events-si (Spring Integration), events-push (websockets and SSE), platform-ui, fresh-security and updates to email-confirmation, invitation-only and others!

Enjoy!

Hooking up Platform Core Security API to your Security Provider

So, now that the abstracted Security API of Platform Core is publicly available, it would seem a good time to show an example of how you would hook this up to a security provider. The Platform Core Security API provides methods and tags that give plugins and your application access to  hitherto painful to obtain information about the currently logged in user and security UI functions like logging out. The beauty is that you have a single API to call in to, no matter whether the application author is using Spring Security, Shiro or any other authentication provider.

In future it is likely most if not all security plugins for Grails will include the bridge required to do this, but its early days so we have to roll this ourselves at the moment.

It is however very simple. Let's write one for Spring Security. The steps required are:

1. Create a bean that implements the SecurityBridge interface of the Security API
2. Register this in the bean context as "grailsSecurityBridge"
3. There is no step 3

So here's a sample implementation of the SecurityBridge using Spring Security:

https://gist.github.com/2225545

Notice that you would need to change this to point at your own auth controller.

After that you just need to register this in your application context as the "grailsSecurityBridge" bean. For an application this means a declaration in resources.groovy, for a plugin this means a declaration in doWithSpring. Both would look something like this:

https://gist.github.com/2225569

Next you run your app and start using the Security API tags and methods with impunity!

Platform Core Plugin Released

I'm pleased to announce the release of the first 1.0.M1 milestone of Platform Core plugin for Grails. Platform Core is part of what was previously referred to as Grails Plugin Platform. We decided to split out the UI parts into a different plugin, and we will call it Grails Platform in future as most of the functionality is also for use by Applications.

The core part provides "glue" APIs that make plugin development easier and makes it easier for Applications to integrate plugins - across Grails 1.3 and 2+ releases.

This release is intended to provide early access to a reasonably stable subset of the APIs.

This code is already in production in some applications.

What is in the 1.0.M1 release?

This release includes code for a range of APIs, but only a subset of them are currently considered stable. We have not documented the unstable APIs for this reason, and they are not for public use at this time.

The APIs that are intended for use now are:

• Configuration API for plugins to alter application config, to declare and validate their own config options
• Security API which provides a standard set of tags and utility methods that are independent of the underlying security provider
• UI Extensions in the form of some useful UI-related methods and tags for rendering display messages, smart links, buttons and company branding info.

See the documentation.

The APIs that are not intended for use and are considered incomplete currently include; Navigation, Conventions, Injection and Events.

However, much work has been done on these and they are internally already in use to implement the Configuration and Security APIs.

Platform Roadmap

The Platform also encompasses Platform UI APIs for standardizing UI widgets and Theming. Here is the current release roadmap:

platform-ui 1.0.RC1 - circa 10 April This will make the Theme and UI Set APIs available, with a default Grails theme

platform-core 1.0.M2 - circa 10 April Stable Events and Navigation API

platform-core 1.0.M3 - circa 24 April Stable Injection API

platform-core 1.0.M4 - circa 10 May Stable Conventions API

platform-core 1.0.RC1 - circa 30 May platform-ui 1.0.RC1 Bug fixes and prep for final release of both

We cannot commit to these dates - they are just estimates - but most of the work is done so with luck we can hit or beat them.

This plugin is the result of a collaboration between myself and Stéphane Maldini.

New plugins added to Grailsrocks support

I'm pleased to announce that as of today the Platform Core and Rocks plugins are included in the list of Grailsrocks supported plugins. This means that all existing Grailsrocks support customers and all new customers will qualify for support on these plugins at no extra cost.

Expect the portfolio to broaden further in the coming months!

Thanks for all your support so far.