Posts tagged ‘webservice’

It has been quiet for a while, but with good reason. Over the last 2 months I have been travelling a lot. And with a lot I mean Schotland, India, England, USA and England again, all for business. And the way it goes is that by the time you get back to the hotel from your appointments you have loads of email from the office waiting for you. With all that travel I had a grand total of one day off in India and 2 in the USA, which I spend away from the computer.

But now that I am back I have started to tie up some of the loose ends for the ForumClient for the Adobe forums. First and foremost, I have made it portable so it can now also access the Jive forums. The main reason for that is that it offers me another server to run tests against so that I can more easily determine whether issues are between the keyboard and the chair or if they are real server issues. Unfortunately with the number of bugs in the server software and the lack of documentation this is a real necessity. Most interesting for users is probably that forums, thread and messages now have right-click menu’s to mark as (un)read and that I squashed most of the bugs in the counts of unread messages. And I have started some work on getting messages to display better by adding some CSS to the message display.

Last but not least, at MAX I sat down with some people from Adobe and we had a good discussion on some possible future directions. One of those is a Flex version for mobile users (try the current forums on a mobile to see how badly that is needed) which Adobe would need to support by publishing a cross domain policy file. Second we had some discussion on the consequences of making this Open Source. I have decided that I will be publishing the sourcecode for this client at some time in the future. No definite timeline, but it won’t be beforea new version of ClearSpace has been deployed for the Adobe forums.

Download version 0.1.0 and give it a try.

Since I am in Bangalore for a training I dropped in on the “Adobe Flash Platform Tools Preview” this evening. The agenda promised short sessions on Flash Builder 4, Flash Catalyst and LiveCycle ES, followed by food and networking. The Flash Builder (previously known as Flex Builder) session was solid. It showed Data Centric Development, where services are defined in Flash Builder and can then easily be wired into the UI because all the code for the services and value objects is generated. (See Raghu’s blog for the demo screencast). Next up was Flash Catalyst, showing a design - development workflow where a .psd file was transformed to a .fxp, which was then imported in Flash Builder to wire the data in through the new services management. Last was LiveCycle ES. Unfortunately, but understandably considering the audience, this was all about data management and not process management. What was new for me was that apparently this now wires directly into Hibernate so you don’t need to write any server side code anymore, you can have everything generated.

The Q&A focused mainly on the designer - developer workflow with Flash catalyst. The main question that was repeated several times in different words was whether this workflow put any additional constraints on the designer. And each time the answer was that good development on the design side, including the judicious use of layers, was all it took. I think this reflected the audience of architects and project managers, from developers I would expect more technical oriented questions.

Afterward the food and networking were great. Not just the Flash team from Adobe was there, but also people from the LiveCycle team and the ColdFusion team., so I got an opportunity to thank some people in person for fixes and new features I am not allowed to mention yet. And I also met up with some of the people we do business with in India.

As promised my hacks to get past the the quirks in the webservice API for the Adobe forums. I am deliberately not publishing the full application, the lack of local caching in it makes it more of a DOS tool then a forum client.

Authentication

The PermissionsService authenticate method doesn’t work since the Adobe forums do not use the standard Jive login methods, but a custom Adobe SSO login method. To get a login on the forums from AIR replay what a browser does when logging in to the forums. First visit the index page of the forums to GET a few cookies, then POST the credentials to the Adobe authentication server, then GET the index page of the forums again to allow the forums to do a call back to the Adobe authentication server to collect the user profile. So all in all it takes 3 HTTP requests to log on.
I have included the code snippet that my AIR app uses to log in to the forums below for those who wish to experiment with it. Be warned that the full sequence takes on average 15 seconds.

?View Code ACTIONSCRIPT
// General variables
private const _AdobeAuthURL:String = "http://www.adobe.com/cfusion/entitlement/index.cfm?loc=en&e=ca&returnurl=http%3A%2F%2Fforums%2Eadobe%2Ecom%2Flogin%2Ejspa";
private const _ForumRootURL:String = "http://forums.adobe.com/index.jspa";
 
private function startLoginSequence():void {
	writeLog("Starting login ");
	getForumRoot(preAuthResult);
}
private function getForumRoot(resultFn:Function):void {
	// just GET the forum root to collect cookies
	var forumRootService:HTTPService = new HTTPService();
	forumRootService.method = "GET";
	forumRootService.url = _ForumRootURL;
	forumRootService.useProxy = false;
	forumRootService.resultFormat = "text";
	forumRootService.addEventListener(FaultEvent.FAULT, faultHandler);
	forumRootService.addEventListener(ResultEvent.RESULT, resultFn);
	forumRootService.send();
}
private function preAuthResult(event:Event):void {
	// We have now collected the forum cookies, log in to the Adobe ID
	login();
}
private function login():void {
	// login does a login request to the main Adobe site
 
	// credentials Object with all name value pairs
	var credentials:Object = new Object();
	credentials['returnURL'] = 'http://forums.adobe.com%2Flogin.jspa';
	credentials['up_login'] = 'yes';
	credentials['ignore_email_validation'] = 'yes';
	credentials['up_username'] = "spam@vandieten.net";
	credentials['has_pwd'] = "true";
	credentials['up_password'] = "sihtyrt";
 
	// login Service
	var loginService:HTTPService = new HTTPService();
	loginService.method = "POST";
	loginService.url = _AdobeAuthURL;
	loginService.useProxy = false;
	loginService.resultFormat = "text";
	loginService.addEventListener(FaultEvent.FAULT, faultHandler);
	loginService.addEventListener(ResultEvent.RESULT, loginResult);
	loginService.send(credentials);
}
private function loginResult(event:ResultEvent):void {
	// check if we are really logged in
	var loggedInTestString:String = '<a href="http://www.adobe.com/cfusion/membership/logout.cfm">Sign out</a>';
	if (event.result.indexOf(loggedInTestString) != -1) {
		// Login success
		// Do a new HTTP request to the forums to propagate the login from Adobe to Jive
		getForumRoot(postAuthResult);
	} else {
		// Login failure
		throw("Username password combination incorrect.");
	}
}
private function postAuthResult(event:ResultEvent):void {
	// Fully logged in, ready to use the API
	writeLog("Login complete");
 
	// extractUserDetails(event.result);
}

Once we are logged in we can use all of the APIs to get the actual useful information from the forums. The FlexBuilder WSDL import tool works quite well with the APIs, all cases where it failed turned out to be mistakes in my programming. To get stared call getRecursiveCommunities of the CommunityService to get a list of all the communities (forums) available. Depending on how busy the forums are, internet bandwidth, traffic and the position of the moon this can take between 25 seconds and 5 minutes. From the list of forums you get you can drill down into the list of threads (ForumService getThreadsByCommunityID: up to 50 seconds to get the thread list of the DreamWeaver forum, the busiest forum with about 50K threads) and then into the list of messages per thread (ForumService getMessagesByThreadID: usually less then a second). When you get the messages you will also get the users and all things you need to display a tree of who responded to who.

Getting your own userID

Apart from the login methods in the API there appears to be another problem in the webservice API (or maybe just something I haven’t figured out). Searching for a user, either by his username or by his emailaddress, has never returned any results for me. (I do in fact get a 500 Internal server error when I try to use the UserService getUserByUsername.) But the user details are needed for all API methods to post messages. The workaround I implemented at last for that is to just do a string lookup in the HTML of the forum start page (the commented out call to extractUserDetails in the previous code snippet):

?View Code ACTIONSCRIPT
public function extractUserDetails(pageObj:Object):void {
	var page:String = pageObj as String;
	var userIDPreString:String = 'quickuserprofile.getUserProfileTooltip(';
	var userIDPreStringIndex:int = page.indexOf(userIDPreString) + 39;
	var userIDPostString:String = ')';
	var userIDPostStringIndex:int = page.indexOf(userIDPostString, userIDPreStringIndex);
	_userID = page.substring(userIDPreStringIndex, userIDPostStringIndex) as int;
}

Obviously depending on visiting some sites in a certain order to pick up cookies along the way and scraping generated HTML is rather fragile. The smallest change in the HTML or the authentication will break any application that uses this. Ideally Adobe would implement some authentication webservices on its own site to facilitate logging in from an application.

With this the basic services to list forums, retrieve threads, read messages and post your own messages can be accessed without further surprises. I intend to continue working on my ForumClient, but it will be a while. I will need a proper design for the application, I am thinking about modules to support different forum software API’s, storing configuration data in XML, message caching in a SQLite database per configured site etc. Then I need to develop the whole thing. And in the mean time real life is catching up and I am going on a training tour of Europe, so I will have little time for the next three weeks. Drop me a line if you are interested in helping out, or don’t wait for me and just get started on your own client :)

I managed to get a proof of concept AIR client for the Adobe forums running. It can log in, browse communities, threads, and messages and can even reply. But since it doesn’t do much with local storage yet it doesn’t remember which messages are already read. I’ll post some more on the hacks to log in to the forums soon, but for now I have screenshots.

ForumClient Screenshot

ForumClient Screenshot

ForumClient screenshot

ForumClient screenshot