Two weeks ago Adobe informed us we could preview and test the new version of the online community forums. At first glance these forums offer an exciting new look with many new features. A single login system where we can use our Adobe ID to log on to the forums, RSS support, an improved search and room for attachments are great new features.
But when I started digging at what features the forums offered the excitement pretty soon turned into a feeling of unease. The old forums were an open system, accessible through NNTP, which provided two way interaction that allowed people to run their own interface and program against the forums through the NNTP ‘API’. Yet the new forums appeared to be a closed system following the old ‘ thou shalt watch me the way I want’ ways. So the questions that pretty soon bubbled up were not of excitement. Is this all? How am I going to work around this layout? How am I going to work with this workflow? And then the feeling of unease turned into a feeling of disappointment.
The new forums are state of the art. The web interface is so much better then the previous one. And yet they are a disappointment. Because they may be state of the art, but they do nothing new. Nothing we haven’t seen before. Nothing to inspire us. Adobe is merely catching up, not leading the way. And in the process Adobe is throwing away the one open interface I had to program against the forums.
You see, I am an avid NNTP user. When I start my client, it automatically goes out on the web to collect all new messages I follow. That is currently 27 newsgroups (on the Adobe, Macromedia, prerelease and other servers) with on average 40 message per day each. And when these messages are collected, they are sorted, filtered, grouped and prioritized for me. Messages get filtered out if I don’t like the sender or subject, people that post a direct response to me get priority, threads I have hidden before won’t resurface, etc. All of that takes maybe 20 seconds, and then a thousand messages have been reduced to two dozen priority messages from people who are waiting for me and have responded to me, a bunch of messages I may find interesting, and the rest that I don’t find interesting is gone. Now how is that going to work with these new forums? What is the closed system of these new forums offering that allow me to automate my workflow to the same level as NNTP offers? And I know that many other NNTP users struggle with the same questions.
Now I have done some research into possible solutions. And surprisingly enough the answer is really, really simple. It took me two hours to build a system that converted the email feed from the new forums to NNTP. All it takes is a subscription on all forums and a small script to convert email headers to NNTP headers. That means you have to replace the header “To: Joh Doe <JohnDoe@example.invalid>” with the header “Newsgroups: general.english_discussions”. And then I write that converted email to the spool folder of the Apache James NNTP server and I have an operational email to NNTP conversion. Sure, it doesn’t have avatars, author profiles, ‘mark as answer’ functionality, but as a NNTP user I really couldn’t care less about that. And the beauty of it is that this is two-way access because all I need to do is click “Reply to user” instead of “Reply to newsgroup” and the email will go to the Reply by email functionality for the forums.
Now to get the last bit of functionality implemented, I just need Adobe to fix their last bug. Email has all these hidden header fields that tell email clients how different messages relate to each other. And Adobe is filling them out incorrectly, so I can not sort messages correctly. So if Adobe could please fix the References and / or In-Reply-To headers in the outgoing email that would make me really happy. And if I am really happy, I might just open up my NNTP server for others.
And it gets better. Because the new forums could easily become inspiring. Because as it turns out, the new Adobe forums have a webservice API. And that API is enabled. And we can program against that API using Flex or more likely AIR (because of the local storage and to bypass any cross domain problems) and build their own interfaces for the forums and customize them. Write plugins for them, for instance to easily link to documentation or copy examples. Or people could write their own automation rules again, so they can sort, filter and prioritize messages the same way they can now through the NNTP ‘API’. I have a bunch of ideas for things I could do (which I probably won’t if Adobe fixes the email bugs), and I am sure there are many more people with even better ideas.
But for that to happen Adobe first needs to do something. The forums may have a webservice API right now, but it doesn’t have any documentation. Sure, there is some documentation from the vendor of the forum, but the implementation of the Adobe ID based login system of the new forums is different. And without a login no API access.
I really think Adobe has an opportunity to make these forums great. The number or forums that truly integrates web, email and NNTP is very limited. Adobe could be the first one to not only do that, but also allow unlimited AIR extensions through its webservices and empower the community to build its own tools. Or is Adobe, the company that is preaching the gospel of the Rich Internet Applications, really forcing all users to go back to a web interface?
Adam Cameron says:
Interesting post Jochem. I’ve said on the forums directly, and will repeat it here in case anyone @ Adobe is paying attention (they specifically don’t pay attention to their forums, by their own admission): unless the new forum system supplies an NNTP feed, I’m going to stop using them. And - as I’m one of the main “question answerers” on the CF forums - this will be a not insignificant loss to them. I invite other NNTP-based participants to do the same.
I don’t care how good a web-based UI for a forum is, it’ll just never be as good as a dedicated news reader application. And I’ve yet to see a web UI on a forum that actually *is* any good, anyhow.
This all sounds like a backwards step to me.
–
2009/04/01, 08:33Adam
Jochem says:
For future reference I am including what an Adobe person had to say on this subject in the test forums (which have been closed now the new forums have been released):
2009/04/07, 10:06Ian Skinner says:
Thanks for a great post Jochem, I have always respected your responses in the various forums where we have crossed paths.
I look forward to following any progress you may have with regaining NNTP access and|or web service API replacements.
If you do not mind a side question, I would like to know more about your news reader that allowed you to ignore threads when desired. That is a feature I have wanted, but not quite enough to go find on my own. But your post presented an opportunity to satisfy a long, if somewhat low priority, annoyance.
2009/04/07, 17:52Ken Ford says:
@Jochem I sure hope that you, or someone else, can come up with an AIR application that will treat the forums like NNTP access.
I find the new forums unuseable because of the speed.
@Adam My participation in the forums is down to almost nothing. And hopefully Adobe will notice that the NNTP users were the responders to most of the questions on the forums.
2009/04/08, 13:25kanguyen says:
Jochem, tI’m interested in knowing more about the email headers and how this might work. As you know, the characters that are appended at the end of the subject lines (which makes it impossible to sort by subject) is there to allow the system to know where to post the reply when users reply by email.
If we had unique identifiers in the references/in-reply-to fields, does the subject line’s uniqueness become a non-issue and allow the threading to work?
2009/04/09, 18:25kanguyen@adobe says:
Jochem, the threading issue has been something we’ve seen a lot of feedback on and we definitely want to try to address this. I’m trying to gather some more information to understand what might be feasible.
As you know, the characters that are appended at the end of the subject lines (which makes it impossible to sort by subject) is there to allow the system to know where to post the reply when users reply by email.
If we had unique message identifiers in the references/in-reply-to fields, does the subject line’s uniqueness become a non-issue and allow the threading to work?
Thanks in advance for sharing your knowledge on the topic.
2009/04/09, 18:59Ian Skinner says:
@ Kanguyen
That is my rudimentary understanding. I am sure Jochem can say more when he has time. But my impression is that this was not a new problem that Jive had to solve. It was long ago addressed in the e-mail standards for list servers. If they would just format the e-mails properly we - OK people like Jochem - could build on functionality and their system would still know where to post the replies received from users by e-mail.
2009/04/09, 19:00Jochem says:
Fat email clients will thread by Message-ID and References / In-Reply-To headers, even if the subject changes. I have heard conflicting stories on how Gmail and other web based email clients handle them.
2009/04/09, 19:01kanguyen@adobe says:
Thanks Ian and Jochem! I am following up with the folks at Jive to see what might be possible. I know this is an important one to make e-mail a viable option and to support Jochem’s NNTP efforts.
2009/04/09, 19:08Jochem says:
@Ian:
What I have been asking for is to just make sure all outgoing e-mail from the forums have the right headers and leave the unique code in the subject for what it is. That is the least invasive solution and leaves the Jive import routines alone to continue to work the way they do now, and depends on the email client to thread properly. As I said, this works for email clients, but I don’t know about web based email systems like Gmail.
What you are suggesting is taking it one step further: remove the unique code from the subject and depend on the email client to put in the correct
Message-IDheaders when responding. That largely works, especially since Jive creates unique reply-to email addresses for all combinations of subscribed user and thread. Some email clients however do not include a proper References and/or In-Reply-To header and then Jive would have no other option but to append the message to the end of the thread. This will most definitely allow web based systems such as Gmail to thread, but may cause some fat clients to always post at the end of the thread.IMHO, the first part is quick and easy and somebody should start coding tomorrow. The second part is something that needs to be carefully considered and the best way forward there may be to just grab a sample of a few thousand replies after the first part has been implemented to see how good the average users email client is.
2009/04/09, 19:14Ian Skinner says:
@Jochem
That makes sense, and I know *I* would be a bit happier if my e-mails where threaded in my desk-top client.
2009/04/09, 19:25“it could be bunnies” » Blog Archive » Slow progress on the new Adobe forums says:
[...] « The new Adobe forums: unfulfilled potential [...]
2009/04/09, 21:46