Woop! Confluence data imported into git and displayed with webslinger!

[Adam Heath-2]

So, about a month ago, people on this list wanted to see an example
website implemented in webslinger.  At the time, I had a preliminary
version of specialpurpose/ofbizwebsite converted.  I mentioned that I
would finish that up, and make it available.

However, that component in svn only has 2 real pages; not enough for a
demonstration.  So I went looking for more content; I found it on
cwiki.apache.org.

The next question became how to get that data into something more
useable.  Well, here is where the fun begins. :)

I have an importer that uses confluence's rpc, to fetch the following
items:

* pages, and all their versions
   * labels(no versions)
   * parent/child relationships(no versions)
   * attachments, and all their versions
   * comments(no versions stored)
* blogs, with versions
   * comments(no versions stored)
* users
   * followers/following
   * profilepics

All this information in then used to create 2 brand new git
repositories; one to store user data(which is supposed to be shared
across all spaces), and another to store the OFBIZ space data.

The importer is smart enough to run multiple times, and only add what
has changed.

For items that don't have versions already, there are bulk commits
that occur.  Over time, as the importer is continually run, history
will build up.  When the switch occurs, the new system will store
history for every change.

Pages are no longer stored by ID.  They are stored by name.  Renames
are handled during import as well(requires updating all parent/child
relationships, all referenced labels, etc).

Now, on to the read-only side of all this.

* home page link(/) is updated automatically.
* attachment icon for pages with attachments is shown
* Links to added by, last edited by are shown per page.
* Raw wiki markup is imported, and I use mylyn to convert it to html.
  This is not perfect.
* Page tree heiarchy; the selected node is auto-opened, but javascript
is not used to expand/collapse.
* attachments on pages(tools menu)
* Viewing the history for a page(parses git data using jgit).
* page tree list
* alphabetical page list(a-z bins), with pagination
* recently updated pages
* blog summary
* all labels
* popular labels
* all attachments, with pagination
* User profile(individual history, and metadata)
* User network

Things not implemented:

* viewing an actual change/diff
* comparing versions
* page summary
* permissions(can't fetch the data, don't have enough permissions on
the doogie user in confluence)
* blog detail, no comments either
* label detail(showing which pages have a label attached)
* attachment version display
* global user directory
* personal profile(no user login capability yet)
* no user profile actions(on the left side)
* default profilepic set not imported(license issues, so they are all 404)
* user status(no rpc, hardly anyone in ofbiz actually filed out status
updates).
* dashboard
* confluence macros in text blobs are not handled(mylyn doesn't
support them)

The timeframe it took me to write this: one week of initial importer
development, 3.5 weeks of continuing importer development, and
frontend development.  All of this is completely from scratch, no
previous application code existed.  I've been working completely in my
spare time(weekends too).  A single person.

Now, here it comes.  The url to the site.
http://ofbizdemo.brainfood.com/.

Things to note.  There are *no* database calls *at all*.  It's all
done with files on disk.  History browsing is backed by git, using
jgit to read it directly in java.  CSS styling is rather poor.  Most
unimplemented pages should do something nice(instead of a big read
'Not Yet Implemented'); at least there shouldn't be an exceptions on
those pages.

[Adam Heath-2]

On 10/11/2010 12:41 PM, Adam Heath wrote:

> So, about a month ago, people on this list wanted to see an example
> website implemented in webslinger. At the time, I had a preliminary
> version of specialpurpose/ofbizwebsite converted. I mentioned that I
> would finish that up, and make it available.
>
> However, that component in svn only has 2 real pages; not enough for a
> demonstration. So I went looking for more content; I found it on
> cwiki.apache.org.
>
> The next question became how to get that data into something more
> useable. Well, here is where the fun begins. :)
>
> I have an importer that uses confluence's rpc, to fetch the following
> items:
>
> * pages, and all their versions
> * labels(no versions)
> * parent/child relationships(no versions)
> * attachments, and all their versions
> * comments(no versions stored)
> * blogs, with versions
> * comments(no versions stored)
> * users
> * followers/following
> * profilepics
>
> All this information in then used to create 2 brand new git
> repositories; one to store user data(which is supposed to be shared
> across all spaces), and another to store the OFBIZ space data.
>
> The importer is smart enough to run multiple times, and only add what
> has changed.
>
> For items that don't have versions already, there are bulk commits that
> occur. Over time, as the importer is continually run, history will build
> up. When the switch occurs, the new system will store history for every
> change.
>
> Pages are no longer stored by ID. They are stored by name. Renames are
> handled during import as well(requires updating all parent/child
> relationships, all referenced labels, etc).
>
> Now, on to the read-only side of all this.
>
> * home page link(/) is updated automatically.
> * attachment icon for pages with attachments is shown
> * Links to added by, last edited by are shown per page.
> * Raw wiki markup is imported, and I use mylyn to convert it to html.
> This is not perfect.
> * Page tree heiarchy; the selected node is auto-opened, but javascript
> is not used to expand/collapse.
> * attachments on pages(tools menu)
> * Viewing the history for a page(parses git data using jgit).
> * page tree list
> * alphabetical page list(a-z bins), with pagination
> * recently updated pages
> * blog summary
> * all labels
> * popular labels
> * all attachments, with pagination
> * User profile(individual history, and metadata)
> * User network
>
> Things not implemented:
>
> * viewing an actual change/diff
> * comparing versions
> * page summary
> * permissions(can't fetch the data, don't have enough permissions on the
> doogie user in confluence)
> * blog detail, no comments either
> * label detail(showing which pages have a label attached)
> * attachment version display
> * global user directory
> * personal profile(no user login capability yet)
> * no user profile actions(on the left side)
> * default profilepic set not imported(license issues, so they are all 404)
> * user status(no rpc, hardly anyone in ofbiz actually filed out status
> updates).
> * dashboard
> * confluence macros in text blobs are not handled(mylyn doesn't support
> them)
>
> The timeframe it took me to write this: one week of initial importer
> development, 3.5 weeks of continuing importer development, and frontend
> development. All of this is completely from scratch, no previous
> application code existed. I've been working completely in my spare
> time(weekends too). A single person.
>
> Now, here it comes. The url to the site. http://ofbizdemo.brainfood.com/.

Some interesting urls to hit:

1: http://ofbizdemo.brainfood.com/person/jacques.le.roux/network
2: http://ofbizdemo.brainfood.com/person/bjfreeman/network
3:
http://ofbizdemo.brainfood.com/page/Apache%20OFBiz%20Service%20Providers/history
4: http://ofbizdemo.brainfood.com/pages/recent

Keep in mind that the software hasn't been heavily optimized.  Some
things could definately be cached more.  And, due to various
circumstances, the loaded site data gets garbage collected at
times(I've made certain that there are no static globals to keep
things around), so sometimes it has to re-load the site configuration,
and compile a bunch of stuff on the fly.  That's solvable by giving
more memory to the instance.

>
> Things to note. There are *no* database calls *at all*. It's all done
> with files on disk. History browsing is backed by git, using jgit to
> read it directly in java. CSS styling is rather poor. Most unimplemented
> pages should do something nice(instead of a big read 'Not Yet
> Implemented'); at least there shouldn't be an exceptions on those pages.

[Jacques Le Roux]

Impressive, now I know what Webslinger is and what it is capable of!

Thanks

Jacques

From: "Adam Heath" <[hidden email]>

> On 10/11/2010 12:41 PM, Adam Heath wrote:
>> So, about a month ago, people on this list wanted to see an example
>> website implemented in webslinger. At the time, I had a preliminary
>> version of specialpurpose/ofbizwebsite converted. I mentioned that I
>> would finish that up, and make it available.
>>
>> However, that component in svn only has 2 real pages; not enough for a
>> demonstration. So I went looking for more content; I found it on
>> cwiki.apache.org.
>>
>> The next question became how to get that data into something more
>> useable. Well, here is where the fun begins. :)
>>
>> I have an importer that uses confluence's rpc, to fetch the following
>> items:
>>
>> * pages, and all their versions
>> * labels(no versions)
>> * parent/child relationships(no versions)
>> * attachments, and all their versions
>> * comments(no versions stored)
>> * blogs, with versions
>> * comments(no versions stored)
>> * users
>> * followers/following
>> * profilepics
>>
>> All this information in then used to create 2 brand new git
>> repositories; one to store user data(which is supposed to be shared
>> across all spaces), and another to store the OFBIZ space data.
>>
>> The importer is smart enough to run multiple times, and only add what
>> has changed.
>>
>> For items that don't have versions already, there are bulk commits that
>> occur. Over time, as the importer is continually run, history will build
>> up. When the switch occurs, the new system will store history for every
>> change.
>>
>> Pages are no longer stored by ID. They are stored by name. Renames are
>> handled during import as well(requires updating all parent/child
>> relationships, all referenced labels, etc).
>>
>> Now, on to the read-only side of all this.
>>
>> * home page link(/) is updated automatically.
>> * attachment icon for pages with attachments is shown
>> * Links to added by, last edited by are shown per page.
>> * Raw wiki markup is imported, and I use mylyn to convert it to html.
>> This is not perfect.
>> * Page tree heiarchy; the selected node is auto-opened, but javascript
>> is not used to expand/collapse.
>> * attachments on pages(tools menu)
>> * Viewing the history for a page(parses git data using jgit).
>> * page tree list
>> * alphabetical page list(a-z bins), with pagination
>> * recently updated pages
>> * blog summary
>> * all labels
>> * popular labels
>> * all attachments, with pagination
>> * User profile(individual history, and metadata)
>> * User network
>>
>> Things not implemented:
>>
>> * viewing an actual change/diff
>> * comparing versions
>> * page summary
>> * permissions(can't fetch the data, don't have enough permissions on the
>> doogie user in confluence)
>> * blog detail, no comments either
>> * label detail(showing which pages have a label attached)
>> * attachment version display
>> * global user directory
>> * personal profile(no user login capability yet)
>> * no user profile actions(on the left side)
>> * default profilepic set not imported(license issues, so they are all 404)
>> * user status(no rpc, hardly anyone in ofbiz actually filed out status
>> updates).
>> * dashboard
>> * confluence macros in text blobs are not handled(mylyn doesn't support
>> them)
>>
>> The timeframe it took me to write this: one week of initial importer
>> development, 3.5 weeks of continuing importer development, and frontend
>> development. All of this is completely from scratch, no previous
>> application code existed. I've been working completely in my spare
>> time(weekends too). A single person.
>>
>> Now, here it comes. The url to the site. http://ofbizdemo.brainfood.com/.
>
> Some interesting urls to hit:
>
> 1: http://ofbizdemo.brainfood.com/person/jacques.le.roux/network
> 2: http://ofbizdemo.brainfood.com/person/bjfreeman/network
> 3:
> http://ofbizdemo.brainfood.com/page/Apache%20OFBiz%20Service%20Providers/history
> 4: http://ofbizdemo.brainfood.com/pages/recent
>
> Keep in mind that the software hasn't been heavily optimized.  Some
> things could definately be cached more.  And, due to various
> circumstances, the loaded site data gets garbage collected at
> times(I've made certain that there are no static globals to keep
> things around), so sometimes it has to re-load the site configuration,
> and compile a bunch of stuff on the fly.  That's solvable by giving
> more memory to the instance.
>
>>
>> Things to note. There are *no* database calls *at all*. It's all done
>> with files on disk. History browsing is backed by git, using jgit to
>> read it directly in java. CSS styling is rather poor. Most unimplemented
>> pages should do something nice(instead of a big read 'Not Yet
>> Implemented'); at least there shouldn't be an exceptions on those pages.
>

[Adam Heath-2]

On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
> Impressive, now I know what Webslinger is and what it is capable of!

Actually, this is just one application.  Webslinger(-core) is an
enabling technology, that enables anything to be written quickly.  As
I said, I've only spent probably 2 actual weeks on the application itself.

[Scott Gray-2]

On 12/10/2010, at 10:03 AM, Adam Heath wrote:

> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>> Impressive, now I know what Webslinger is and what it is capable of!
>
> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.

The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?

I think I understand better now why Ean and yourself were somewhat negative towards the possibility of a jackrabbit integration, do you see this as some sort of alternative?

Regards
Scott

[Adam Heath-2]

On 10/11/2010 04:25 PM, Scott Gray wrote:
> On 12/10/2010, at 10:03 AM, Adam Heath wrote:
>
>> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>>> Impressive, now I know what Webslinger is and what it is capable of!
>>
>> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.
>
> The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?

It means that webslinger could run all of cwiki.apache.org, being
fully java dynamic.  The front page is currently giving me 250req/s
with single concurrency, and 750req/s with a concurrency of 5.  And,
ofbiz would be running along side, so that we could do other things as
well.

> I think I understand better now why Ean and yourself were somewhat
 > negative towards the possibility of a jackrabbit integration, do
 > you see this as some sort of alternative?

Storing content in the database is wrong.  How do you use normal
editors(vim/emacs/dreamweaver/eclipse/photoshop) to manipulate files?
  How do you run find/grep?  What revision control do you
use(git/svn/whatever)?  The webslinger mantra is to reuse existing
toolsets as much as possible.  That means using the filesystem, which
then gives you nfs/samba access for sharing, etc.

The filesystem api we use is commons-vfs; we don't actually use
commons-vfs itself, most of the implementation and filesystems have
been rewritten to actually be non-blocking and performant and not have
thread leaks or memory leaks or dead-locks.  We don't use bsf(too much
reflection, too much synchronization).

[Ean Schuessler]

Scott Gray wrote:
> The main question in my mind is what does all this mean for OFBiz?
> Obviously because webslinger is currently in the framework you
> envisage it playing some sort of role in the ERP applications, but
> what exactly?
We see knowledge sharing as an important ERP function.
> I think I understand better now why Ean and yourself were somewhat negative towards the possibility of a jackrabbit integration, do you see this as some sort of alternative?
>  
Some sort of alternative, though I would see Jackrabbit more as an
alternative to our modded CommonsVFS+Lucene. I'm mostly antagonistic to
a database oriented content management approach because I don't feel
like any of the tools out there (including Jackrabbit) realistically
deal with the situation of having a long-term development project
running in tandem with a live server. All of the database driven tools
(Wordpress, Drupal, Joomla, Plone, Alfresco, LifeRay) fail to deliver a
solution for distributed revision control. For me, that seems like a
critical weakness because I've been through more than a few overhauls of
a large corporate information management infrastructure. Work goes on in
parallel both in the live server and the development environment. If you
don't have tools to manage the process of merging those streams of
information then you are in for a tough time.

Jackrabbit is very interesting, mostly because it extends the filesystem
concept to blend more seamlessly with what the web seems to "want" its
filesystem to look like. I think it would be fully possible for us to
replace CommonsVFS with Jackrabbit but I'm not entirely clear that it is
worth it. Any CMS that cannot present itself as a vanilla filesystem is
fundamentally hampered by the unfortunate reality is that most programs
expect to work with that model. I suppose it depends on where you want
to be inconvenienced.

--
Ean Schuessler, CTO
[hidden email]
214-720-0700 x 315
Brainfood, Inc.
http://www.brainfood.com

[Scott Gray-2]

On 12/10/2010, at 11:45 AM, Adam Heath wrote:

> On 10/11/2010 04:25 PM, Scott Gray wrote:
>> On 12/10/2010, at 10:03 AM, Adam Heath wrote:
>>
>>> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>>>> Impressive, now I know what Webslinger is and what it is capable of!
>>>
>>> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.
>>
>> The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?
>
> It means that webslinger could run all of cwiki.apache.org, being fully java dynamic.  The front page is currently giving me 250req/s with single concurrency, and 750req/s with a concurrency of 5.  And, ofbiz would be running along side, so that we could do other things as well.

That wasn't what I was asking but since you mention it, what does that actually mean for us?  Part of reason we moved to the ASF was so that we could rely on their infrastructure instead of maintaining our own.  Assuming we replaced confluence with webslinger then what do we do if you disappear from the scene in a year's time?  The idea of learning a new obscure tool doesn't sound very appealing.

>> I think I understand better now why Ean and yourself were somewhat
> > negative towards the possibility of a jackrabbit integration, do
> > you see this as some sort of alternative?
>
> Storing content in the database is wrong.  How do you use normal editors(vim/emacs/dreamweaver/eclipse/photoshop) to manipulate files?  How do you run find/grep?  What revision control do you use(git/svn/whatever)?  The webslinger mantra is to reuse existing toolsets as much as possible.  That means using the filesystem, which then gives you nfs/samba access for sharing, etc.
>
> The filesystem api we use is commons-vfs; we don't actually use commons-vfs itself, most of the implementation and filesystems have been rewritten to actually be non-blocking and performant and not have thread leaks or memory leaks or dead-locks.  We don't use bsf(too much reflection, too much synchronization).

Alternative, got it.

[Adam Heath-2]

On 10/11/2010 06:26 PM, Scott Gray wrote:

> On 12/10/2010, at 11:45 AM, Adam Heath wrote:
>
>> On 10/11/2010 04:25 PM, Scott Gray wrote:
>>> On 12/10/2010, at 10:03 AM, Adam Heath wrote:
>>>
>>>> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>>>>> Impressive, now I know what Webslinger is and what it is capable of!
>>>>
>>>> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.
>>>
>>> The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?
>>
>> It means that webslinger could run all of cwiki.apache.org, being fully java dynamic.  The front page is currently giving me 250req/s with single concurrency, and 750req/s with a concurrency of 5.  And, ofbiz would be running along side, so that we could do other things as well.
>
> That wasn't what I was asking but since you mention it, what does

 > that actually mean for us?  Part of reason we moved to the ASF was
 > so that we could rely on their infrastructure instead of maintaining
 > our own.  Assuming we replaced confluence with webslinger then what
 > do we do if you disappear from the scene in a year's time?  The idea
 > of learning a new obscure tool doesn't sound very appealing.

Who said that this was going to stay a brainfood-only project?  We
have every intention of making webslinger(-core) a public, community
project.  There isn't anything really like this.

* Nested servlet container(minor point).
* Filesystem overlay(think unionfs).
* Many servlet-like configuration points can be configured dynamically
at runtime thru the filesystem.

Again, since all this stuff is in the filesystem, git/svn work on all
aspects.  Merging between previous, development, workstation, and
production is quite simple to do.

Because of the overlay capability, it's also easy to upgrade a base
code module, with a light-weight file layout, and have the content
site transparently sit on top, with a unified view of everything.

>>> I think I understand better now why Ean and yourself were somewhat
>>> negative towards the possibility of a jackrabbit integration, do
>>> you see this as some sort of alternative?
>>
>> Storing content in the database is wrong.  How do you use normal editors(vim/emacs/dreamweaver/eclipse/photoshop) to manipulate files?  How do you run find/grep?  What revision control do you use(git/svn/whatever)?  The webslinger mantra is to reuse existing toolsets as much as possible.  That means using the filesystem, which then gives you nfs/samba access for sharing, etc.
>>
>> The filesystem api we use is commons-vfs; we don't actually use commons-vfs itself, most of the implementation and filesystems have been rewritten to actually be non-blocking and performant and not have thread leaks or memory leaks or dead-locks.  We don't use bsf(too much reflection, too much synchronization).
>
> Alternative, got it.

[Scott Gray-2]

On 12/10/2010, at 12:37 PM, Adam Heath wrote:

> On 10/11/2010 06:26 PM, Scott Gray wrote:
>> On 12/10/2010, at 11:45 AM, Adam Heath wrote:
>>
>>> On 10/11/2010 04:25 PM, Scott Gray wrote:
>>>> On 12/10/2010, at 10:03 AM, Adam Heath wrote:
>>>>
>>>>> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>>>>>> Impressive, now I know what Webslinger is and what it is capable of!
>>>>>
>>>>> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.
>>>>
>>>> The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?
>>>
>>> It means that webslinger could run all of cwiki.apache.org, being fully java dynamic.  The front page is currently giving me 250req/s with single concurrency, and 750req/s with a concurrency of 5.  And, ofbiz would be running along side, so that we could do other things as well.
>>
>> That wasn't what I was asking but since you mention it, what does
> > that actually mean for us?  Part of reason we moved to the ASF was
> > so that we could rely on their infrastructure instead of maintaining
> > our own.  Assuming we replaced confluence with webslinger then what
> > do we do if you disappear from the scene in a year's time?  The idea
> > of learning a new obscure tool doesn't sound very appealing.
>
> Who said that this was going to stay a brainfood-only project?  

No one and I didn't make that assumption.

> We have every intention of making webslinger(-core) a public, community project.  There isn't anything really like this.

I'm sure every dead open source project had the intention of building a thriving community but it doesn't always work out that way.  What I am asking is what will the OFBiz documentation gain by being hosted on webslinger(-core?) that makes it worth the risk of the project being abandoned and us having to move it all back to confluence or whatever the ASF is using by then?

And what is (-core)?  Does that imply that there is a webslinger(-pro) edition that OFBiz users can take advantage of by contracting with or licensing from brainfood?  I don't think a little skepticism  is out of order when you tell us how wonderful it would be for OFBiz to include webslinger if your company stands to benefit from its inclusion.  I'm not even saying that's a bad thing, I just prefer to have the full picture.

> * Nested servlet container(minor point).
> * Filesystem overlay(think unionfs).
> * Many servlet-like configuration points can be configured dynamically at runtime thru the filesystem.
>
> Again, since all this stuff is in the filesystem, git/svn work on all aspects.  Merging between previous, development, workstation, and production is quite simple to do.
>
> Because of the overlay capability, it's also easy to upgrade a base code module, with a light-weight file layout, and have the content site transparently sit on top, with a unified view of everything.
>
>>>> I think I understand better now why Ean and yourself were somewhat
>>>> negative towards the possibility of a jackrabbit integration, do
>>>> you see this as some sort of alternative?
>>>
>>> Storing content in the database is wrong.  How do you use normal editors(vim/emacs/dreamweaver/eclipse/photoshop) to manipulate files?  How do you run find/grep?  What revision control do you use(git/svn/whatever)?  The webslinger mantra is to reuse existing toolsets as much as possible.  That means using the filesystem, which then gives you nfs/samba access for sharing, etc.
>>>
>>> The filesystem api we use is commons-vfs; we don't actually use commons-vfs itself, most of the implementation and filesystems have been rewritten to actually be non-blocking and performant and not have thread leaks or memory leaks or dead-locks.  We don't use bsf(too much reflection, too much synchronization).
>>
>> Alternative, got it.
>

[Nico Toerl]

On 10/12/10 01:41, Adam Heath wrote:

<snip>
> Now, here it comes.  The url to the site.
> http://ofbizdemo.brainfood.com/.
>
> Things to note.  There are *no* database calls *at all*.  It's all
> done with files on disk.  History browsing is backed by git, using
> jgit to read it directly in java.  CSS styling is rather poor.  Most
> unimplemented pages should do something nice(instead of a big read
> 'Not Yet Implemented'); at least there shouldn't be an exceptions on
> those pages.

that sounded real interesting and i thought i have to have a look at
this, unfortunately all i got is:


  HTTP Status 500 -

------------------------------------------------------------------------

*type* Exception report

*message*

*description* _The server encountered an internal error () that
prevented it from fulfilling this request._

*exception*

java.lang.NullPointerException
        WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
        org.webslinger.ext.code.CodeType.run(CodeType.java:74)
        org.webslinger.WebslingerPlanner.invokeContent(WebslingerPlanner.java:531)
        org.webslinger.WebslingerPlanner.runDirect(WebslingerPlanner.java:286)
        org.webslinger.WebslingerServletContext.runDirectNoThrow(WebslingerServletContext.java:1406)
        org.webslinger.WebslingerServletContext$RequestEventsRequestListener.runEvent(WebslingerServletContext.java:1068)
        org.webslinger.WebslingerServletContext$RequestEventsRequestListener.requestInitialized(WebslingerServletContext.java:1076)
        org.webslinger.WebslingerServletContext$RequestFilterChain.doFilter(WebslingerServletContext.java:861)
        org.webslinger.WebslingerServletContext.service(WebslingerServletContext.java:429)
        org.webslinger.WebslingerServletContext.service(WebslingerServletContext.java:292)
        org.webslinger.servlet.WebslingerServlet.service(WebslingerServlet.java:52)
        javax.servlet.http.HttpServlet.service(HttpServlet.java:717)

*note* _The full stack trace of the root cause is available in the
Apache Tomcat/6.0.29 logs._


cheers
Nico

--
Nico Toerl                      
SeniorSysadmin / IT-department
Virtual Village

Tel. [phone] ext.7042

[Adam Heath-2]

On 10/11/2010 10:07 PM, Nico Toerl wrote:

> On 10/12/10 01:41, Adam Heath wrote:
>
> <snip>
>> Now, here it comes.  The url to the site.
>> http://ofbizdemo.brainfood.com/.
>>
>> Things to note.  There are *no* database calls *at all*.  It's all
>> done with files on disk.  History browsing is backed by git, using
>> jgit to read it directly in java.  CSS styling is rather poor.  Most
>> unimplemented pages should do something nice(instead of a big read
>> 'Not Yet Implemented'); at least there shouldn't be an exceptions on
>> those pages.
>
> that sounded real interesting and i thought i have to have a look at
> this, unfortunately all i got is:
>
>
>    HTTP Status 500 -
>
> ------------------------------------------------------------------------
>
> *type* Exception report
>
> *message*
>
> *description* _The server encountered an internal error () that
> prevented it from fulfilling this request._
>
> *exception*
>
> java.lang.NullPointerException
> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)

Hmm, nice, thanks.

Your user-agent is:

"Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
  Gecko/20100824 Firefox/3.6.9"

The (x86_64) is what is causing the problem, I hadn't seen this type
of string in the wild.  The regex doesn't like nested ().  It's fixed now.

[Marc Morin]

With all the other technologies in ofbiz, seems like webslinger just adds more stuff onto the pile.  I don't want to argue the technical merits of database or file system persistence for a CMS, but it appears like ofbiz would benefit from reducing the number of technologies used, and increase the amount of re-use of technologies it already has.

So, for me, that means entity/service/screen/presentment models are the core technologies.   Galvanizing initiatives around those appear to provide leverage.

Now don't get me wrong, the "CMS" that is native in ofbiz is incomplete and needs a lot of work...  and for our use case of providing self edited web sites and ecommerce sites, that appears a better starting point.  We have done things to add self editing etc... but we need to put a lot more effort into that to ensure that there is a real solution.

my $0.02.


Marc Morin
Emforium Group Inc.
ALL-IN Software
519-772-6824 ext 201
[hidden email]

----- Original Message -----

> On 10/11/2010 10:07 PM, Nico Toerl wrote:
> > On 10/12/10 01:41, Adam Heath wrote:
> >
> > <snip>
> >> Now, here it comes. The url to the site.
> >> http://ofbizdemo.brainfood.com/.
> >>
> >> Things to note. There are *no* database calls *at all*. It's all
> >> done with files on disk. History browsing is backed by git, using
> >> jgit to read it directly in java. CSS styling is rather poor. Most
> >> unimplemented pages should do something nice(instead of a big read
> >> 'Not Yet Implemented'); at least there shouldn't be an exceptions
> >> on those pages.
> >
> > that sounded real interesting and i thought i have to have a look at
> > this, unfortunately all i got is:
> >
> >
> >    HTTP Status 500 -
> >
> > ------------------------------------------------------------------------
> >
> > *type* Exception report
> >
> > *message*
> >
> > *description* _The server encountered an internal error () that
> > prevented it from fulfilling this request._
> >
> > *exception*
> >
> > java.lang.NullPointerException
> > WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>
> Hmm, nice, thanks.
>
> Your user-agent is:
>
> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
> Gecko/20100824 Firefox/3.6.9"
>
> The (x86_64) is what is causing the problem, I hadn't seen this type
> of string in the wild. The regex doesn't like nested (). It's fixed
> now.

[Adrian Crum]

Actually, a discussion of database versus filesystem storage of content
would be worthwhile. So far there has been some hyperbole, but few facts.

-Adrian

On 10/12/2010 7:32 AM, Marc Morin wrote:

> With all the other technologies in ofbiz, seems like webslinger just adds more stuff onto the pile.  I don't want to argue the technical merits of database or file system persistence for a CMS, but it appears like ofbiz would benefit from reducing the number of technologies used, and increase the amount of re-use of technologies it already has.
>
> So, for me, that means entity/service/screen/presentment models are the core technologies.   Galvanizing initiatives around those appear to provide leverage.
>
> Now don't get me wrong, the "CMS" that is native in ofbiz is incomplete and needs a lot of work...  and for our use case of providing self edited web sites and ecommerce sites, that appears a better starting point.  We have done things to add self editing etc... but we need to put a lot more effort into that to ensure that there is a real solution.
>
> my $0.02.
>
>
> Marc Morin
> Emforium Group Inc.
> ALL-IN Software
> 519-772-6824 ext 201
> [hidden email]
>
> ----- Original Message -----
>> On 10/11/2010 10:07 PM, Nico Toerl wrote:
>>> On 10/12/10 01:41, Adam Heath wrote:
>>>
>>> <snip>
>>>> Now, here it comes. The url to the site.
>>>> http://ofbizdemo.brainfood.com/.
>>>>
>>>> Things to note. There are *no* database calls *at all*. It's all
>>>> done with files on disk. History browsing is backed by git, using
>>>> jgit to read it directly in java. CSS styling is rather poor. Most
>>>> unimplemented pages should do something nice(instead of a big read
>>>> 'Not Yet Implemented'); at least there shouldn't be an exceptions
>>>> on those pages.
>>>
>>> that sounded real interesting and i thought i have to have a look at
>>> this, unfortunately all i got is:
>>>
>>>
>>>     HTTP Status 500 -
>>>
>>> ------------------------------------------------------------------------
>>>
>>> *type* Exception report
>>>
>>> *message*
>>>
>>> *description* _The server encountered an internal error () that
>>> prevented it from fulfilling this request._
>>>
>>> *exception*
>>>
>>> java.lang.NullPointerException
>>> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>>
>> Hmm, nice, thanks.
>>
>> Your user-agent is:
>>
>> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
>> Gecko/20100824 Firefox/3.6.9"
>>
>> The (x86_64) is what is causing the problem, I hadn't seen this type
>> of string in the wild. The regex doesn't like nested (). It's fixed
>> now.
>

[Adam Heath-2]

On 10/11/2010 06:58 PM, Scott Gray wrote:

> On 12/10/2010, at 12:37 PM, Adam Heath wrote:
>
>> On 10/11/2010 06:26 PM, Scott Gray wrote:
>>> On 12/10/2010, at 11:45 AM, Adam Heath wrote:
>>>
>>>> On 10/11/2010 04:25 PM, Scott Gray wrote:
>>>>> On 12/10/2010, at 10:03 AM, Adam Heath wrote:
>>>>>
>>>>>> On 10/11/2010 02:37 PM, Jacques Le Roux wrote:
>>>>>>> Impressive, now I know what Webslinger is and what it is capable of!
>>>>>>
>>>>>> Actually, this is just one application.  Webslinger(-core) is an enabling technology, that enables anything to be written quickly.  As I said, I've only spent probably 2 actual weeks on the application itself.
>>>>>
>>>>> The main question in my mind is what does all this mean for OFBiz?  Obviously because webslinger is currently in the framework you envisage it playing some sort of role in the ERP applications, but what exactly?
>>>>
>>>> It means that webslinger could run all of cwiki.apache.org, being fully java dynamic.  The front page is currently giving me 250req/s with single concurrency, and 750req/s with a concurrency of 5.  And, ofbiz would be running along side, so that we could do other things as well.
>>>
>>> That wasn't what I was asking but since you mention it, what does
>>> that actually mean for us?  Part of reason we moved to the ASF was
>>> so that we could rely on their infrastructure instead of maintaining
>>> our own.  Assuming we replaced confluence with webslinger then what
>>> do we do if you disappear from the scene in a year's time?  The idea
>>> of learning a new obscure tool doesn't sound very appealing.
>>
>> Who said that this was going to stay a brainfood-only project?
>
> No one and I didn't make that assumption.
>
>> We have every intention of making webslinger(-core) a public, community project.  There isn't anything really like this.
>
> I'm sure every dead open source project had the intention of building a thriving community but it doesn't always work out that way.  What I am asking is what will the OFBiz documentation gain by being hosted on webslinger(-core?) that makes it worth the risk of the project being abandoned and us having to move it all back to confluence or whatever the ASF is using by then?
>
> And what is (-core)?  Does that imply that there is a
> webslinger(-pro) edition that OFBiz users can take advantage of
> by contracting with or licensing from brainfood?  I don't think
> a little skepticism  is out of order when you tell us how wonderful
> it would be for OFBiz to include webslinger if your company stands
> to benefit from its inclusion.  I'm not even saying that's a bad
> thing, I just prefer to have the full picture.

Yeah, I'm not surprised you picked up on that.  It's a very good question.

webslinger-core is the enabler.  It has no application logic.  It just
makes it simple(r) to write applications.  It would be like taking the
entityengine, serviceengine, all the widget systems, and the
controller, but with *no* config files, no entitymodel, no service
definitions, etc.  webslinger-core is the system-level classes, and
nothing else.

Webslinger, however, is the combination of the core, and all the
runtime classes/css/html templates.  This would be similiar to the
actions, ftl, and widget definitions, but only in framework.

Finally, a webslinger application would then be everything that exists
in an ofbiz applications/foo or specialpurpose/foo folder.

There will never be a difference in webslinger-core, between an
internal/external system.  It would take more time to try and keep
those things separate.

[Adam Heath-2]

On 10/12/2010 10:25 AM, Adrian Crum wrote:
> Actually, a discussion of database versus filesystem storage of content
> would be worthwhile. So far there has been some hyperbole, but few facts.

How do you edit database content?  What is the procedure?  Can a
simple editor be used?  By simple, I mean low-level, like vi.

How do you find all items in your content store that contain a certain
text word?  Can grep and find be used?

How do you handle moving changes between a production server, that is
being directly managed by the client, and multiple developer
workstations, which then all have to go first to a staging server?
Each system in this case has its own set of code changes, and
config+data changes, that then have to be selectively picked for
staging, before finally being merged with production.

What about revision control?  Can you go back in time to see what the
code+data looked like?  Are there separate revision systems, one for
the database, and another for the content?  And what about the code?

For users/systems that aren't capable of using revision control, is
there a way for them to mount/browse the content store?  Think
nfs/samba here.

Storing everything directly into the filesystem lets you reuse
existing tools, that have been perfected over countless generations of
man-years.

>
> -Adrian
>
> On 10/12/2010 7:32 AM, Marc Morin wrote:
>> With all the other technologies in ofbiz, seems like webslinger just
>> adds more stuff onto the pile. I don't want to argue the technical
>> merits of database or file system persistence for a CMS, but it
>> appears like ofbiz would benefit from reducing the number of
>> technologies used, and increase the amount of re-use of technologies
>> it already has.
>>
>> So, for me, that means entity/service/screen/presentment models are
>> the core technologies. Galvanizing initiatives around those appear to
>> provide leverage.
>>
>> Now don't get me wrong, the "CMS" that is native in ofbiz is
>> incomplete and needs a lot of work... and for our use case of
>> providing self edited web sites and ecommerce sites, that appears a
>> better starting point. We have done things to add self editing etc...
>> but we need to put a lot more effort into that to ensure that there is
>> a real solution.
>>
>> my $0.02.
>>
>>
>> Marc Morin
>> Emforium Group Inc.
>> ALL-IN Software
>> 519-772-6824 ext 201
>> [hidden email]
>>
>> ----- Original Message -----
>>> On 10/11/2010 10:07 PM, Nico Toerl wrote:
>>>> On 10/12/10 01:41, Adam Heath wrote:
>>>>
>>>> <snip>
>>>>> Now, here it comes. The url to the site.
>>>>> http://ofbizdemo.brainfood.com/.
>>>>>
>>>>> Things to note. There are *no* database calls *at all*. It's all
>>>>> done with files on disk. History browsing is backed by git, using
>>>>> jgit to read it directly in java. CSS styling is rather poor. Most
>>>>> unimplemented pages should do something nice(instead of a big read
>>>>> 'Not Yet Implemented'); at least there shouldn't be an exceptions
>>>>> on those pages.
>>>>
>>>> that sounded real interesting and i thought i have to have a look at
>>>> this, unfortunately all i got is:
>>>>
>>>>
>>>> HTTP Status 500 -
>>>>
>>>> ------------------------------------------------------------------------
>>>>
>>>>
>>>> *type* Exception report
>>>>
>>>> *message*
>>>>
>>>> *description* _The server encountered an internal error () that
>>>> prevented it from fulfilling this request._
>>>>
>>>> *exception*
>>>>
>>>> java.lang.NullPointerException
>>>> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>>>>
>>>
>>> Hmm, nice, thanks.
>>>
>>> Your user-agent is:
>>>
>>> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
>>> Gecko/20100824 Firefox/3.6.9"
>>>
>>> The (x86_64) is what is causing the problem, I hadn't seen this type
>>> of string in the wild. The regex doesn't like nested (). It's fixed
>>> now.
>>

[Adrian Crum]

On 10/12/2010 8:55 AM, Adam Heath wrote:

> On 10/12/2010 10:25 AM, Adrian Crum wrote:
>> Actually, a discussion of database versus filesystem storage of content
>> would be worthwhile. So far there has been some hyperbole, but few facts.
>
> How do you edit database content? What is the procedure? Can a simple
> editor be used? By simple, I mean low-level, like vi.
>
> How do you find all items in your content store that contain a certain
> text word? Can grep and find be used?
>
> How do you handle moving changes between a production server, that is
> being directly managed by the client, and multiple developer
> workstations, which then all have to go first to a staging server? Each
> system in this case has its own set of code changes, and config+data
> changes, that then have to be selectively picked for staging, before
> finally being merged with production.
>
> What about revision control? Can you go back in time to see what the
> code+data looked like? Are there separate revision systems, one for the
> database, and another for the content? And what about the code?
>
> For users/systems that aren't capable of using revision control, is
> there a way for them to mount/browse the content store? Think nfs/samba
> here.
>
> Storing everything directly into the filesystem lets you reuse existing
> tools, that have been perfected over countless generations of man-years.

I believe Jackrabbit has WebDAV and VFS front ends that will accommodate
file system tools. Watch the movie:

http://www.day.com/day/en/products/crx.html

-Adrian

>
>>
>> -Adrian
>>
>> On 10/12/2010 7:32 AM, Marc Morin wrote:
>>> With all the other technologies in ofbiz, seems like webslinger just
>>> adds more stuff onto the pile. I don't want to argue the technical
>>> merits of database or file system persistence for a CMS, but it
>>> appears like ofbiz would benefit from reducing the number of
>>> technologies used, and increase the amount of re-use of technologies
>>> it already has.
>>>
>>> So, for me, that means entity/service/screen/presentment models are
>>> the core technologies. Galvanizing initiatives around those appear to
>>> provide leverage.
>>>
>>> Now don't get me wrong, the "CMS" that is native in ofbiz is
>>> incomplete and needs a lot of work... and for our use case of
>>> providing self edited web sites and ecommerce sites, that appears a
>>> better starting point. We have done things to add self editing etc...
>>> but we need to put a lot more effort into that to ensure that there is
>>> a real solution.
>>>
>>> my $0.02.
>>>
>>>
>>> Marc Morin
>>> Emforium Group Inc.
>>> ALL-IN Software
>>> 519-772-6824 ext 201
>>> [hidden email]
>>>
>>> ----- Original Message -----
>>>> On 10/11/2010 10:07 PM, Nico Toerl wrote:
>>>>> On 10/12/10 01:41, Adam Heath wrote:
>>>>>
>>>>> <snip>
>>>>>> Now, here it comes. The url to the site.
>>>>>> http://ofbizdemo.brainfood.com/.
>>>>>>
>>>>>> Things to note. There are *no* database calls *at all*. It's all
>>>>>> done with files on disk. History browsing is backed by git, using
>>>>>> jgit to read it directly in java. CSS styling is rather poor. Most
>>>>>> unimplemented pages should do something nice(instead of a big read
>>>>>> 'Not Yet Implemented'); at least there shouldn't be an exceptions
>>>>>> on those pages.
>>>>>
>>>>> that sounded real interesting and i thought i have to have a look at
>>>>> this, unfortunately all i got is:
>>>>>
>>>>>
>>>>> HTTP Status 500 -
>>>>>
>>>>> ------------------------------------------------------------------------
>>>>>
>>>>>
>>>>>
>>>>> *type* Exception report
>>>>>
>>>>> *message*
>>>>>
>>>>> *description* _The server encountered an internal error () that
>>>>> prevented it from fulfilling this request._
>>>>>
>>>>> *exception*
>>>>>
>>>>> java.lang.NullPointerException
>>>>> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>>>>>
>>>>>
>>>>
>>>> Hmm, nice, thanks.
>>>>
>>>> Your user-agent is:
>>>>
>>>> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
>>>> Gecko/20100824 Firefox/3.6.9"
>>>>
>>>> The (x86_64) is what is causing the problem, I hadn't seen this type
>>>> of string in the wild. The regex doesn't like nested (). It's fixed
>>>> now.
>>>
>
>

[Adam Heath-2]

On 10/12/2010 11:06 AM, Adrian Crum wrote:

> On 10/12/2010 8:55 AM, Adam Heath wrote:
>> On 10/12/2010 10:25 AM, Adrian Crum wrote:
>>> Actually, a discussion of database versus filesystem storage of content
>>> would be worthwhile. So far there has been some hyperbole, but few
>>> facts.
>>
>> How do you edit database content? What is the procedure? Can a simple
>> editor be used? By simple, I mean low-level, like vi.
>>
>> How do you find all items in your content store that contain a certain
>> text word? Can grep and find be used?
>>
>> How do you handle moving changes between a production server, that is
>> being directly managed by the client, and multiple developer
>> workstations, which then all have to go first to a staging server? Each
>> system in this case has its own set of code changes, and config+data
>> changes, that then have to be selectively picked for staging, before
>> finally being merged with production.
>>
>> What about revision control? Can you go back in time to see what the
>> code+data looked like? Are there separate revision systems, one for the
>> database, and another for the content? And what about the code?
>>
>> For users/systems that aren't capable of using revision control, is
>> there a way for them to mount/browse the content store? Think nfs/samba
>> here.
>>
>> Storing everything directly into the filesystem lets you reuse existing
>> tools, that have been perfected over countless generations of man-years.
>
> I believe Jackrabbit has WebDAV and VFS front ends that will accommodate
> file system tools. Watch the movie:
>
> http://www.day.com/day/en/products/crx.html

Front end it wrong.  It still being the store itself is in some other
system(database).  The raw store needs to be managed by the
filesystem, so standard tools can move it between locations, or do
backups, etc.  Putting yet another layer just to emulate file access
is the wrong way.

<brainstorming>
Let's make a content management system.  Yeah!  Let's do it!  So, we
need to be able to search for content, and mantain links between
relationships.  Let's write brand new code to do that, and put it in
the database.

Ok, next, we need to pull the information out of the database, and
serve it thru an http server.  Oh, damn, it's not running fast.  Let's
have a cache that resides someplace faster than the database.  Oh, I
know, memory!  Shit, it's using too much memory.  Let's put the cache
in the filesystem.  Updates now remove the cache, and have it get
rebuilt.  That means read-only access is faster, but updates then have
to rebuild tons of stuff.

Hmm.  We have a designer request to be able to use photoshop to edit
images.  The server in question is a preview server, is hosted, and
not on his immediate network.  Let's create a new webdav access
method, to make the content look like a filesystem.

Our system is getting heavily loaded.  Let's have a separate database
server, with multiple web frontends.  Cool, that works.

The system is still heavily loaded, we need a super-huge database server.

Crap, still falling over.  Time to have multiple read-only databases.
</brainstorming>

or...

<brainstorming>
Let's store all our content into the filesystem.  That way, things
like ExpanDrive(remote ssh fs access for windows) will work for remote
hosted machines.  Caching isn't a problem anymore, as the raw store is
in files.  Servers have been doing file sharing for decades, it's a
well known problem.  Let's have someone else maintain the file sharing
code, we'll just use it to support multiple frontends.  And, ooh, our
designers will be able to use the tools they are familiar with to
manipulate things.  And, we won't have the extra code running to
maintain all the stuff in the multiple databases.  Cool, we can even
use git, with rebase and merge, to do all sorts of fancy branching and
push/pulling between multiple development scenarios.
</brainstorming>

If the raw store was in the filesystem in the first place, then all
this additional layering wouldn't be needed, to make the final output
end up looking like a filesystem, which is what was being replaced all
along.

>
> -Adrian
>
>>
>>>
>>> -Adrian
>>>
>>> On 10/12/2010 7:32 AM, Marc Morin wrote:
>>>> With all the other technologies in ofbiz, seems like webslinger just
>>>> adds more stuff onto the pile. I don't want to argue the technical
>>>> merits of database or file system persistence for a CMS, but it
>>>> appears like ofbiz would benefit from reducing the number of
>>>> technologies used, and increase the amount of re-use of technologies
>>>> it already has.
>>>>
>>>> So, for me, that means entity/service/screen/presentment models are
>>>> the core technologies. Galvanizing initiatives around those appear to
>>>> provide leverage.
>>>>
>>>> Now don't get me wrong, the "CMS" that is native in ofbiz is
>>>> incomplete and needs a lot of work... and for our use case of
>>>> providing self edited web sites and ecommerce sites, that appears a
>>>> better starting point. We have done things to add self editing etc...
>>>> but we need to put a lot more effort into that to ensure that there is
>>>> a real solution.
>>>>
>>>> my $0.02.
>>>>
>>>>
>>>> Marc Morin
>>>> Emforium Group Inc.
>>>> ALL-IN Software
>>>> 519-772-6824 ext 201
>>>> [hidden email]
>>>>
>>>> ----- Original Message -----
>>>>> On 10/11/2010 10:07 PM, Nico Toerl wrote:
>>>>>> On 10/12/10 01:41, Adam Heath wrote:
>>>>>>
>>>>>> <snip>
>>>>>>> Now, here it comes. The url to the site.
>>>>>>> http://ofbizdemo.brainfood.com/.
>>>>>>>
>>>>>>> Things to note. There are *no* database calls *at all*. It's all
>>>>>>> done with files on disk. History browsing is backed by git, using
>>>>>>> jgit to read it directly in java. CSS styling is rather poor. Most
>>>>>>> unimplemented pages should do something nice(instead of a big read
>>>>>>> 'Not Yet Implemented'); at least there shouldn't be an exceptions
>>>>>>> on those pages.
>>>>>>
>>>>>> that sounded real interesting and i thought i have to have a look at
>>>>>> this, unfortunately all i got is:
>>>>>>
>>>>>>
>>>>>> HTTP Status 500 -
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> *type* Exception report
>>>>>>
>>>>>> *message*
>>>>>>
>>>>>> *description* _The server encountered an internal error () that
>>>>>> prevented it from fulfilling this request._
>>>>>>
>>>>>> *exception*
>>>>>>
>>>>>> java.lang.NullPointerException
>>>>>> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>> Hmm, nice, thanks.
>>>>>
>>>>> Your user-agent is:
>>>>>
>>>>> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
>>>>> Gecko/20100824 Firefox/3.6.9"
>>>>>
>>>>> The (x86_64) is what is causing the problem, I hadn't seen this type
>>>>> of string in the wild. The regex doesn't like nested (). It's fixed
>>>>> now.
>>>>
>>
>>

[Adrian Crum]

On 10/12/2010 9:23 AM, Adam Heath wrote:

> On 10/12/2010 11:06 AM, Adrian Crum wrote:
>> On 10/12/2010 8:55 AM, Adam Heath wrote:
>>> On 10/12/2010 10:25 AM, Adrian Crum wrote:
>>>> Actually, a discussion of database versus filesystem storage of content
>>>> would be worthwhile. So far there has been some hyperbole, but few
>>>> facts.
>>>
>>> How do you edit database content? What is the procedure? Can a simple
>>> editor be used? By simple, I mean low-level, like vi.
>>>
>>> How do you find all items in your content store that contain a certain
>>> text word? Can grep and find be used?
>>>
>>> How do you handle moving changes between a production server, that is
>>> being directly managed by the client, and multiple developer
>>> workstations, which then all have to go first to a staging server? Each
>>> system in this case has its own set of code changes, and config+data
>>> changes, that then have to be selectively picked for staging, before
>>> finally being merged with production.
>>>
>>> What about revision control? Can you go back in time to see what the
>>> code+data looked like? Are there separate revision systems, one for the
>>> database, and another for the content? And what about the code?
>>>
>>> For users/systems that aren't capable of using revision control, is
>>> there a way for them to mount/browse the content store? Think nfs/samba
>>> here.
>>>
>>> Storing everything directly into the filesystem lets you reuse existing
>>> tools, that have been perfected over countless generations of man-years.
>>
>> I believe Jackrabbit has WebDAV and VFS front ends that will accommodate
>> file system tools. Watch the movie:
>>
>> http://www.day.com/day/en/products/crx.html
>
> Front end it wrong. It still being the store itself is in some other
> system(database). The raw store needs to be managed by the filesystem,
> so standard tools can move it between locations, or do backups, etc.
> Putting yet another layer just to emulate file access is the wrong way.
>
> <brainstorming>
> Let's make a content management system. Yeah! Let's do it! So, we need
> to be able to search for content, and mantain links between
> relationships. Let's write brand new code to do that, and put it in the
> database.
>
> Ok, next, we need to pull the information out of the database, and serve
> it thru an http server. Oh, damn, it's not running fast. Let's have a
> cache that resides someplace faster than the database. Oh, I know,
> memory! Shit, it's using too much memory. Let's put the cache in the
> filesystem. Updates now remove the cache, and have it get rebuilt. That
> means read-only access is faster, but updates then have to rebuild tons
> of stuff.
>
> Hmm. We have a designer request to be able to use photoshop to edit
> images. The server in question is a preview server, is hosted, and not
> on his immediate network. Let's create a new webdav access method, to
> make the content look like a filesystem.
>
> Our system is getting heavily loaded. Let's have a separate database
> server, with multiple web frontends. Cool, that works.
>
> The system is still heavily loaded, we need a super-huge database server.
>
> Crap, still falling over. Time to have multiple read-only databases.
> </brainstorming>
>
> or...
>
> <brainstorming>
> Let's store all our content into the filesystem. That way, things like
> ExpanDrive(remote ssh fs access for windows) will work for remote hosted
> machines. Caching isn't a problem anymore, as the raw store is in files.
> Servers have been doing file sharing for decades, it's a well known
> problem. Let's have someone else maintain the file sharing code, we'll
> just use it to support multiple frontends. And, ooh, our designers will
> be able to use the tools they are familiar with to manipulate things.
> And, we won't have the extra code running to maintain all the stuff in
> the multiple databases. Cool, we can even use git, with rebase and
> merge, to do all sorts of fancy branching and push/pulling between
> multiple development scenarios.
> </brainstorming>
>
> If the raw store was in the filesystem in the first place, then all this
> additional layering wouldn't be needed, to make the final output end up
> looking like a filesystem, which is what was being replaced all along.

Okay. Will webslinger provide a JCR interface?

-Adrian

>>>> On 10/12/2010 7:32 AM, Marc Morin wrote:
>>>>> With all the other technologies in ofbiz, seems like webslinger just
>>>>> adds more stuff onto the pile. I don't want to argue the technical
>>>>> merits of database or file system persistence for a CMS, but it
>>>>> appears like ofbiz would benefit from reducing the number of
>>>>> technologies used, and increase the amount of re-use of technologies
>>>>> it already has.
>>>>>
>>>>> So, for me, that means entity/service/screen/presentment models are
>>>>> the core technologies. Galvanizing initiatives around those appear to
>>>>> provide leverage.
>>>>>
>>>>> Now don't get me wrong, the "CMS" that is native in ofbiz is
>>>>> incomplete and needs a lot of work... and for our use case of
>>>>> providing self edited web sites and ecommerce sites, that appears a
>>>>> better starting point. We have done things to add self editing etc...
>>>>> but we need to put a lot more effort into that to ensure that there is
>>>>> a real solution.
>>>>>
>>>>> my $0.02.
>>>>>
>>>>>
>>>>> Marc Morin
>>>>> Emforium Group Inc.
>>>>> ALL-IN Software
>>>>> 519-772-6824 ext 201
>>>>> [hidden email]
>>>>>
>>>>> ----- Original Message -----
>>>>>> On 10/11/2010 10:07 PM, Nico Toerl wrote:
>>>>>>> On 10/12/10 01:41, Adam Heath wrote:
>>>>>>>
>>>>>>> <snip>
>>>>>>>> Now, here it comes. The url to the site.
>>>>>>>> http://ofbizdemo.brainfood.com/.
>>>>>>>>
>>>>>>>> Things to note. There are *no* database calls *at all*. It's all
>>>>>>>> done with files on disk. History browsing is backed by git, using
>>>>>>>> jgit to read it directly in java. CSS styling is rather poor. Most
>>>>>>>> unimplemented pages should do something nice(instead of a big read
>>>>>>>> 'Not Yet Implemented'); at least there shouldn't be an exceptions
>>>>>>>> on those pages.
>>>>>>>
>>>>>>> that sounded real interesting and i thought i have to have a look at
>>>>>>> this, unfortunately all i got is:
>>>>>>>
>>>>>>>
>>>>>>> HTTP Status 500 -
>>>>>>>
>>>>>>> ------------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> *type* Exception report
>>>>>>>
>>>>>>> *message*
>>>>>>>
>>>>>>> *description* _The server encountered an internal error () that
>>>>>>> prevented it from fulfilling this request._
>>>>>>>
>>>>>>> *exception*
>>>>>>>
>>>>>>> java.lang.NullPointerException
>>>>>>> WEB_45$INF.Events.System.Request.DetectUserAgent_46$jn.run(DetectUserAgent.jn:166)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Hmm, nice, thanks.
>>>>>>
>>>>>> Your user-agent is:
>>>>>>
>>>>>> "Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-GB; rv:1.9.2.9)
>>>>>> Gecko/20100824 Firefox/3.6.9"
>>>>>>
>>>>>> The (x86_64) is what is causing the problem, I hadn't seen this type
>>>>>> of string in the wild. The regex doesn't like nested (). It's fixed
>>>>>> now.
>>>>>
>>>
>>>
>
>

[Adam Heath-2]

On 10/12/2010 11:26 AM, Adrian Crum wrote:

> On 10/12/2010 9:23 AM, Adam Heath wrote:
>> On 10/12/2010 11:06 AM, Adrian Crum wrote:
>>> On 10/12/2010 8:55 AM, Adam Heath wrote:
>>>> On 10/12/2010 10:25 AM, Adrian Crum wrote:
>>>>> Actually, a discussion of database versus filesystem storage of
>>>>> content
>>>>> would be worthwhile. So far there has been some hyperbole, but few
>>>>> facts.
>>>>
>>>> How do you edit database content? What is the procedure? Can a simple
>>>> editor be used? By simple, I mean low-level, like vi.
>>>>
>>>> How do you find all items in your content store that contain a certain
>>>> text word? Can grep and find be used?
>>>>
>>>> How do you handle moving changes between a production server, that is
>>>> being directly managed by the client, and multiple developer
>>>> workstations, which then all have to go first to a staging server? Each
>>>> system in this case has its own set of code changes, and config+data
>>>> changes, that then have to be selectively picked for staging, before
>>>> finally being merged with production.
>>>>
>>>> What about revision control? Can you go back in time to see what the
>>>> code+data looked like? Are there separate revision systems, one for the
>>>> database, and another for the content? And what about the code?
>>>>
>>>> For users/systems that aren't capable of using revision control, is
>>>> there a way for them to mount/browse the content store? Think nfs/samba
>>>> here.
>>>>
>>>> Storing everything directly into the filesystem lets you reuse existing
>>>> tools, that have been perfected over countless generations of
>>>> man-years.
>>>
>>> I believe Jackrabbit has WebDAV and VFS front ends that will accommodate
>>> file system tools. Watch the movie:
>>>
>>> http://www.day.com/day/en/products/crx.html
>>
>> Front end it wrong. It still being the store itself is in some other
>> system(database). The raw store needs to be managed by the filesystem,
>> so standard tools can move it between locations, or do backups, etc.
>> Putting yet another layer just to emulate file access is the wrong way.
>>
>> <brainstorming>
>> Let's make a content management system. Yeah! Let's do it! So, we need
>> to be able to search for content, and mantain links between
>> relationships. Let's write brand new code to do that, and put it in the
>> database.
>>
>> Ok, next, we need to pull the information out of the database, and serve
>> it thru an http server. Oh, damn, it's not running fast. Let's have a
>> cache that resides someplace faster than the database. Oh, I know,
>> memory! Shit, it's using too much memory. Let's put the cache in the
>> filesystem. Updates now remove the cache, and have it get rebuilt. That
>> means read-only access is faster, but updates then have to rebuild tons
>> of stuff.
>>
>> Hmm. We have a designer request to be able to use photoshop to edit
>> images. The server in question is a preview server, is hosted, and not
>> on his immediate network. Let's create a new webdav access method, to
>> make the content look like a filesystem.
>>
>> Our system is getting heavily loaded. Let's have a separate database
>> server, with multiple web frontends. Cool, that works.
>>
>> The system is still heavily loaded, we need a super-huge database server.
>>
>> Crap, still falling over. Time to have multiple read-only databases.
>> </brainstorming>
>>
>> or...
>>
>> <brainstorming>
>> Let's store all our content into the filesystem. That way, things like
>> ExpanDrive(remote ssh fs access for windows) will work for remote hosted
>> machines. Caching isn't a problem anymore, as the raw store is in files.
>> Servers have been doing file sharing for decades, it's a well known
>> problem. Let's have someone else maintain the file sharing code, we'll
>> just use it to support multiple frontends. And, ooh, our designers will
>> be able to use the tools they are familiar with to manipulate things.
>> And, we won't have the extra code running to maintain all the stuff in
>> the multiple databases. Cool, we can even use git, with rebase and
>> merge, to do all sorts of fancy branching and push/pulling between
>> multiple development scenarios.
>> </brainstorming>
>>
>> If the raw store was in the filesystem in the first place, then all this
>> additional layering wouldn't be needed, to make the final output end up
>> looking like a filesystem, which is what was being replaced all along.
>
> Okay. Will webslinger provide a JCR interface?

It could.  Or maybe jackrabbit should have it's filesystem backends
improved(or created).

However, the major problem we have with all those other systems, is
still a big performance issue.  Synchronization sucks for load.
Webslinger doesn't synchronize.  It makes *very* heavy use of
concurrent programming techniques.  The problem arises when certain
api definitions require you to call multiple methods to fetch, then
update.  Such methods are broken, when doing non-blocking algorithms.
  So the fix in those situations is to have a synchronized block.  But
then you have a choke point.

*Any* time you have 2 separate methods, get(or contains), followed by
a put(or remove), you must deal with multiple threads doing the exact
same thing.  You can either synchronize, or alter the later methods
with put(key, old, new) and remove(key, old).  The crux of the
concurrent model is to move mutator type calls into a single method,
that can eventually do something close to CAS.